This notebook contains codes showing how to build a predictive classification model integrating XGBoost BayesOptimization in R Programming. XGBoost, eXtreme Gradient Boosting, is a powerful technique for building predictive models from complex data. It is popular for its fast and accurate results.
Bayes Optimization is a strategy for selecting the best parameters for a model. Instead of utilising an exhaustive combination through grid search, this strategy uses Bayesian statistics to predict which combination of hyperparameters might lead to better performance. This method reduces the time and resources used to build the model.
Together, using XGBoost with Bayes Optimization in R allows for efficient and effective modelling, helping to pinpoint the optimal settings for the best predictive performance. This notebook provides a demonstration on how to integrate Bayes Optimization into model selection. However, it does not cover extensive feature engineering and feature selection, which are crucial for building robust models.
-
A Guide on XGBoost hyperparameters tuning: https://www.kaggle.com/code/prashant111/a-guide-on-xgboost-hyperparameters-tuning
-
Understand your dataset with XGBoost: https://cran.r-project.org/web/packages/xgboost/vignettes/discoverYourData.html
-
Kaggle Grandmaster Guide: https://forecastegy.com/posts/xgboost-hyperparameter-tuning-with-optuna/
-
R Documents: https://cran.r-project.org/web/packages/rBayesianOptimization/rBayesianOptimization.pdf
-
Math behind Bayes Optimization: https://sassafras13.github.io/BayesianOptimization/