XGBoost
Tianqi Chen, Carlos Guestrin · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · 2016
Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges. We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. More importantly, we provide insights on cache access patterns, data compression and sharding to build a scalable tree boosting system. By combining these insights, XGBoost scales beyond billions of examples using far fewer resources than existing systems.
Cited by 53,129 works. doi:10.1145/2939672.2939785 Open-access PDF
Authors
References
- Random Forests
- Greedy function approximation: A gradient boosting machine.
- Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)
- Stochastic gradient boosting
- Scikit-learn: Machine Learning in Python
- Space-efficient online computation of quantile summaries
- LIBLINEAR: A Library for Large Linear Classification
- Generalized Boosted Models: A guide to the gbm package
- Practical Lessons from Predicting Clicks on Ads at Facebook
- McRank: Learning to Rank Using Multiple Classification and Gradient Boosting