GLMix: Generalized Linear Mixed Models For Large-Scale Response Prediction
Abstract
Generalized linear model (GLM) is a widely used class of models for statistical inference and response prediction problems. For instance, in order to recommend relevant content to a user or optimize for revenue, many web companies use logistic regression models to predict the probability of the user's clicking on an item (e.g., ad, news article, job). In scenarios where the data is abundant, having a more fine-grained model at the user or item level would potentially lead to more accurate prediction, as the user's personal preferences on items and the item's specific attraction for users can be better captured. One common approach is to introduce ID-level regression coefficients in addition to the global regression coefficients in a GLM setting, and such models are called generalized linear mixed models (GLMix) in the statistical literature. However, for big data sets with a large number of ID-level coefficients, fitting a GLMix model can be computationally challenging. In this paper, we report how we successfully overcame the scalability bottleneck by applying parallelized block coordinate descent under the Bulk Synchronous Parallel (BSP) paradigm. We deployed the model in the LinkedIn job recommender system, and generated 20% to 40% more job applications for job seekers on LinkedIn.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Large longitudinal studies introduce new challenges for data modelling.
“they also introduce new challenges for data modelling.”
From Human-centred artificial intelligence for mobile health sensing: challenges and opportunities (2023) · cites this paper as [23]
Cites
Fitting Linear Mixed-Effects Models Using <b>lme4</b>
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835 · see citing works
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075 · see citing works
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,863 · see citing works
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R. M., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,232 · see citing works
LIBLINEAR: A Library for Large Linear Classification
Fan, R.-E., Chang, K., Hsieh, C., Xiang-rui, W., & Lin, C. (2008) · doi:10.5555/1390681.1442794 · Cited by 6,680 · see citing works
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289 · see citing works
Approximate Inference in Generalized Linear Mixed Models
Breslow, N. E., & Clayton, D. G. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594284 · Cited by 4,595 · see citing works
(2010) · Cited by 4,411 · see citing works
A Bridging Model for Parallel Computation
Valiant, L. G. (1990) · ACM eBooks · doi:10.1145/3749740.3749753 · Cited by 3,697 · see citing works
The Group Lasso for Logistic Regression
Meier, L., van de Geer, S., & Bühlmann, P. (2008) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2007.00627.x · Cited by 1,681 · see citing works
Scaling distributed machine learning with the parameter server
Li, M., Andersen, D. G., Park, J. W., Smola, A. J., Ahmed, A. A., Josifovski, V., Long, J. E., Shekita, E. J., & Su, B.-Y. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685095 · Cited by 1,178 · see citing works
Scandinavian Journal of Statistics
Danish Society for Theoretical Statistics, Tilastoseura, S., & Föreningen, S. (2010) · Scandinavian Journal of Statistics · doi:10.1111/(issn)1467-9469 · Cited by 985 · see citing works
Internet Recommendation Systems
Ansari, A., Essegaier, S., & Kohli, R. (2000) · Journal of Marketing Research · doi:10.1509/jmkr.37.3.363.18779 · Cited by 711 · see citing works
Generalized Linear Mixed Models
McCulloch, C. E., & Neuhaus, J. M. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat07540 · Cited by 467 · see citing works
Bilinear Mixed-Effects Models for Dyadic Data
Hoff, P. D. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000001015 · Cited by 351 · see citing works
MLbase: A Distributed Machine-learning System
Kraska, T., Talwalkar, A., & Duchi, J. C. (2013) · Cited by 310 · see citing works
Communication-efficient algorithms for statistical optimization
Zhang, Y., Duchi, J. C., & Wainwright, M. J. (2012) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426691 · Cited by 307 · see citing works
A Reliable Effective Terascale Linear Learning SystemPreprint
Agarwal, A., Chapelle, O., Dudı́k, M., & Langford, J. C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1110.4198 · Cited by 280 · see citing works
Scalable Coordinate Descent Approaches to Parallel Matrix Factorization for Recommender Systems
Yu, H., Hsieh, C., Si, S., et al. (2012) · Proceedings · doi:10.1109/icdm.2012.168 · Cited by 266 · see citing works
Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models
McDonald, R., Mohri, M., Silberman, N., Walker, D. L., & Mann, G. (2009) · neural information processing systems · Cited by 233 · see citing works
Parallel Coordinate Descent for L1-Regularized Loss MinimizationPreprint
Bradley, J. K., Kyrola, A., Bickson, D., & Guestrin, C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1105.5379 · Cited by 204 · see citing works
Large-scale sparse logistic regression
Liu, J., Chen, J., & Ye, J. (2009) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557082 · Cited by 196 · see citing works
Communication-efficient algorithms for statistical optimization
(2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2567769 · Cited by 195 · see citing works
Estimation for High-Dimensional Linear Mixed-Effects Models Using ℓ1-PenalizationPreprint
van de Geer, S. (2010) · Cited by 149 · see citing works
(2009) · Journal of Marketing Research · doi:10.1509/jmkr.46.2.i · Cited by 149 · see citing works
Xing, E. P., Ho, Q., Dai, W., Kim, J.-K., Wei, J., Lee, S., Zheng, X., Xie, P., Kumar, A., & Yu, Y. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783323 · Cited by 117 · see citing works
Scaling factorization machines to relational data
Rendle, S. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2535573.2488340 · Cited by 101 · see citing works
Agarwal, D., Long, B., Traupman, J., Xin, D., & Zhang, L. (2014) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2556195.2556252 · Cited by 75 · see citing works
Self-citationBlock coordinate descent algorithms for large-scale sparse multiclass classification
Blondel, M., Seki, K., & Uehara, K. (2013) · Machine Learning · doi:10.1007/s10994-013-5367-2 · Cited by 72 · see citing works
Generalized Linear Mixed Models
McCulloch, C. E. (2003) · NSF-CBMS regional conference series in probability and statistics · doi:10.1214/cbms/1462106059 · Cited by 68 · see citing works
Large-scale collaborative prediction using a nonparametric random effects model
Yu, K., Lafferty, J., Zhu, S., et al. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553525 · Cited by 62 · see citing works
Social Network and Click-through Prediction with Factorization Machines
Rendle, S. (2012) · Knowledge Discovery and Data Mining · Cited by 38 · see citing works
A Two-Stage Ensemble of Diverse Models for Advertisement Ranking in KDD Cup 2012
Wu, K.-W., Ferng, C.-S., Ho, C.-H., et al. (2012) · Cited by 22 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Human-centred artificial intelligence for mobile health sensing: challenges and opportunities
Dang, T., Spathis, D., Ghosh, A., & Mascolo, C. (2023) · Royal Society Open Science · doi:10.1098/rsos.230806 · Cited by 30 · see citing works
- Authors
- Zhang, X., Zhou, Y., Ma, Y., Chen, B.-C., Zhang, L., & Agarwal, D.
- Affiliations
- LinkedIn (United States) (US)
- Venue
- Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
- Publisher
- ACM
- Published
- 2016-08-13
- Pages
- 363-372
- OpenAlex
- W2516369484
- Cited by
- 71
- References
- 33
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 4 · 2023: 6 · 2022: 4 · 2021: 16 · 2020: 10 · 2019: 9 · 2018: 6 · 2017: 10
- FWCI
- 7.04
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- generalized linear mixed models, parallel coordinate descent, Bulk Synchronous Parallel model, scalability, job recommendation, LinkedIn, click prediction, large-scale regression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
