The DOI matches its Crossref record; 144 of the 144 references it cites were found in a registry or an index.
Challenges of Big Data analysis
Open accessAbstract
Big Data bring new opportunities to modern society and challenges to data scientists. On the one hand, Big Data hold great promises for discovering subtle population patterns and heterogeneities that are not possible with small-scale data. On the other hand, the massive sample size and high dimensionality of Big Data introduce unique computational and statistical challenges, including scalability and storage bottleneck, noise accumulation, spurious correlation, incidental endogeneity and measurement errors. These challenges are distinguished and require new computational and statistical paradigm. This paper gives overviews on the salient features of Big Data and how these features impact on paradigm change on statistical and computational methods as well as computing architectures. We also provide various new perspectives on the Big Data analysis and computation. In particular, we emphasize on the viability of the sparsest solution in high-confidence set and point out that exogenous assumptions in most statistical methods for Big Data cannot be validated due to incidental endogeneity. They can lead to wrong statistical inferences and consequently wrong scientific conclusions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Overfitting is a key risk in this context, which means that a machine may learn a complete model that also explains the errors, consequently failing to generalize appropriately beyond its training data (Fan, Han, & Liu, 2014).”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Fan, Han, & Liu, 2014
Experimental — accuracy not yet measuredIn the citing paper
“In addition, the combination of datasets in the big data age needs to cope with a number of challenges listed in [20,61], for example efficient data representation, redundancy reduction and data compression, spurious correlations, and noise accumulation.”
From Opening up Smart Cities: Citizen-Centric Challenges and Opportunities from GIScience (2016) · cites this paper as [61]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Donoho, D. L. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.871582 · Cited by 23,994
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Donoho, D. L. (2004) · Cited by 16,703
Gene Expression Omnibus: NCBI gene expression and hybridization array data repository
Edgar, R. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.1.207 · Cited by 13,288
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,917
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
An Introduction To Compressive Sampling
Candes, E. J., & Wakin, M. B. (2008) · IEEE Signal Processing Magazine · doi:10.1109/msp.2007.914731 · Cited by 10,017
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
B, J. F. A., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,919
Ideal Spatial Adaptation by Wavelet Shrinkage
Donoho, D. L., & Johnstone, I. M. (1994) · Biometrika · doi:10.1093/biomet/81.3.425 · Cited by 8,730
Robust principal component analysis?
Candès, E. J., Li, X., Ma, Y., & Wright, J. N. (2011) · Journal of the ACM · doi:10.1145/1970392.1970395 · Cited by 7,048
Sparse MRI: The application of compressed sensing for rapid MR imaging
Lustig, M., Donoho, D., & Pauly, J. M. (2007) · Magnetic Resonance in Medicine · doi:10.1002/mrm.21391 · Cited by 6,952
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,734
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (2001) · SIAM Review · doi:10.1137/s003614450037906x · Cited by 5,030
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.
Daubechies, I., Defrise, M., & De Mol, C. (2004) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20042 · Cited by 4,958
Enhancing Sparsity by Reweighted ℓ 1 Minimization
Candès, E. J., Wakin, M. B., & Boyd, S. (2008) · Journal of Fourier Analysis and Applications · doi:10.1007/s00041-008-9045-x · Cited by 4,878
ANew Look at the Statistical Model Identification
AKAlKE, H. (2011) · Cited by 4,177
Determining the Number of Factors in Approximate Factor Models
Bai, J.-S., & Ng, S. (2000) · Econometrica · doi:10.1111/1468-0262.00273 · Cited by 4,141
Nearly unbiased variable selection under minimax concave penalty
Zhang, C. (2010) · The Annals of Statistics · doi:10.1214/09-aos729 · Cited by 3,909
Figueiredo, M. A. T., Nowak, R. D., & Wright, S. J. (2007) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2007.910281 · Cited by 3,531
The Dantzig selector: Statistical estimation when p is much larger than n
Candès, E. J., & Tao, T. (2007) · The Annals of Statistics · doi:10.1214/009053606000001523 · Cited by 3,202
Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ 1 minimization
Donoho, D. L., & Elad, M. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0437847100 · Cited by 2,970
Forecasting Using Principal Components From a Large Number of Predictors
Stock, J. H., & Watson, M. W. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502388618960 · Cited by 2,946
Di Martino, A., Yan, C.-G., Li, Q., Denio, E., Castellanos, F. X., Alaerts, K., Anderson, J. S., Assaf, M., Bookheimer, S. Y., Dapretto, M., et al. (2014) · Molecular Psychiatry · doi:10.1038/mp.2013.78 · Cited by 2,780
Sure Independence Screening for Ultrahigh Dimensional Feature Space
Fan, J., & Lv, J. (2008) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2008.00674.x · Cited by 2,643
Extensions of Lipschitz mappings into a Hilbert space
Johnson, W. B., & Lindenstrauss, J. (1984) · Contemporary mathematics - American Mathematical Society · doi:10.1090/conm/026/737400 · Cited by 2,632
Statistics for High-Dimensional Data
Bühlmann, P., & van de Geer, S. (2011) · Springer series in statistics · doi:10.1007/978-3-642-20192-9 · Cited by 2,618
Simultaneous analysis of Lasso and Dantzig selector
Bickel, P. J., Ritov, Y., & Tsybakov, A. B. (2009) · The Annals of Statistics · doi:10.1214/08-aos620 · Cited by 2,412
The positive false discovery rate: a Bayesian interpretation and the q-value
Storey, J. D. (2003) · The Annals of Statistics · doi:10.1214/aos/1074290335 · Cited by 2,204
Predicting the future with social media
Asur, S., & Huberman, B. A. (2010) · 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology · doi:10.1109/wi-iat.2010.63 · Cited by 2,081
Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis
Leek, J. T., & Storey, J. D. (2007) · PLoS Genetics · doi:10.1371/journal.pgen.0030161 · Cited by 1,930
Pathwise coordinate optimization
Friedman, J. H., Hastie, T. J., Höfling, H., & Tibshirani, R. J. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas131 · Cited by 1,906
Discrete Cosine Transform: Algorithms, Advantages, Applications
Rao, K. R., Yip, P. C. Y., & Britaňák, V. (1990) · Cited by 1,873
Low, Y., Bickson, D., Gonzalez, J. E., Guestrin, C., Kyrola, A., & Hellerstein, J. M. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2212351.2212354 · Cited by 1,755
Inferential Theory for Factor Models of Large Dimensions
Bai, J. (2003) · Econometrica · doi:10.1111/1468-0262.00392 · Cited by 1,690
Statistics for High-Dimensional Data: Methods, Theory and Applications
Bhlmann, P., & van de Geer, S. (2011) · Cited by 1,527
Database-friendly random projections: Johnson-Lindenstrauss with binary coins
Achlioptas, D. (2003) · Journal of Computer and System Sciences · doi:10.1016/s0022-0000(03)00025-4 · Cited by 1,374
Engle, R. F., Hendry, D. F., & Richard, J.-F. (1983) · Econometrica · doi:10.2307/1911990 · Cited by 1,346
Nonlinear Programming: Analysis and Methods
Avriel, M. (1976) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,339
Nonlinear Programming Analysis and Methods
Rodin, E. Y. (1977) · Computers & Mathematics with Applications · doi:10.1016/0898-1221(77)90071-2 · Cited by 1,300
Personal omics profiling reveals dynamic molecular and medical phenotypes
Chen, R., Mias, G. I., Li-Pook-Than, J., Jiang, L., Lam, H. Y. K., Chen, R., Miriami, E., Karczewski, K. J., Hariharan, M., Dewey, F. E., et al. (2012) · Cell · doi:10.1016/j.cell.2012.02.009 · Cited by 1,291
Covariance regularization by thresholding
Bickel, P. J., & Levina, E. (2008) · The Annals of Statistics · doi:10.1214/08-aos600 · Cited by 1,231
Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems
Nesterov, Y. (2012) · SIAM Journal on Optimization · doi:10.1137/100802001 · Cited by 1,187
Covariance regularization by thresholding
Bickel, P. J., & Levina, E. (2007) · Cited by 1,118
A Survey of Dimension Reduction Techniques
Fodor, I. K. (2002) · doi:10.2172/15002155 · Cited by 1,109
Multiple Rare Alleles Contribute to Low Plasma Levels of HDL Cholesterol
Cohen, J. C., Kiss, R. S., Pertsemlidis, A., Marcel, Y. L., McPherson, R., & Hobbs, H. H. (2004) · Science · doi:10.1126/science.1099870 · Cited by 1,108
Penalized Regressions: The Bridge versus the Lasso
Fu, W. J. (1998) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1998.10474784 · Cited by 1,096
ArrayExpress--a public repository for microarray gene expression data at the EBI
Parkinson, H. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki056 · Cited by 1,077
A Constrained ℓ 1 Minimization Approach to Sparse Precision Matrix Estimation
Cai, T., Liu, W., & Luo, X. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10155 · Cited by 1,000
Mapping authors in intellectual space: A technical overview
McCain, K. W. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6 · Cited by 987
Extensions of compressed sensing
Tsaig, Y., & Donoho, D. L. (2005) · Signal Processing · doi:10.1016/j.sigpro.2005.05.029 · Cited by 958
One-step sparse estimates in nonconcave penalized likelihood modelsPreprint
Zou, H., & Li, R. (2008) · Cited by 915
Large Covariance Estimation by Thresholding Principal Orthogonal Complements
Fan, J., Liao, Y., & Mincheva, M. (2013) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12016 · Cited by 900
Breheny, P. J., & Huang, J. (2011) · The Annals of Applied Statistics · doi:10.1214/10-aoas388 · Cited by 838
The Generalized Dynamic Factor Model
Forni, M., Hallin, M., Lippi, M., & Reichlin, L. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000002050 · Cited by 825
Optimization Transfer Using Surrogate Objective Functions
Lange, K. L., Hunter, D. R., & Yang, I. (2000) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2000.10474858 · Cited by 809
CUR matrix decompositions for improved data analysis
Mahoney, M. W., & Drineas, P. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0803205106 · Cited by 799
ArrayExpress--a public repository for microarray gene expression data at the EBI
Brāzma, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg091 · Cited by 776
Feature Screening via Distance Correlation Learning
Li, R., Zhong, W., & Zhu, L. (2012) · Journal of the American Statistical Association · doi:10.1080/01621459.2012.695654 · Cited by 748
Coordinate descent algorithms for lasso penalized regressionPreprint
Wu, T. T., & Lange, K. L. (2008) · Cited by 740
Database-friendly random projections
Achlioptas, D. (2001) · ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS) · doi:10.1145/375551.375608 · Cited by 712
Worthey, E., Mayer, A. N., Syverson, G., Helbling, D., Bonacci, B. B., Decker, B., Serpe, J. M., Dasu, T., Tschannen, M., Veith, R., et al. (2011) · Genetics in Medicine · doi:10.1097/gim.0b013e3182088158 · Cited by 704
High dimensional covariance matrix estimation using a factor model
B, J. F. A., Fan, Y., & Lv, J. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2008.09.017 · Cited by 688
One-step sparse estimates in nonconcave penalized likelihood models
Zou, H., & Li, R. (2008) · The Annals of Statistics · doi:10.1214/009053607000000802 · Cited by 684
Extensions of Lipschitz mappings into a Hilbert space
Johnson, W. B. (1984) · Medical Entomology and Zoology · Cited by 661
Risk bounds for model selection via penalization
Barron, A. R., Birgé, L., & Massart, P. (1999) · Probability Theory and Related Fields · doi:10.1007/s004400050210 · Cited by 659
Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions
Agarwal, A., Negahban, S., & Wainwright, M. J. (2012) · The Annals of Statistics · doi:10.1214/12-aos1000 · Cited by 654
Sure independence screening in generalized linear models with NP-dimensionality
Fan, J., & Song, R. (2010) · The Annals of Statistics · doi:10.1214/10-aos798 · Cited by 644
Adaptive Thresholding for Sparse Covariance Matrix Estimation
Cai, T., & Liu, W. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm10560 · Cited by 601
Coordinate descent algorithms for lasso penalized regression
Wu, T. T., & Lange, K. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas147 · Cited by 563
Nonparametric Independence Screening in Sparse Ultra-High-Dimensional Additive Models
Fan, J., Feng, Y., & Song, R. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm09779 · Cited by 547
Factor modeling for high-dimensional time series: Inference for the number of factors
Lam, C., & Yao, Q. (2012) · The Annals of Statistics · doi:10.1214/12-aos970 · Cited by 546
Computational Intelligence: Imitating Life
Marks, R. J., Żurada, J. M., & Robinson, C. J. (1994) · IEEE Press eBooks · Cited by 537
High-dimensional semiparametric Gaussian copula graphical models
Liu, H., Han, F., Yuan, M., Lafferty, J., & Wasserman, L. (2012) · The Annals of Statistics · doi:10.1214/12-aos1037 · Cited by 536
Self-citationThe case for cloud computing in genome informatics
Stein, L. D. (2010) · Genome biology · doi:10.1186/gb-2010-11-5-207 · Cited by 531
High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
Loh, P.-L., & Wainwright, M. J. (2012) · The Annals of Statistics · doi:10.1214/12-aos1018 · Cited by 529
Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter
Aramaki, E., Maskawa, S., & Morita, M. (2011) · Cited by 515
Regularization of Wavelet Approximations
Antoniadis, A., & B, J. F. A. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753208942 · Cited by 498
High-dimensional classification using features annealed independence rules
Fan, J., & Fan, Y. (2008) · The Annals of Statistics · doi:10.1214/07-aos504 · Cited by 488
Self-citationUltrahigh dimensional feature selection: beyond the linear model.
Fan, J., Samworth, R. J., & Wu, Y. (2009) · PubMed · Cited by 422
Discoidin domain receptor tyrosine kinases: new players in cancer progression
Valiathan, R. R., Marco, M., Leitinger, B., Kleer, C. G., & Fridman, R. A. (2012) · Cancer and Metastasis Reviews · doi:10.1007/s10555-012-9346-z · Cited by 384
Invited articles section foreword
Vianu, V. (2010) · Journal of the ACM · doi:10.1145/1870103.1870104 · Cited by 382
Variable selection using MM algorithmsPreprint
Hunter, D. R., & Li, R. (2005) · Cited by 375
On the Class of Elliptical Distributions and their Applications to the Theory of Portfolio Choice
Owen, J. S., & Rabinovitch, R. (1983) · The Journal of Finance · doi:10.1111/j.1540-6261.1983.tb02499.x · Cited by 374
Risk Management: Value at Risk and Beyond
Dempster, M. A. H. (2002) · Cited by 344
Variable selection using MM algorithms
Hunter, D. R., & Li, R. (2005) · The Annals of Statistics · doi:10.1214/009053605000000200 · Cited by 339
Robust rank correlation based screening
Li, G., Peng, H., Zhang, J., & Zhu, L. (2012) · The Annals of Statistics · doi:10.1214/12-aos1024 · Cited by 318
A Data-Adaptive Sum Test for Disease Association with Multiple Common or Rare Variants
Han, F., & Pan, W. (2010) · Human Heredity · doi:10.1159/000288704 · Cited by 306
A Direct Estimation Approach to Sparse Linear Discriminant Analysis
Cai, T., & Liu, W. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm11199 · Cited by 268
Variance Estimation Using Refitted Cross-Validation in Ultrahigh Dimensional Regression
Fan, J., Guo, S., & Hao, N. (2011) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2011.01005.x · Cited by 264
Self-citationRegularized rank-based estimation of high-dimensional nonparanormal graphical models
Xue, L., & Zou, H. (2012) · The Annals of Statistics · doi:10.1214/12-aos1041 · Cited by 256
Variable Selection in Finite Mixture of Regression Models
Khalili, A., & Chen, J. (2007) · Journal of the American Statistical Association · doi:10.1198/016214507000000590 · Cited by 241
Principled sure independence screening for Cox models with ultra-high-dimensional covariates
Zhao, S. D., & Li, Y. (2011) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2011.08.002 · Cited by 239
Wavelets in statistics: A review
Antoniadis, A. (1997) · Statistical Methods & Applications · doi:10.1007/bf03178905 · Cited by 227
Estimating False Discovery Proportion Under Arbitrary Covariance Dependence
Fan, J., Han, X., & Gu, W. (2012) · Journal of the American Statistical Association · doi:10.1080/01621459.2012.720478 · Cited by 227
Ultrahigh Dimensional Feature Selection: Beyond The Linear Model
FanJianqing, SamworthRichard, & WuYichao (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755853 · Cited by 216
Parallel Coordinate Descent for L1-Regularized Loss MinimizationPreprint
Bradley, J. K., Kyrola, A., Bickson, D., & Guestrin, C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1105.5379 · Cited by 204
Fast global convergence of gradient methods for high-dimensional statistical recovery
Agarwal, A., Negahban, S. N., & Wainwright, M. J. (2012) · The Annals of Statistics · doi:10.1214/12-aos1032 · Cited by 204
ℓ1-penalization for mixture regression models
Städler, N., Bühlmann, P., & van de Geer, S. (2010) · Test · doi:10.1007/s11749-010-0197-z · Cited by 201
ℓ1-Penalization for Mixture Regression ModelsPreprint
van de Geer, S. (2016) · Cited by 186
Using Generalized Correlation to Effect Variable Selection in Very High Dimensional Problems
Hall, P., & Miller, H. (2009) · Journal of Computational and Graphical Statistics · doi:10.1198/jcgs.2009.08041 · Cited by 181
Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optimaPreprint
Loh, P., & Wainwright, M. J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.2436 · Cited by 169
High‐Dimensional Covariance Estimation
Pourahmadi, M. (2013) · Wiley series in probability and statistics · doi:10.1002/9781118573617 · Cited by 167
Large Vector Auto RegressionsPreprint
Song, S., & Bickel, P. J. (2011) · RePEc: Research Papers in Economics · doi:10.18452/4336 · Cited by 144
Correlatedz-Values and the Accuracy of Large-Scale Statistical Estimates
Efron, B. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2010.tm09129 · Cited by 136
High-dimensional instrumental variables regression and confidence setsPreprint
Gautier, É., & Rose, C. D. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1105.2454 · Cited by 126
Oracle inequalities for the lasso in the Cox model
Huang, J., Sun, T., Ying, Z., Yu, Y., & Zhang, C.-H. (2013) · The Annals of Statistics · doi:10.1214/13-aos1098 · Cited by 107
A comparison of the lasso and marginal regression
Genovese, C. R., Jin, J., Wasserman, L., & Yao, Z. (2012) · doi:10.5555/2188385.2343712 · Cited by 105
Parallel Coordinate Descent for L1-Regularized Loss Minimization
Kyrola, A., Bickson, D., Guestrin, C., & Bradley, J. K. (2011) · International Conference on Machine Learning · Cited by 99
Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
Wang, Z., Liu, H., & Zhang, T. (2014) · The Annals of Statistics · doi:10.1214/14-aos1238 · Cited by 88
Self-citationGradient Directed Regularization for Linear Regression and Classi…cation
FriedmanBogdan, J. H., & Popescu, E. (2004) · Cited by 88
Phase transition in limiting distributions of coherence of high-dimensional random matrices
Cai, T., & Jiang, T. (2011) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2011.11.008 · Cited by 76
Transelliptical Graphical Models
Liu, H., Han, F., & Zhang, C. (2012) · Cited by 69
Self-citationHall, P., Pittelkow, Y., & Ghosh, M. (2008) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2007.00631.x · Cited by 62
Empirical null and false discovery rate analysis in neuroimaging
Schwartzman, A., Dougherty, R. F., Lee, J., Ghahremani, D. G., & Taylor, J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.04.182 · Cited by 60
In Search of Non-Gaussian Components of a High-Dimensional DistributionPreprint
Blanchard, G., Kawanabe, M., Sugiyama, M., Spokoiny, V. G., & Müller, K. (2006) · RePEc: Research Papers in Economics · doi:10.18452/3970 · Cited by 60
(2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511615337 · Cited by 58
Large Vector Auto RegressionsPreprint
Song, S., & Bickel, P. J. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1106.3915 · Cited by 55
Detecting outliers in high-dimensional neuroimaging datasets with robust covariance estimators
Fritsch, V., Varoquaux, G. P., Thyreau, B., Poline, J., & Thirion, B. (2012) · Medical Image Analysis · doi:10.1016/j.media.2012.05.002 · Cited by 53
Scale-Invariant Sparse PCA on High-Dimensional Meta-Elliptical Data
Han, F., & Liu, H. (2013) · Journal of the American Statistical Association · doi:10.1080/01621459.2013.844699 · Cited by 53
Self-citationTransition Matrix Estimation in High Dimensional Time Series
Han, F., & Liu, H. (2013) · Cited by 47
Self-citationCovariate assisted screening and estimation
Ke, Z. T., Jin, J., & Fan, J. (2014) · The Annals of Statistics · doi:10.1214/14-aos1243 · Cited by 43
Self-citationAn overview of recent developments in genomics and associated statistical methods
Bickel, P. J., Brown, J. B., Huang, H., & Li, Q. (2009) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2009.0164 · Cited by 41
Posterior consistency of nonparametric conditional moment restricted models
Liao, Y., & Jiang, W. (2011) · The Annals of Statistics · doi:10.1214/11-aos930 · Cited by 30
Features of Big Data and sparsest solution in high confidence set
Fan, J. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b16720-50 · Cited by 20
Self-citationWhat has Functional Neuroimaging told us about the Mind? So Many Examples, So Little Space
Jonides, J., Nee, D. E., & Berman, M. G. (2006) · Cortex · doi:10.1016/s0010-9452(08)70372-1 · Cited by 18
Principal Component Analysis on non-Gaussian Dependent Data
Han, F., & Liu, H. (2013) · Cited by 14
Self-citationControl of the False Discovery Rate Under Arbitrary Covariance DependencePreprint
Han, X., Gu, W., & Fan, J. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.4397 · Cited by 14
Self-citationEndogeneity in Ultrahigh DimensionPreprint
Fan, J., & Liao, Y. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2045864 · Cited by 13
Self-citationTranselliptical Component Analysis
Han, F., & Liu, H. (2012) · neural information processing systems · Cited by 12
Self-citationWould the field of cognitive neuroscience be advanced by sharing functional MRI data?
Visscher, K., & Weissman, D. H. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-34 · Cited by 12
Factor Modeling for High Dimensional Time Series
Lam, C., Yao, Q., & Bathia, N. (2011) · Contributions to statistics · doi:10.1007/978-3-7908-2736-1_31 · Cited by 11
An overview of recent developments in genomics and associated statistical methods
Eter, B. Y. P., Ickel, J. B., Ames, J., Rown, B. B., & Uang, H. A. H. (2015) · Cited by 4
Hallin, M. (2013) · Data Archiving and Networked Services (DANS) · Cited by 3
Research works printed but not linked: 6
- Donoho, High-dimensional data analysis: the curses and blessings of dimensionality
- Bickel, 2008, Discussion on the paper ‘Sure independence screening for ultrahigh dimensional feature space’ by Fan and Lv, J Roy Stat Soc B, 70, 883
- Friedman, 2003, Gradient directed regularization for linear regression and classification, Technical report
- Milham, 2011, The International Neuroimaging Data-sharing Initiative (INDI) and the Functional Connectomes Project, 17th Annual Meeting of the Organization for Human Brain Mapping
- 2012, The ADHD-200 Consortium. The ADHD-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience, Front Syst Neurosci, 6, 62
- Cochrane, 2001, Asset Pricing
Cited by
Sorted by citations
Beyond the hype: Big data concepts, methods, and analytics
Gandomi, A., & Haider, M. (2015) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2014.10.007 · Cited by 4,226
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning
Sun, Y., Babu, P., & Palomar, D. P. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2601299 · Cited by 1,820
Wang, G., Gunasekaran, A., Ngai, E. W. T., & Papadopoulos, T. (2016) · International Journal of Production Economics · doi:10.1016/j.ijpe.2016.03.014 · Cited by 1,545
Global ecosystem thresholds driven by aridity
Berdugo, M., Delgado‐Baquerizo, M., Soliveres, S., Hernández‐Clemente, R., Zhao, Y., Gaitán, J., Gross, N., Sáiz, H., Maire, V., Lehmann, A., et al. (2020) · Science · doi:10.1126/science.aay5958 · Cited by 1,265
Metadata
- Authors
- Fan, J., Han, F., & Liu, H.
- Affiliations
- Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ 08544, USA;
- Johns Hopkins University (US)
- Princeton University (US)
- Venue
- National Science Review
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-02-05
- Volume
- 1
- Issue
- 2
- Pages
- 293-314
- ISSN
- 2053-714X (ISSN-L)
- 2095-5138
- PMC
- PMC4236847
- OpenAlex
- W2114060717
- Cited by
- 1,482
- References
- 144
- Citations by year
- 2026: 86 · 2025: 132 · 2024: 142 · 2023: 172 · 2022: 118 · 2021: 122 · 2020: 125 · 2019: 157 · 2018: 162 · 2017: 120 · 2016: 90 · 2015: 40 · 2014: 15
- FWCI
- 165.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- National Science Review (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (R01-GM072611, 1206464, 1332109, 1116730)
- National Institutes of Health
- National Institute of Mental Health (R01MH102339)
- National Institute of General Medical Sciences (R01GM072611, R01GM100474)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- big data analytics, high-dimensional data, scalability, noise accumulation, spurious correlations, measurement error, statistical inference, computer architecture
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
