The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Identifying Latent Structures in Panel Data
Abstract
This paper provides a novel mechanism for identifying and estimating latent group structures in panel data using penalized techniques. We consider both linear and nonlinear models where the regression coefficients are heterogeneous across groups but homogeneous within a group and the group membership is unknown. Two approaches are considered—penalized profile likelihood (PPL) estimation for the general nonlinear models without endogenous regressors, and penalized GMM (PGMM) estimation for linear models with endogeneity. In both cases, we develop a new variant of Lasso called classifier‐Lasso (C‐Lasso) that serves to shrink individual coefficients to the unknown group‐specific coefficients. C‐Lasso achieves simultaneous classification and consistent estimation in a single step and the classification exhibits the desirable property of uniform consistency. For PPL estimation, C‐Lasso also achieves the oracle property so that group‐specific parameter estimators are asymptotically equivalent to infeasible estimators that use individual group identity information. For PGMM estimation, the oracle property of C‐Lasso is preserved in some special cases. Simulations demonstrate good finite‐sample performance of the approach in both classification and estimation. Empirical applications to both linear and nonlinear models are presented.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Arellano, M., & Bond, S. (1991) · The Review of Economic Studies · doi:10.2307/2297968 · Cited by 33,272
Testing for unit roots in heterogeneous panels
Im, K. S., Pesaran, M. H., & Shin, Y. (2003) · Journal of Econometrics · doi:10.1016/s0304-4076(03)00092-7 · Cited by 15,435
Unit root tests in panel data: asymptotic and finite-sample properties
Levin, A., Lin, C.-F., & James Chu, C.-S. (2002) · Journal of Econometrics · doi:10.1016/s0304-4076(01)00098-7 · Cited by 12,849
Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,726
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Fan, J., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,900
Hsiao, C. (2014) · doi:10.1017/cbo9781139839327 · Cited by 7,869
Hsiao, C. (2022) · doi:10.1017/9781009057745 · Cited by 7,869
The adaptive lasso and its oracle properties
Zou, H. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000735 · Cited by 7,470
Ethnicity, Insurgency, and Civil War
FEARON, J. D., & LAITIN, D. D. (2003) · American Political Science Review · doi:10.1017/s0003055403000534 · Cited by 7,366
Model Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261
Pooled Mean Group Estimation of Dynamic Heterogeneous Panels
Pesaran, M. H., Shin, Y., & Smith, R. P. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10474156 · Cited by 6,683
Greed and grievance in civil war
Collier, P. (2004) · Oxford Economic Papers · doi:10.1093/oep/gpf064 · Cited by 6,101
Financing Constraints and Corporate Investment
Fazzari, S. M., Hubbard, R. G., Petersen, B. C., Blinder, A. S., & Poterba, J. M. (1988) · Brookings Papers on Economic Activity · doi:10.2307/2534426 · Cited by 5,574
Estimating long-run relationships from dynamic heterogeneous panels
Pesaran, M. H., & Smith, R. (1995) · Journal of Econometrics · doi:10.1016/0304-4076(94)01644-f · Cited by 5,479 · cited 1×
“…to estimate the equations separately for each individual, and whether to rely on the average response from individual time series regressions — see Pesaran and Smith (1995), Baltagi and Griffin (1997), Hsiao, Pesaran, and Tahmiscioglu (1999), Pesaran, Shin and Smith (1999), and the survey by…”
BackgroundSparsity and Smoothness Via the Fused Lasso
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., & Knight, K. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00490.x · Cited by 2,724
Martingale Limit Theory and its Application.
Loynes, R. M., Hall, P., & Heyde, C. C. (1984) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981599 · Cited by 2,685
US Food Aid and Civil Conflict
Nunn, N., & Qian, N. (2014) · American Economic Review · doi:10.1257/aer.104.6.1630 · Cited by 2,032
Consumption and Liquidity Constraints: An Empirical Investigation
Zeldes, S. P. (1989) · Journal of Political Economy · doi:10.1086/261605 · Cited by 1,931
On bias, inconsistency, and efficiency of various estimators in dynamic panel data models
Kiviet, J. F. (1995) · Journal of Econometrics · doi:10.1016/0304-4076(94)01643-e · Cited by 1,926
Panel Data Models With Interactive Fixed Effects
(2009) · Econometrica · doi:10.3982/ecta6135 · Cited by 1,690
Blattman, C., & Miguel, E. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.1.3 · Cited by 1,617
Transition Modeling and Econometric Convergence Tests
Phillips, P. C. B., & Sul, D. (2007) · Econometrica · doi:10.1111/j.1468-0262.2007.00811.x · Cited by 1,451
Self-citationAsymptotic Theory for Econometricians.
Robinson, P. M., & White, H. (1988) · Economica · doi:10.2307/2554258 · Cited by 1,305
Asymptotics for lasso-type estimators
Fu, W., & Knight, K. (2000) · The Annals of Statistics · doi:10.1214/aos/1015957397 · Cited by 1,276
State Capacity, Conflict, and Development
(2010) · Econometrica · doi:10.3982/ecta8073 · Cited by 850
Tuning parameter selectors for the smoothly clipped absolute deviation method
Wang, H., Li, R., & Tsai, C.-L. (2007) · Biometrika · doi:10.1093/biomet/asm053 · Cited by 768
Least Squares After Model Selection in High-Dimensional Sparse Models
Belloni, A., & Chernozhukov, V. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1582594 · Cited by 683
Unit Roots and Cointegration in Panels
Breitung, J., & Pesaran, M. H. (2008) · Advanced Studies in Theoretical and Applied Econometrics · doi:10.1007/978-3-540-75892-1_9 · Cited by 670
The Time Series and Cross-Section Asymptotics of Dynamic Panel Data Estimators
Alvarez, J., & Arellano, M. (2003) · Econometrica · doi:10.1111/1468-0262.00441 · Cited by 637
Capital Flows to Developing Economies: Implications for Saving and Investment
Bosworth, B. P., Collins, S. M., & Reinhart, C. M. (1999) · Brookings Papers on Economic Activity · doi:10.2307/2534664 · Cited by 580
Saving and growth: a reinterpretation
Carroll, C. D., & Weil, D. N. (1994) · Carnegie-Rochester Conference Series on Public Policy · doi:10.1016/0167-2231(94)90006-x · Cited by 545
Ethnicity and Conflict: An Empirical Study
Esteban, J., Mayoral, L., & Ray, D. (2012) · American Economic Review · doi:10.1257/aer.102.4.1310 · Cited by 533
ECONOMETRICS FOR GRUMBLERS: A NEW LOOK AT THE LITERATURE ON CROSS-COUNTRY GROWTH EMPIRICS
Eberhardt, M., & Teal, F. (2011) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2010.00624.x · Cited by 479
Hahn, J., & Kuersteiner, G. (2002) · Econometrica · doi:10.1111/1468-0262.00344 · Cited by 477
Jackknife and Analytical Bias Reduction for Nonlinear Panel Models
Hahn, J., & Newey, W. (2004) · Econometrica · doi:10.1111/j.1468-0262.2004.00533.x · Cited by 452
Grouped Patterns of Heterogeneity in Panel Data
Bonhomme, S., & Manresa, E. (2015) · Econometrica · doi:10.3982/ecta11319 · Cited by 451
Split-panel Jackknife Estimation of Fixed-effect Models
Dhaene, G., & Jochmans, K. (2015) · The Review of Economic Studies · doi:10.1093/restud/rdv007 · Cited by 417
Saving in Developing Countries: Theory and Review
Deaton, A. (1989) · The World Bank Economic Review · doi:10.1093/wber/3.suppl_1.61 · Cited by 391
Why are Latin America's savings rates so low? An international comparative analysis
Edwards, S. (1996) · Journal of Development Economics · doi:10.1016/s0304-3878(96)00424-5 · Cited by 361
Pooled estimators vs. their heterogeneous counterparts in the context of dynamic demand for gasoline
Baltagi, B. H., & Griffin, J. M. (1997) · Journal of Econometrics · doi:10.1016/s0304-4076(96)01802-7 · Cited by 349
Multiple Change-Point Estimation With a Total Variation Penalty
Harchaoui, Z., & Lévy-Leduc, C. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2010.tm09181 · Cited by 311
Maddala, G. S., Trost, R. P., Li, H., & Joutz, F. (1997) · Journal of Business & Economic Statistics · doi:10.1080/07350015.1997.10524691 · Cited by 282
Saving in Developing Countries: An Overview
Loayza, N., Schmidt-Hebbel, K., & Serven, L. (2000) · The World Bank Economic Review · doi:10.1093/wber/14.3.393 · Cited by 271
Bias in dynamic panel estimation with fixed effects, incidental trends and cross section dependence
Phillips, P. C. B., & Sul, D. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2006.03.009 · Cited by 266
Self-citationNonparametric Identification of Finite Mixture Models of Dynamic Discrete Choices
(2009) · Econometrica · doi:10.3982/ecta6763 · Cited by 252
Sparse estimators and the oracle property, or the return of Hodges’ estimator
Leeb, H., & Pötscher, B. M. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.017 · Cited by 229
Bayes estimation of short-run coefficients in dynamic panel data models
Hsiao, C., Pesaran, M. H., & Tahmiscioglu, A. K. (1999) · Analysis of Panels and Limited Dependent Variable Models · doi:10.1017/cbo9780511493140.013 · Cited by 215
BIAS REDUCTION FOR DYNAMIC NONLINEAR PANEL MODELS WITH FIXED EFFECTS
Hahn, J., & Kuersteiner, G. (2011) · Econometric Theory · doi:10.1017/s0266466611000028 · Cited by 210
A Bernstein type inequality and moderate deviations for weakly dependent sequences
Merlevède, F., Peligrad, M., & Rio, E. (2011) · Probability Theory and Related Fields · doi:10.1007/s00440-010-0304-9 · Cited by 178
Modelling Income Processes with Lots of Heterogeneity
BROWNING, M., EJRNAES, M., & ALVAREZ, J. (2010) · Review of Economic Studies · doi:10.1111/j.1467-937x.2010.00612.x · Cited by 167
On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding
Pötscher, B. M., & Leeb, H. (2009) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2009.06.010 · Cited by 166
International differences in social security and saving
Feldstein, M. S. (1980) · Journal of Public Economics · doi:10.1016/0047-2727(80)90041-9 · Cited by 143
Panel Data Models with Grouped Factor Structure Under Unknown Group Membership
Ando, T., & Bai, J. (2015) · Journal of Applied Econometrics · doi:10.1002/jae.2467 · Cited by 135
TESTING HOMOGENEITY IN PANEL DATA MODELS WITH INTERACTIVE FIXED EFFECTS
Su, L., & Chen, Q. (2013) · Econometric Theory · doi:10.1017/s0266466613000017 · Cited by 124
Self-citationEstimation of Panel Data Models with Parameter Heterogeneity when Group Membership is Unknown
Lin, C., & Ng, S. (2012) · Journal of Econometric Methods · doi:10.1515/2156-6674.1000 · Cited by 123
Random Coefficient Panel Data ModelsPreprint
Hsiao, C., & Pesaran, M. H. (2004) · SSRN Electronic Journal · doi:10.2139/ssrn.572783 · Cited by 122
Indirect inference for dynamic panel models
Gouriéroux, C., Phillips, P. C. B., & Yu, J. (2009) · Journal of Econometrics · doi:10.1016/j.jeconom.2009.10.024 · Cited by 122
Self-citationConditional independence, conditional mixing and conditional association
Рао, Б. Л. С. П. (2007) · Annals of the Institute of Statistical Mathematics · doi:10.1007/s10463-007-0152-2 · Cited by 116
A Panel Analysis of Liquidity Constraints and Firm Investment
Hsiao, C., & Tahmiscioglu, A. K. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10473997 · Cited by 113
A NEW PANEL DATA TREATMENT FOR HETEROGENEITY IN TIME TRENDS
Kneip, A., Sickles, R. C., & Song, W. (2012) · Econometric Theory · doi:10.1017/s026646661100034x · Cited by 113
Poverty and Civil War: Revisiting the Evidence
Djankov, S., & Reynal‐Querol, M. (2010) · The Review of Economics and Statistics · doi:10.1162/rest_a_00046 · Cited by 103
ADAPTIVE GMM SHRINKAGE ESTIMATION WITH CONSISTENT MOMENT SELECTION
Liao, Z. (2012) · Econometric Theory · doi:10.1017/s0266466612000783 · Cited by 101
Shrinkage estimation of dynamic panel data models with interactive fixed effects
Lu, X., & Su, L. (2015) · Journal of Econometrics · doi:10.1016/j.jeconom.2015.09.005 · Cited by 92
Self-citationSHRINKAGE ESTIMATION OF REGRESSION MODELS WITH MULTIPLE STRUCTURAL CHANGES
Qian, J., & Su, L. (2015) · Econometric Theory · doi:10.1017/s0266466615000237 · Cited by 75
Self-citationAsset Pricing with a General Multifactor Structure
Ando, T., & Bai, J. (2014) · Journal of Financial Econometrics · doi:10.1093/jjfinec/nbu026 · Cited by 74
X-DIFFERENCING AND DYNAMIC PANEL MODEL ESTIMATION
Han, C., Phillips, P. C. B., & Sul, D. (2013) · Econometric Theory · doi:10.1017/s0266466613000170 · Cited by 73
Self-citationGrouped effects estimators in fixed effects models
Bester, C. A., & Hansen, C. (2013) · Journal of Econometrics · doi:10.1016/j.jeconom.2012.08.022 · Cited by 72
Baltagi, B. H., Bresson, G., & Pirotte, A. (2008) · Springer eBooks · doi:10.1007/978-3-540-75892-1_16 · Cited by 68
Heterogeneity and Microeconometrics Modeling
Browning, M., & Carro, J. M. (2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511607547.004 · Cited by 63
Testing for Common Trends in Semiparametric Panel Data Models with Fixed Effects
Zhang, Y., Su, L., & Phillips, P. C. B. (2011) · Econometrics Journal · doi:10.1111/j.1368-423x.2011.00361.x · Cited by 52
Self-citationModel selection in the presence of incidental parameters
Lee, Y., & Phillips, P. C. B. (2015) · Journal of Econometrics · doi:10.1016/j.jeconom.2015.03.012 · Cited by 31
Self-citation
Cited by
Visualization, Identification, and Estimation in the Linear Panel Event-Study Design
Freyaldenhoven, S., Hansen, C., Pérez Pérez, J., & Shapiro, J. (2021) · doi:10.3386/w29170 · Cited by 116
Metadata
- Authors
- Su, L., Shi, Z., & Phillips, P. C. B.
- Venue
- Econometrica
- Publisher
- The Econometric Society
- Published
- 2016
- Volume
- 84
- Issue
- 6
- Pages
- 2215-2264
- ISSN
- 0012-9682 (ISSN-L)
- OpenAlex
- W3121461680
- Cited by
- 265
- References
- 73
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.