Making the most of statistical analyses: improving interpretation and presentation
Abstract
W e show that social scientists often do not take full advantage of the information available in their statistical results and thus miss opportunities to present quantities that could shed the greatest light on their research questions. In this article we suggest an approach, built on the technique of statistical simulation, to extract the currently overlooked information and present it in a reader-friendly manner. More specifically, we show how to convert the raw results of any statistical procedure into expressions that (1) convey numerically precise estimates of the quantities of greatest substantive interest, (2) include reasonable measures of uncertainty about those estimates, and (3) require little specialized knowledge to understand. The following simple statement satisfies our criteria: “Other things being equal, an additional year of education would increase your annual income by $1,500 on average, plus or minus about $500.” Any smart high school student would understand that sentence, no matter how sophisticated the statistical model and powerful the computers used to produce it. The sentence is substantively informative because it conveys a key quantity of interest in terms the reader wants to know. At the same time, the sentence indicates how uncertain the researcher is about the estimated quantity of interest. Inferences are never certain, so any honest presentation of statistical results must include some qualifier, such as “plus or minus $500” in the present example. Making the Most of Statistical Analyses: Improving Interpretation and Presentation
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Simulation analyses produced the uncertainty estimates for relative risks.
“Uncertainty in the RR was generated by simulation analyses.152”
Cites
Regression Models for Categorical and Limited Dependent Variables.
MTW, & Long, J. S. (1997) · Journal of the American Statistical Association · doi:10.2307/2965458 · Cited by 6,889 · see citing works
Logistic Regression in Rare Events Data
King, G., & Zeng, L. (2001) · Political Analysis · doi:10.1093/oxfordjournals.pan.a004868 · Cited by 4,316 · see citing works
Self-citationMobilization, participation, and democracy in America
Rosenstone, S. J., & Hansen, J. M. (1993) · Medical Entomology and Zoology · Cited by 3,644 · see citing works
Bayes and Empirical Bayes Methods for Data Analysis
Carlin, B. P., & Louis, T. A. (2001) · Technometrics · doi:10.1198/tech.2001.s608 · Cited by 2,110 · see citing works
Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review
Cowles, M. K., & Carlin, B. P. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476956 · Cited by 1,998 · see citing works
Partisan Politics in the Global Economy
Garrett, G. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9780511625633 · Cited by 1,891 · see citing works
Bootstrapping: A Nonparametric Approach to Statistical Inference.
Aa, Mooney, C. Z., & Duval, R. D. (1994) · Journal of the American Statistical Association · doi:10.2307/2290969 · Cited by 1,368 · see citing works
Unifying Political Methodology: The Likelihood Theory of Statistical Inference
King, G. (1989) · Project Muse (Johns Hopkins University) · Cited by 714 · see citing works
Self-citationMarkov Chain Monte Carlo in Practice: A Roundtable Discussion
Kass, R. E., Carlin, B. P., Gelman, A., & Neal, R. M. (1998) · The American Statistician · doi:10.1080/00031305.1998.10480547 · Cited by 705 · see citing works
de Mesquita, B. B., & Siverson, R. M. (1995) · American Political Science Review · doi:10.2307/2082512 · Cited by 662 · see citing works
MTW, & Tanner, M. A. (1997) · Journal of the American Statistical Association · doi:10.2307/2965746 · Cited by 615 · see citing works
Computer Intensive Methods for Testing Hypotheses.
Hollas, E. A., & Noreen, E. W. (1990) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348203 · Cited by 412 · see citing works
A Statistical Model for Multiparty Electoral Data
Katz, J. N., & King, G. (1999) · American Political Science Review · doi:10.2307/2585758 · Cited by 297 · see citing works
Self-citation(1997) · RePEc: Research Papers in Economics · Cited by 256 · see citing works
Comparative Statistical Inference
Bonis, A. J., & Barnett, V. (1976) · Technometrics · doi:10.2307/1267925 · Cited by 245 · see citing works
Democratizing Mexico: public opinion and electoral choices
(1996) · Choice Reviews Online · doi:10.5860/choice.34-1169 · Cited by 241 · see citing works
Resampling: The new statistics
Simon, J. L. (1992) · Cited by 234 · see citing works
Data Analysis for Politics and Policy
Tufte, E. R. (1974) · Medical Entomology and Zoology · Cited by 218 · see citing works
Path Dependence, Competition, and Succession in the Dynamics of Scientific Revolution
Sterman, J. D., & Wittenberg, J. (1999) · Organization Science · doi:10.1287/orsc.10.3.322 · Cited by 165 · see citing works
Self-citationCausal Inferences, Closed Populations, and Measures of Association
Blalock, H. M. (1967) · American Political Science Review · doi:10.2307/1953880 · Cited by 150 · see citing works
Electoral Dynamics in Britain since 1918
Miller, W. L. (1977) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-15851-5 · Cited by 138 · see citing works
Problems in Making Policy Inferences from the Coleman Report
Cain, G. G., & Watts, H. W. (1970) · American Sociological Review · doi:10.2307/2093201 · Cited by 134 · see citing works
Estimating the Expected Predictive Accuracy of Econometric Models
Fair, R. C. (1980) · International Economic Review · doi:10.2307/2526186 · Cited by 124 · see citing works
Bootstrap Statistical Inference: Examples and Evaluations for Political Science
Mooney, C. Z. (1996) · American Journal of Political Science · doi:10.2307/2111639 · Cited by 75 · see citing works
Comparative Statistical Inference (2nd ed.).
Brockett, P. L., & Barnett, V. (1983) · Journal of the American Statistical Association · doi:10.2307/2288682 · Cited by 57 · see citing works
Electoral Surprise and the Midterm Loss in US Congressional Elections
Scheve, K. F., & Tomz, M. R. (1999) · British Journal of Political Science · doi:10.1017/s000712349900023x · Cited by 41 · see citing works
Self-citationProbability and Statistics: Experimental Results of a Radically Different Teaching Method
Simon, J. L., Atkinson, D. T., & Shevokas, C. (1976) · American Mathematical Monthly · doi:10.1080/00029890.1976.11994231 · Cited by 34 · see citing works
Bootstrapping: A Nonparametric Approach to Statistical Inference.
Liao, T. F., Mooney, C. Z., Duval, R. D., et al. (1995) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2076915 · Cited by 24 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Lim, S. S., Vos, T., Flaxman, A. D., Danaei, G., Shibuya, K., Adair-Rohani, H., AlMazroa, M. A., Amann, M., Anderson, H. R., Andrews, K. G., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61766-8 · Cited by 11,805 · see citing works
Murray, C. J. L., Vos, T., Lozano, R., Naghavi, M., Flaxman, A. D., Michaud, C., Ezzati, M., Shibuya, K., Salomon, J. A., Abdalla, S., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61689-4 · Cited by 8,888 · see citing works
The Spread of Obesity in a Large Social Network over 32 Years
Christakis, N. A., & Fowler, J. H. (2007) · New England Journal of Medicine · doi:10.1056/nejmsa066082 · Cited by 5,007 · see citing works
A general approach to causal mediation analysis.
Imai, K., Keele, L., & Tingley, D. (2010) · Psychological Methods · doi:10.1037/a0020761 · Cited by 4,143 · see citing works
The collective dynamics of smoking in a large social network
Christakis, N. A., & Fowler, J. H. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa0706154 · Cited by 2,390 · see citing works
Fowler, J. H., & Christakis, N. A. (2008) · BMJ · doi:10.1136/bmj.a2338 · Cited by 1,990 · see citing works
Cultural cognition of scientific consensus.
Kahan, D. M., Jenkins‐Smith, H., & Braman, D. (2011) · Journal of Risk Research · doi:10.1080/13669877.2010.511246 · Cited by 1,686 · see citing works
Social contagion theory: examining dynamic social networks and human behavior
Christakis, N. A., & Fowler, J. H. (2013) · Statistics in Medicine · doi:10.1002/sim.5408 · Cited by 1,303 · see citing works
What to do about missing values in time series cross- section data
Honaker, J., & King, G. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2010.00447.x · Cited by 966 · see citing works
Self-citationMotivated numeracy and enlightened self-government
KAHAN, D. M., PETERS, E., DAWSON, E. C., & SLOVIC, P. (2017) · Behavioural Public Policy · doi:10.1017/bpp.2016.2 · Cited by 883 · see citing works
Health metrics and evaluation: strengthening the science
Murray, C. J., & Frenk, J. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60526-7 · Cited by 849 · see citing works
Wang, H., Dwyer-Lindgren, L., Lofgren, K. T., Rajaratnam, J. K., Marcus, J. R., Levin-Rector, A., Levitz, C. E., Lopez, A. D., & Murray, C. J. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61719-x · Cited by 810 · see citing works
Does Conflict Beget Conflict? Explaining Recurring Civil War
Walter, B. F. (2004) · Journal of Peace Research · doi:10.1177/0022343304043775 · Cited by 690 · see citing works
UK health performance: findings of the Global Burden of Disease Study 2010
Murray, C. J., Richards, M. A., Newton, J. N., Fenton, K. A., Anderson, H. R., Atkinson, C., Bennett, D., Bernabé, E., Blencowe, H., Bourne, R., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)60355-4 · Cited by 553 · see citing works
Social network determinants of depression
Rosenquist, J. N., Fowler, J. H., & Christakis, N. A. (2011) · Molecular Psychiatry · doi:10.1038/mp.2010.13 · Cited by 543 · see citing works
Product fit uncertainty in online markets: nature, effects, and antecedents
Hong, Y. (., & Pavlou, P. A. (2014) · Information Systems Research · doi:10.1287/isre.2014.0520 · Cited by 453 · see citing works
Why is the developed world obese?
Bleich, S. N., Cutler, D., Murray, C., & Adams, A. (2008) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.29.020907.090954 · Cited by 404 · see citing works
Mahajan, S., Caraballo, C., Lu, Y., Valero-Elizondo, J., Massey, D., Annapureddy, A. R., Roy, B., Riley, C., Murugiah, K., Onuma, O., et al. (2021) · JAMA · doi:10.1001/jama.2021.9907 · Cited by 368 · see citing works
Beyond Good Intentions: Designing CSR Initiatives for Greater Social Impact
Barnett, M. L., Henriques, I., & Husted, B. W. (2020) · Journal of Management · doi:10.1177/0149206319900539 · Cited by 358 · see citing works
Ideology, Motivated Reasoning, and Cognitive Reflection: An Experimental Study
Kahan, D. M. (2012) · doi:10.2139/ssrn.2182588 · Cited by 331 · see citing works
How the news media activate public expression and influence national agendas
King, G., Schneer, B., & White, A. (2017) · Science · doi:10.1126/science.aao1100 · Cited by 333 · see citing works
Self-citationRegional Inequalities and Civil Conflict in Sub-Saharan Africa
Østby, G., Nordås, R., & Rød, J. K. (2009) · International Studies Quarterly · doi:10.1111/j.1468-2478.2009.00535.x · Cited by 302 · see citing works
Hargittai, E., & Shaw, A. (2015) · Information, Communication & Society · doi:10.1080/1369118x.2014.957711 · Cited by 257 · see citing works
Ikeda, N., Inoue, M., Iso, H., Ikeda, S., Satoh, T., Noda, M., Mizoue, T., Imano, H., Saito, E., Katanoda, K., et al. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001160 · Cited by 240 · see citing works
Crossing the Water's Edge: Elite Rhetoric, Media Coverage, and the Rally-Round-the-Flag Phenomenon
Groeling, T., & Baum, M. A. (2008) · The Journal of Politics · doi:10.1017/s0022381608081061 · Cited by 243 · see citing works
Obermeyer, Z., Murray, C. J. L., & Gakidou, E. (2008) · BMJ · doi:10.1136/bmj.a137 · Cited by 200 · see citing works
Fix it and They Might Stay: School Facility Quality and Teacher Retention in Washington, D.C.
Buckley, J., Schneider, M., & Shang, Y. (2005) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1111/j.1467-9620.2005.00506.x · Cited by 180 · see citing works
Obermeyer, Z., Rajaratnam, J. K., Park, C. H., Gakidou, E., Hogan, M. C., Lopez, A. D., & Murray, C. J. L. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000260 · Cited by 110 · see citing works
Rice or riots: On food production and conflict severity across India
Wischnath, G., & Buhaug, H. (2014) · Political Geography · doi:10.1016/j.polgeo.2014.07.004 · Cited by 93 · see citing works
The Cultural Orientation of Mass Political Opinion
Gastil, J., Braman, D., Kahan, D., & Slovic, P. (2011) · PS: Political Science & Politics · doi:10.1017/s1049096511001326 · Cited by 87 · see citing works
Carrasco-Farré, C. (2022) · Humanities and Social Sciences Communications · doi:10.1057/s41599-022-01174-9 · Cited by 84 · see citing works
Demographics, not beverage consumption, is associated with diet quality
Forshee, R. A., & Storey, M. L. (2006) · International Journal of Food Sciences and Nutrition · doi:10.1080/09637480600991240 · Cited by 68 · see citing works
Environmental migrants and social-movement participation
Koubi, V., Nguyen, Q., Spilker, G., & Böhmelt, T. (2021) · Journal of Peace Research · doi:10.1177/0022343320972153 · Cited by 59 · see citing works
Liu, D. M., & Salganik, M. J. (2019) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023119849803 · Cited by 47 · see citing works
A Bayesian Alternative to Synthetic Control for Comparative Case Studies
Pang, X., Liu, L., & Xu, Y. (2022) · Political Analysis · doi:10.1017/pan.2021.22 · Cited by 57 · see citing works
“It captures three sources of uncertainties: (1) the uncertainties from the data generating processes (DGPs), or fundamental uncertainties (King, Tomz, and Wittenberg 2000); (2) the uncertainties from parameter estimation; and (3) the uncertainties from choosing the most suitable model, while…”
BackgroundStoll, L. C., Lilley, T. G., & Block, R. (2021) · Journal of Interpersonal Violence · doi:10.1177/0886260518807912 · Cited by 14 · see citing works
Kemp, C. G., Mntambo, N., Bachmann, M., Bhana, A., Rao, D., Grant, M., Hughes, J. P., Simoni, J. M., Weiner, B. J., Gigaba, S. G., et al. (2020) · Global Mental Health · doi:10.1017/gmh.2020.11 · Cited by 12 · see citing works
EL COSTO DE COORDINAR: NÚMERO DE CANDIDATOS PRESIDENCIALES EN AMÉRICA LATINA 1993-2010
LUJÁN, D. (2017) · Revista de ciencia política (Santiago) · doi:10.4067/s0718-090x2017000100002 · Cited by 7 · see citing works
A Fast, Easy, and Efficient Estimator for Multiparty Electoral Data
Honaker, J., Katz, J. N., & King, G. (2002) · Political Analysis · doi:10.1093/pan/10.1.84 · Cited by 5 · see citing works
Self-citation
- Authors
- King, G., Tomz, M., & Wittenberg, J.
- Venue
- American Journal of Political Science
- Publisher
- JSTOR
- Published
- 2000-04
- Volume
- 44
- Issue
- 2
- Pages
- 347
- ISSN
- 0092-5853 (ISSN-L)
- OpenAlex
- W3125554979
- Cited by
- 3,232
- References
- 28
- Citations by year
- 2026: 61 · 2025: 71 · 2024: 77 · 2023: 86 · 2022: 75 · 2021: 108 · 2020: 112 · 2019: 123 · 2018: 144 · 2017: 126 · 2016: 137 · 2015: 149 · 2014: 211 · 2013: 224 · 2012: 197
- FWCI
- 75.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Political Science (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Keywords
- statistical simulation, statistical interpretation, education and income, confidence intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
