Using online, crowdsourcing platforms for data collection in personality disorder research: the example of Amazon’s Mechanical Turk
Abstract
The use of crowdsourcing platforms such as Amazon's Mechanical Turk (MTurk) for data collection in the behavioral sciences has increased substantially in the past several years due in large part to (a) the ability to recruit large samples, (b) the inexpensiveness of data collection, (c) the speed of data collection, and (d) evidence that the data collected are, for the most part, of equal or better quality to that collected in undergraduate research pools. In this review, we first evaluate the strengths and potential limitations of this approach to data collection. Second, we examine how MTurk has been used to date in personality disorder (PD) research and compare the characteristics of such research to PD research conducted in other settings. Third, we compare PD trait data from the Section III trait model of the DSM-5 collected via MTurk to data collected using undergraduate and clinical samples with regard to internal consistency, mean-level differences, and factor structure. Overall, we conclude that platforms such as MTurk have much to offer PD researchers, especially for certain kinds of research (e.g., where large samples are required and there is a need for iterative sampling). Whether MTurk itself remains the predominant model of such platforms is unclear, however, and will largely depend on decisions related to cost effectiveness and the development of alternatives that offer even greater flexibility. (PsycINFO Database Record
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Data gathered using MTurk are high-quality, with high levels of internal consistency compared to data collected using different methods.
“Additionally, the data gathered using MTurk are high-quality, with high levels of internal consistency compared to data collected using different methods [61].”
From Internalized Racism and Mental Health: The Moderating Role of Collective Racial Self-Esteem (2024) · cites this paper as [61]
Data collected via MTurk is considered equal or better quality compared to data collected from undergraduate research pools.
“Compared to data collected from undergraduate research pools, the data collected via MTurk is considered equal or better quality (Miller et al., 2017).”
From Virtual reality adoption during the COVID-19 pandemic: A uses and gratifications perspective (2021) · cites this paper as Miller et al., 2017
Cites
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996 · see citing works
Constructing validity: Basic issues in objective scale development.
Clark, L. A., & Watson, D. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.309 · Cited by 6,567 · see citing works
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008 · see citing works
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · see citing works
At what sample size do correlations stabilize?
Schönbrodt, F. D., & Perugini, M. (2013) · Journal of Research in Personality · doi:10.1016/j.jrp.2013.05.009 · Cited by 2,232 · see citing works
Introducing the Short Dark Triad (SD3): A Brief Measure of Dark Personality Traits
Jones, D. N., & Paulhus, D. L. (2014) · Assessment · doi:10.1177/1073191113514105 · Cited by 2,162 · see citing works
Sears, D. O. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.3.515 · Cited by 2,138 · see citing works
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136 · see citing works
Initial construction of a maladaptive personality trait model and inventory for DSM-5
Krueger, R. F., Derringer, J., Markon, K. E., Watson, D., & Skodol, A. E. (2012) · Psychological Medicine · doi:10.1017/s0033291711002674 · Cited by 1,908 · see citing works
Hauser, D. J., & Schwarz, N. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0578-z · Cited by 1,739 · see citing works
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618 · see citing works
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617 · see citing works
Using Mechanical Turk to Study Clinical Populations
Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013) · Clinical Psychological Science · doi:10.1177/2167702612469015 · Cited by 1,139 · see citing works
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072 · see citing works
The viability of crowdsourcing for survey research
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0081-0 · Cited by 1,049 · see citing works
Samuel, D. B., & Widiger, T. A. (2008) · Clinical Psychology Review · doi:10.1016/j.cpr.2008.07.002 · Cited by 1,018 · see citing works
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950 · see citing works
Landers, R. N., & Behrend, T. S. (2015) · Industrial and Organizational Psychology · doi:10.1017/iop.2015.13 · Cited by 714 · see citing works
The role of impulsivity in the Dark Triad of personality
Jones, D. N., & Paulhus, D. L. (2011) · Personality and Individual Differences · doi:10.1016/j.paid.2011.04.011 · Cited by 618 · see citing works
The labor economics of paid crowdsourcing
Horton, J. J., & Chilton, L. B. (2010) · ACM Conference on Electronic Commerce (EC) · doi:10.1145/1807342.1807376 · Cited by 618 · see citing works
Internet Research in Psychology
Gosling, S. D., & Mason, W. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015321 · Cited by 458 · see citing works
Maples, J. L., Carter, N. T., Few, L. R., Crego, C., Gore, W. L., Samuel, D. B., Williamson, R. L., Lynam, D. R., Widiger, T. A., Markon, K. E., et al. (2015) · Psychological Assessment · doi:10.1037/pas0000120 · Cited by 337 · see citing works
The importance of assessing clinical phenomena in Mechanical Turk research.
Hall, K. A. A., Çek, D., Shaw, A. M., & Timpano, K. R. (2015) · Psychological Assessment · doi:10.1037/pas0000217 · Cited by 311 · see citing works
The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers
Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500005611 · Cited by 304 · see citing works
A test of two brief measures of the dark triad: The dirty dozen and short dark triad.
Maples, J. L., Lamkin, J., & Miller, J. D. (2013) · Psychological Assessment · doi:10.1037/a0035084 · Cited by 284 · see citing works
Litman, L., Robinson, J., & Rosenzweig, C. (2014) · Behavior Research Methods · doi:10.3758/s13428-014-0483-x · Cited by 275 · see citing works
Grandiose and vulnerable narcissism from the perspective of the interpersonal circumplex
Miller, J. D., Price, J., Gentile, B., Lynam, D. R., & Campbell, W. K. (2012) · Personality and Individual Differences · doi:10.1016/j.paid.2012.04.026 · Cited by 196 · see citing works
Assessing the reliability of the M5-120 on Amazon’s mechanical Turk
Holden, C. J., Dennie, T., & Hicks, A. D. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.02.020 · Cited by 169 · see citing works
Ten aspects of the Big Five in the Personality Inventory for DSM–5.
DeYoung, C. G., Carey, B. E., Krueger, R. F., & Ross, S. R. (2016) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000170 · Cited by 165 · see citing works
The psychology of spite and the measurement of spitefulness.
Marcus, D. K., Zeigler‐Hill, V., Mercer, S. H., & Norris, A. L. (2014) · Psychological Assessment · doi:10.1037/a0036039 · Cited by 147 · see citing works
Negative affect intensity influences drinking to cope through facets of emotion dysregulation
Veilleux, J. C., Skinner, K. D., Reese, E. D., & Shaver, J. A. (2013) · Personality and Individual Differences · doi:10.1016/j.paid.2013.11.012 · Cited by 128 · see citing works
Participants at Your Fingertips
Johnson, D. R., & Borden, L. A. (2012) · Teaching of Psychology · doi:10.1177/0098628312456615 · Cited by 119 · see citing works
Feitosa, J., Joseph, D. L., & Newman, D. A. (2014) · Personality and Individual Differences · doi:10.1016/j.paid.2014.11.017 · Cited by 118 · see citing works
Using Nonnaive Participants Can Reduce Effect SizesPreprint
Chandler, J., Paolacci, G., Péer, E., Mueller, P. A., & Ratliff, K. A. (2015) · RePEc: Research Papers in Economics · Cited by 118 · see citing works
Collecting response times using Amazon Mechanical Turk and Adobe Flash
Simcox, T., & Fiez, J. A. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0345-y · Cited by 97 · see citing works
Miller, J. D., Few, L. R., Wilson, L. F., Gentile, B., Widiger, T. A., MacKillop, J., & Campbell, W. K. (2013) · Psychological Assessment · doi:10.1037/a0032536 · Cited by 88 · see citing works
Does assessing suicidality frequently and repeatedly cause harm? a randomized control study
Law, M. K., Furr, R. M., Arnold, E. M., Mneimne, M., Jaquett, C., & Fleeson, W. (2015) · Psychological Assessment · doi:10.1037/pas0000118 · Cited by 82 · see citing works
Convergent and Discriminant Validity of the Five Factor Form
Rojas, S. L., & Widiger, T. A. (2013) · Assessment · doi:10.1177/1073191113517260 · Cited by 66 · see citing works
An online daily diary study of alcohol use using A mazon's M echanical T urk
Boynton, M. H., & Richman, L. S. (2014) · Drug and Alcohol Review · doi:10.1111/dar.12163 · Cited by 63 · see citing works
Péer, E., Samat, S., Brandimarte, L., & Acquisti, A. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2594183 · Cited by 44 · see citing works
Corker, K. S., Donnellan, M. B., Kim, S. Y., Schwartz, S. J., & Zamboanga, B. L. (2015) · Journal of Personality · doi:10.1111/jopy.12224 · Cited by 35 · see citing works
Assessment of dependency by the FFDI: Comparisons to the PID-5 and maladaptive agreeableness
Gore, W. L., & Widiger, T. A. (2015) · Personality and Mental Health · doi:10.1002/pmh.1308 · Cited by 23 · see citing works
Flagged first, then by citations
1 of the 44 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Demographics and Dynamics of Mechanical Turk Workers
Difallah, D. E., Filatova, E., & Ipeirotis, P. G. (2018) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3159652.3159661 · Cited by 518 · see citing works
The Use of Online Panel Data in Management Research: A Review and Recommendations
Porter, C. O. L. H., Outlaw, R., Gale, J., & Cho, T. S. (2018) · Journal of Management · doi:10.1177/0149206318811569 · Cited by 346 · see citing works
Exploring the structure of narcissism: Toward an integrated solution
Crowe, M. L., Lynam, D. R., Campbell, W. K., & Miller, J. D. (2019) · Journal of Personality · doi:10.1111/jopy.12464 · Cited by 317 · see citing works
The use of crowdsourcing in addiction science research: Amazon Mechanical Turk.
Strickland, J. C., & Stoops, W. W. (2018) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000235 · Cited by 249 · see citing works
Who Are the Turkers? A Characterization of MTurk Workers Using the Personality Assessment Inventory
McCredie, M. N., & Morey, L. C. (2018) · Assessment · doi:10.1177/1073191118760709 · Cited by 235 · see citing works
Martela, F., & Riekki, T. J. J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01157 · Cited by 252 · see citing works
Blagov, P. S. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550620936439 · Cited by 217 · see citing works
Measuring the dark core of personality.
Moshagen, M., Zettler, I., & Hilbig, B. E. (2019) · Psychological Assessment · doi:10.1037/pas0000778 · Cited by 199 · see citing works
A Critical Appraisal of the Dark-Triad Literature and Suggestions for Moving Forward
Miller, J. D., Vize, C. E., Crowe, M. L., & Lynam, D. R. (2019) · Current Directions in Psychological Science · doi:10.1177/0963721419838233 · Cited by 178 · see citing works
Virtual reality adoption during the COVID-19 pandemic: A uses and gratifications perspective
Ball, C., Huang, K.-T., & Francis, J. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2021.101728 · Cited by 152 · see citing works
“Compared to data collected from undergraduate research pools, the data collected via MTurk is considered equal or better quality (Miller et al., 2017).”
Uncovering the structure of agreeableness from self‐report measures
Crowe, M. L., Lynam, D. R., & Miller, J. D. (2017) · Journal of Personality · doi:10.1111/jopy.12358 · Cited by 147 · see citing works
Kwapil, T. R., Gross, G. M., Silvia, P. J., Raulin, M. L., & Barrantes‐Vidal, N. (2017) · Schizophrenia Research · doi:10.1016/j.schres.2017.07.001 · Cited by 142 · see citing works
Validity of the DSM–5 Levels of Personality Functioning Scale–Self Report
Hopwood, C. J., Good, E. W., & Morey, L. C. (2018) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1420660 · Cited by 128 · see citing works
The basic trait of Antagonism: An unfortunately underappreciated construct
Lynam, D. R., & Miller, J. D. (2019) · Journal of Research in Personality · doi:10.1016/j.jrp.2019.05.012 · Cited by 130 · see citing works
Sleep, C. E., Lynam, D. R., Widiger, T. A., Crowe, M. L., & Miller, J. D. (2019) · Psychological Assessment · doi:10.1037/pas0000620 · Cited by 122 · see citing works
Participant carelessness and fraud: Consequences for clinical research and potential solutions.
Chandler, J., Sisso, I., & Shapiro, D. N. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000479 · Cited by 121 · see citing works
Development and Initial Validation of the Duke Misophonia Questionnaire
Rosenthal, M. Z., Anand, D., Cassiello-Robbins, C., Williams, Z. J., Guetta, R. E., Trumbull, J., & Kelley, L. D. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.709928 · Cited by 117 · see citing works
Dark Triad traits and their structure: An empirical approach
Rogoza, R., & Cieciuch, J. (2018) · Current Psychology · doi:10.1007/s12144-018-9834-6 · Cited by 119 · see citing works
Patihis, L., & Pendergrast, M. H. (2018) · Clinical Psychological Science · doi:10.1177/2167702618773315 · Cited by 95 · see citing works
Murphy, B. A., Costello, T. H., Watts, A. L., Cheong, Y. F., Berg, J. M., & Lilienfeld, S. O. (2018) · Assessment · doi:10.1177/1073191118777636 · Cited by 84 · see citing works
Oltmanns, J. R., & Widiger, T. A. (2018) · Psychological Assessment · doi:10.1037/pas0000616 · Cited by 82 · see citing works
A comprehensive comparison of the ICD-11 and DSM–5 section III personality disorder models.
McCabe, G. A., & Widiger, T. A. (2020) · Psychological Assessment · doi:10.1037/pas0000772 · Cited by 93 · see citing works
Rogoza, R., Cieciuch, J., Strus, W., & Baran, T. (2019) · European Journal of Personality · doi:10.1002/per.2206 · Cited by 80 · see citing works
Evaluating the assessment of the ICD-11 personality disorder diagnostic system.
Oltmanns, J. R., & Widiger, T. A. (2019) · Psychological Assessment · doi:10.1037/pas0000693 · Cited by 109 · see citing works
Slepian, P. M., Ankawi, B., & France, C. R. (2019) · Annals of Behavioral Medicine · doi:10.1093/abm/kaz051 · Cited by 70 · see citing works
Validation of the Narcissistic Grandiosity Scale and creation of reduced item variants.
Crowe, M. L., Carter, N. T., Campbell, W. K., & Miller, J. D. (2016) · Psychological Assessment · doi:10.1037/pas0000281 · Cited by 70 · see citing works
Prinzing, M., Zhou, J., West, T. N., Nguyen, K. D. L., Wells, J. L., & Fredrickson, B. L. (2020) · The Journal of Positive Psychology · doi:10.1080/17439760.2020.1858336 · Cited by 69 · see citing works
Kratom use as more than a “self-treatment”
Smith, K. E., Dunn, K. E., Rogers, J. M., Grundmann, O., McCurdy, C. R., Garcia‐Romeu, A., Schriefer, D., Swogger, M. T., & Epstein, D. H. (2022) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2022.2083967 · Cited by 65 · see citing works
Parra, C. M., Gupta, M., & Cadden, T. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.06.009 · Cited by 60 · see citing works
We are in this together: sport brand involvement and fans’ well-being
Su, Y., Du, J., Biscaia, R., & Inoue, Y. (2021) · European Sport Management Quarterly · doi:10.1080/16184742.2021.1978519 · Cited by 57 · see citing works
Smith, K. E., Rogers, J. M., Dunn, K. E., Grundmann, O., McCurdy, C. R., Schriefer, D., & Epstein, D. H. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.765917 · Cited by 54 · see citing works
Sestino, A. (2023) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103488 · Cited by 52 · see citing works
Diminished self-concept and social exclusion: Disparagement humor from the target’s perspective
Ford, T. E., Buie, H. S., Mason, S. D., Olah, A. R., Breeden, C. J., & Ferguson, M. A. (2019) · Self and Identity · doi:10.1080/15298868.2019.1653960 · Cited by 53 · see citing works
Truhan, T. E., Wilson, P. C., Mõttus, R., & Papageorgiou, K. Α. (2020) · Personality and Individual Differences · doi:10.1016/j.paid.2020.110502 · Cited by 51 · see citing works
The Problem with Online Data Collection: Predicting Invalid Responding in Undergraduate Samples
Al-Salom, P., & Miller, C. J. (2017) · Current Psychology · doi:10.1007/s12144-017-9674-9 · Cited by 48 · see citing works
Smith, K. E., Dunn, K. E., Grundmann, O., Garcia‐Romeu, A., Rogers, J. M., Swogger, M. T., & Epstein, D. H. (2021) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000518 · Cited by 49 · see citing works
Online Methods in Adolescent Self-Injury Research: Challenges and Recommendations
Smith, D. M. Y., Lipson, S. M., Wang, S. B., & Fox, K. R. (2021) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2021.1875325 · Cited by 46 · see citing works
Computer criminal behavior is related to psychopathy and other antisocial behavior
Seigfried‐Spellar, K. C., Villacís-Vukadinović, N., & Lynam, D. R. (2017) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2017.06.003 · Cited by 46 · see citing works
Reliability of MTurk Data From Masters and Workers
Rouse, S. V. (2019) · Journal of Individual Differences · doi:10.1027/1614-0001/a000300 · Cited by 44 · see citing works
Service employees’ STARA awareness and proactive service performance
Hur, W., & Shin, Y. (2024) · Journal of Services Marketing · doi:10.1108/jsm-03-2023-0115 · Cited by 52 · see citing works
Chasson, G. S., Guy, A. A., Bates, S., & Corrigan, P. W. (2017) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2017.12.006 · Cited by 42 · see citing works
Pathological Narcissism: A Study of Burden on Partners and Family
Day, N. J. S., Bourke, M. E., Townsend, M. L., & Grenyer, B. F. S. (2020) · Journal of Personality Disorders · doi:10.1521/pedi_2019_33_413 · Cited by 40 · see citing works
Social status as one key indicator of successful psychopathy: An initial empirical investigation
Persson, B. N., & Lilienfeld, S. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.01.020 · Cited by 34 · see citing works
Relational maintenance, collectivism, and coping strategies among Black populations during COVID-19
Stamps, D. L., Mandell, L., & Lucas, R. (2021) · Journal of Social and Personal Relationships · doi:10.1177/02654075211025093 · Cited by 38 · see citing works
Costello, T. H., Unterberger, A., Watts, A. L., & Lilienfeld, S. O. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00185 · Cited by 37 · see citing works
Crego, C., & Widiger, T. A. (2019) · Psychological Assessment · doi:10.1037/pas0000757 · Cited by 38 · see citing works
Zhang, J. (., & Liu, R. (2023) · Telematics and Informatics · doi:10.1016/j.tele.2023.102075 · Cited by 38 · see citing works
Dark personality voters find dark politicians more relatable and fit for office
Hart, W., Richardson, K., & Tortoriello, G. K. (2018) · Journal of Research in Personality · doi:10.1016/j.jrp.2018.05.007 · Cited by 36 · see citing works
Rogers, J. M., Smith, K. E., Strickland, J. C., & Epstein, D. H. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.789075 · Cited by 35 · see citing works
Hur, W., Shin, Y., & Kim, J. Y. (2023) · Current Psychology · doi:10.1007/s12144-023-04714-x · Cited by 39 · see citing works
Exploring Gender Differences in Machiavellianism Using a Measurement Invariance Approach
Collison, K. L., South, S. C., Vize, C. E., Miller, J. D., & Lynam, D. R. (2020) · Journal of Personality Assessment · doi:10.1080/00223891.2020.1729773 · Cited by 33 · see citing works
Humor styles predict emotional and behavioral responses to COVID-19
Olah, A. R., & Ford, T. E. (2021) · Humor - International Journal of Humor Research · doi:10.1515/humor-2021-0009 · Cited by 31 · see citing works
What Crowdsourcing Can Offer to Cross-Cultural Psychological Science
Cuccolo, K., Irgens, M. S., Zlokovich, M. S., Grahe, J. E., & Edlund, J. E. (2020) · Cross-Cultural Research · doi:10.1177/1069397120950628 · Cited by 30 · see citing works
Amoah, J. O., Egala, S. B., Keelson, S. A., Bruce, E., Dziwornu, R. K., & Duah, F. A. (2023) · Cogent Business & Management · doi:10.1080/23311975.2022.2163796 · Cited by 40 · see citing works
Drinkwater, K. G., Dagnall, N., Denovan, A., & Williams, C. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.670959 · Cited by 29 · see citing works
A brief tale of the two faces of narcissism and the two facets of pride
Rogoza, R., Kwiatkowska, M. M., Kowalski, C. M., & Ślaski, S. (2018) · Personality and Individual Differences · doi:10.1016/j.paid.2018.01.027 · Cited by 28 · see citing works
Manago, B. N., Mize, T. D., & Doan, L. (2021) · Sociological Methodology · doi:10.1177/00811750211014242 · Cited by 27 · see citing works
Meet your public relations team: People with dark traits may help you manage your image
Hart, W., Richardson, K., & Tortoriello, G. K. (2018) · Personality and Individual Differences · doi:10.1016/j.paid.2018.06.019 · Cited by 26 · see citing works
Crowe, M. L., Weiss, B., Sleep, C. E., Harris, A. M., Carter, N. T., Lynam, D. R., & Miller, J. D. (2020) · Assessment · doi:10.1177/1073191120907959 · Cited by 25 · see citing works
Stigmatization of opioid addiction based on prescription, sex and age
Weeks, C., & Stenstrom, D. M. (2020) · Addictive Behaviors · doi:10.1016/j.addbeh.2020.106469 · Cited by 25 · see citing works
Hong, G. C. C., Conduit, R., Wong, J., Benedetto, M. D., & Lee, E. (2020) · Journal of Attention Disorders · doi:10.1177/1087054720940417 · Cited by 25 · see citing works
Where’s your sense of humor? Political identity moderates evaluations of disparagement humor
Buie, H. S., Ford, T. E., Olah, A. R., Argüello‐Gutiérrez, C., & Mendiburo‐Seguel, A. (2021) · Group Processes & Intergroup Relations · doi:10.1177/1368430221998792 · Cited by 26 · see citing works
The General Factors of Personality Disorder, Psychopathology, and Personality
McCabe, G. A., Oltmanns, J. R., & Widiger, T. A. (2021) · Journal of Personality Disorders · doi:10.1521/pedi_2021_35_530 · Cited by 24 · see citing works
Feeling good about oneself heightens, not hinders, the goodness in narcissism
Hart, W., Tortoriello, G. K., & Richardson, K. (2018) · Current Psychology · doi:10.1007/s12144-018-9993-5 · Cited by 24 · see citing works
Strickland, J. C., & Victor, G. A. (2019) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2019.10.013 · Cited by 24 · see citing works
Searching for Machiavelli but finding psychopathy and narcissism.
Persson, B. N. (2019) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000323 · Cited by 28 · see citing works
Profiling public and private self-consciousness on self-presentation tactic use
Hart, W., Tortoriello, G. K., & Richardson, K. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.04.010 · Cited by 23 · see citing works
Private rank aggregation under local differential privacy
Yan, Z., Li, G., & Liu, J. (2020) · International Journal of Intelligent Systems · doi:10.1002/int.22261 · Cited by 23 · see citing works
Gallor, S. M. (2017) · Psi Chi Journal of Psychological Research · doi:10.24839/2325-7342.jn22.4.254 · Cited by 22 · see citing works
Wuth, A., Mishra, S., Beshai, S., & Feeney, J. R. (2021) · Current Psychology · doi:10.1007/s12144-020-01274-2 · Cited by 21 · see citing works
Hart, W., Richardson, K., & Tortoriello, G. K. (2018) · Journal of Interpersonal Violence · doi:10.1177/0886260518777551 · Cited by 20 · see citing works
Wilson, N. C., & Seigfried‐Spellar, K. C. (2022) · Social Science Computer Review · doi:10.1177/08944393211065868 · Cited by 19 · see citing works
Merz, Z. C., Zane, K. L., Emmert, N. A., Lace, J. W., & Grant, A. F. (2019) · Brain Injury · doi:10.1080/02699052.2019.1581949 · Cited by 18 · see citing works
Alleyne, E. K. A., & Parfitt, C. H. (2018) · Aggressive Behavior · doi:10.1002/ab.21768 · Cited by 17 · see citing works
Aslam, M., Nazri, M., Omar, S. B., & Bustaman, H. A. (2020) · Cogent Business & Management · doi:10.1080/23311975.2020.1771117 · Cited by 19 · see citing works
Public perceptions and disparities in access to telehealth orthopaedic services in the COVID-19 era
Puzzitiello, R. N., Moverman, M. A., Pagani, N. R., Ryan, S. P., Salzler, M. J., Jawa, A., & Menendez, M. E. (2021) · Journal of the National Medical Association · doi:10.1016/j.jnma.2021.02.007 · Cited by 16 · see citing works
Weiss, B., Sleep, C. E., Beller, N. M., Erritzøe, D., & Campbell, W. K. (2023) · Journal of Psychedelic Studies · doi:10.1556/2054.2023.00291 · Cited by 15 · see citing works
Stanton, K. J., & Watson, D. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.05.046 · Cited by 15 · see citing works
Biberdzic, M., Grenyer, B. F. S., Normandin, L., Ensink, K., & Clarkin, J. F. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-03926-y · Cited by 17 · see citing works
Huang, K.-T., Ball, C., & Francis, J. (2023) · American Behavioral Scientist · doi:10.1177/00027642231156775 · Cited by 19 · see citing works
Comparing data quality from an online and in-person lab sample on dynamic theory of mind tasks
Krendl, A. C., Hugenberg, K., & Kennedy, D. P. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02152-y · Cited by 14 · see citing works
Parsing the Hypomanic Personality: Explicating the Nature of Specific Dimensions Defining Mania Risk
Stanton, K. J., McArtor, D. B., & Watson, D. (2017) · Assessment · doi:10.1177/1073191117725170 · Cited by 14 · see citing works
Internalized Racism and Mental Health: The Moderating Role of Collective Racial Self-Esteem
Sanders, S., Williams, T., Berry, A., Garcia-Aguilera, C., Robinson, K., Martin, R., & Jones, P. (2024) · Behavioral Sciences · doi:10.3390/bs14111003 · Cited by 14 · see citing works
Learning the statistics of pronoun reference: By word or by category?
Ye, Y.-N., & Arnold, J. E. (2023) · Cognition · doi:10.1016/j.cognition.2023.105546 · Cited by 11 · see citing works
An approach to task recommendation in crowdsourcing based on 2-tuple fuzzy linguistic method
Zhang, X., & Su, J. (2018) · Kybernetes · doi:10.1108/k-12-2017-0468 · Cited by 13 · see citing works
Psychopathy and Ratings of Persuasiveness: Examining Their Relations in Weaker and Stronger Contexts
Weiss, B., Lynam, D. R., & Miller, J. D. (2018) · Clinical Psychological Science · doi:10.1177/2167702618783733 · Cited by 13 · see citing works
Development and initial validation of the HEXACO-Triarchic scales.
Ruchensky, J. R., Donnellan, M. B., & Edens, J. F. (2018) · Psychological Assessment · doi:10.1037/pas0000609 · Cited by 12 · see citing works
Hanniball, K. B., Gatner, D. T., Douglas, K. S., Viljoen, J. L., & Aknin, L. B. (2019) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000333 · Cited by 13 · see citing works
Strickland, J. C., & Stoops, W. W. (2019) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2019.1689990 · Cited by 12 · see citing works
Understanding compensatory travel
Kim, J. Y., Kim, J., & Koo, C. (2024) · Annals of Tourism Research · doi:10.1016/j.annals.2023.103712 · Cited by 13 · see citing works
Revisiting the Impact of Goals on Affect and Effort*
Lane, D. (2021) · Accounting Perspectives · doi:10.1111/1911-3838.12277 · Cited by 11 · see citing works
Time-of-day and day-of-week variations in Amazon Mechanical Turk survey responses
Binder, C. C. (2021) · Journal of Macroeconomics · doi:10.1016/j.jmacro.2021.103378 · Cited by 11 · see citing works
Petri, J. M., Weathers, F. W., Witte, T. K., & Silverstein, M. W. (2019) · Assessment · doi:10.1177/1073191119880963 · Cited by 11 · see citing works
The effect of persuasive/possessing information regarding GMOs on consumer attitudes
Stanton, J. L., Rezai, G., & Baglione, S. L. (2021) · Future Foods · doi:10.1016/j.fufo.2021.100076 · Cited by 10 · see citing works
Preston, O. C., Anestis, J. C., Watts, A. L., Bulla, B. A., Harrop, T. M., Laare, J. R.-V., & Lilienfeld, S. O. (2021) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-021-09918-8 · Cited by 10 · see citing works
Maladaptive variants of adaptive traits and bloated specific factors
Oltmanns, J. R., & Widiger, T. A. (2018) · Journal of Research in Personality · doi:10.1016/j.jrp.2018.08.006 · Cited by 9 · see citing works
Sleep, C. E., Lynam, D. R., Campbell, W. K., & Miller, J. D. (2019) · Journal of Personality Disorders · doi:10.1521/pedi_2019_33_422 · Cited by 10 · see citing works
Individual Differences in Psychological Stress Associated with Data Breach Experiences
Sears, C. R., & Cunningham, D. R. (2024) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp4030028 · Cited by 11 · see citing works
Research Ethics in the Age of Digital Platforms
Molina, J. L., Tubaro, P., Casilli, A. A., & Ortega, A. S. (2023) · Science and Engineering Ethics · doi:10.1007/s11948-023-00437-1 · Cited by 9 · see citing works
Borderline personality disorder features are associated with inflexible social interpretations
Duda, J. M., Fineberg, S. K., Deng, W., Ma, Y., Everaert, J., Cannon, T. D., & Joormann, J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.12.036 · Cited by 9 · see citing works
- Authors
- Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., & Lynam, D. R.
- Affiliations
- University of Georgia (US)
- Purdue University West Lafayette (US)
- Venue
- Personality Disorders Theory Research and Treatment
- Published
- 2017
- Volume
- 8
- Issue
- 1
- Pages
- 26-34
- ISSN
- 1949-2715 (ISSN-L)
- 1949-2723
- OpenAlex
- W2565779926
- Cited by
- 179
- References
- 44
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 12 · 2023: 22 · 2022: 13 · 2021: 25 · 2020: 21 · 2019: 37 · 2018: 27 · 2017: 13 · 2016: 1
- FWCI
- 32.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Personality Disorders Theory Research and Treatment (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Psychology › Clinical Psychology › Personality Disorders and Psychopathology
- Social Sciences › Psychology › Clinical Psychology › Personality Traits and Psychology
- Keywords
- Amazon Mechanical Turk, crowdsourcing, online data collection, personality disorders, DSM-5, trait data, sample recruitment, data quality, factor structure, internal consistency, cost-effectiveness
- MeSH terms
- Data Collection, Humans, Personality Disorders, Internet, Biomedical Research, Crowdsourcing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review