Rating the ratings: Assessing the psychometric quality of rating data.

Saal, Frank E., Downey, Ronald G., Lahey, Mary A.·1980·Psychological Bulletin

Cited by (567)

Items 201–300 of 567

  1. A Monitoring and Evaluation Sustainability Framework for Road Freight Transporters in South Africa
    Chumasande Lalendle, Leila Louise Goedhals-Gerber, Joubert van Eeden · 2021 · Sustainability
  2. Judging behaviour and rater errors: an application of the many-facet rasch model.
    Noor Lide Abu Kassim · 2011 · GEMA Online Journal of Language Studies
  3. A note on the response effects of laboratory- versus respondent-located computer-administered questioning
    Eugene D. Jaffe, Hanoch Pasternak · 1997 · Industrial Marketing Management
  4. Bias in psychotherapist ratings of client transference and insight.
    Rayna D. Markin, Dennis M. Kivlighan · 2007 · Psychotherapy
  5. Assessment center construct-related validity: Stepping beyond the MTMM matrix
    Mark C. Bowler, David J. Woehr · 2009 · Journal of Vocational Behavior
  6. A critical evaluation of the validity and the reliability of global competency constructs for supervisor assessment of junior medical trainees
    Darryl Andrew McGill, Cees P. M. van der Vleuten, Mike J Clarke · 2012 · Advances in Health Sciences Education
  7. The Reliability of Simulated Clients' Quality-of-Care Ratings
    Federico R. León, Gustavo Quiroz, Alfredo Brazzoduro · 1994 · Studies in Family Planning
  8. The Retention Effect of Withholding Performance Information
    Korok Ray · 2007 · The Accounting Review
  9. Designs for evaluating the validity and accuracy of performance ratings
    Terry L. Dickinson · 1987 · Organizational Behavior and Human Decision Processes
  10. The Role of Motor Excess and Instrumented Activity Measurement in Attention Deficit Hyperactivity Disorder
    Warren W. Tryon · 1993 · Behavior Modification
  11. STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports
    Tegan McCaslin, Jide Alaga, Samira Nedungadi, Seth Donoughe, Tom Reed, Rishi Bommasani, C. Painter, Luca Righetti · 2025 · arXiv.org
  12. On the history of rating scales
    Paul McReynolds, Klaus Ludwig · 1987 · Personality and Individual Differences
  13. The limits of institutional isomorphism in the design of e-recruitment websites: a comparative analysis of the USA and Spain
    Cristina Simón, José Esteves · 2015 · The International Journal of Human Resource Management
  14. Properties of the Achievement of Therapeutic Objectives Scale (ATOS): A Generalizability Theory study
    Lene Berggraf, Pål Ulvenes, Bruce E. Wampold, Asle Hoffart, Leigh McCullough · 2012 · Psychotherapy Research
  15. Halo effect: Conceptual definition and empirical exploration with regard to South Korean subsidiaries of US and Japanese multinational corporations
    Mi‐Young Oh, Jyotika Ramaprasad · 2003 · Journal of Communication Management
  16. Ratings of performance in multisource feedback: comparing performance theories of residents and nurses
    Muhammad Tariq, Marjan Govaerts, Azam Saeed Afzal, Syed Ahsan Ali, Tabassum Zehra · 2020 · BMC Medical Education
  17. Evaluating the Quality of Ratings Obtained from Standard-Seting Judges
    George Engelhard, Gregory Ethan Stone · 1998 · Educational and Psychological Measurement
  18. The stability of marker characteristics across tests of the same subject and across subjects.
    Iasonas Lamprianou · 2006 · PubMed
  19. Detecting Differential Rater Functioning in Severity and Centrality: The Dual DRF Facets Model
    Kuan‐Yu Jin, Thomas Eckes · 2021 · Educational and Psychological Measurement
  20. An Examination of Negative Halo Error in Ratings
    Charles E. Lance, Sebastiano A. Fisicaro, Julie A. LaPointe · 1990 · Educational and Psychological Measurement
  21. Content and Effect of Introduction Programmes to Increase Retention and Decrease Turnover of Newly Graduated Nurses in Hospitals: Umbrella Review
    Connie Bøttcher Berthelsen, Carrinna Aviaja Hansen · 2024 · Journal of Clinical Nursing
  22. Establishing reliable procedures for rating ell students’ reading comprehension using oral retellings
    Richard R. Sudweeks, Connie B. Glissmeyer, Timothy G. Morrison, Bradley R. Wilcox, Mark W. Tanner · 2003 · Reading Research and Instruction
  23. Implementation contextual factors related to community-based active travel to school interventions: a mixed methods interview study
    MacKenzie N. Koester, Carolina M. Bejarano, Ann McGrath Davis, Ross C. Brownson, Jon F. Kerner, James F. Sallis, Chelsea Steel, Jordan Carlson · 2021 · Implementation Science Communications
  24. Performance Appraisal
    Angelo S. DeNisi · 2013 · Oxford University Press eBooks
  25. A study of validity and usability evidence for non-technical skills assessment tools in simulated adult resuscitation scenarios
    Helen Higham, Paul Robert Greig, Nick Crabtree, George Hadjipavlou, Duncan Young, Charles A. Vincent · 2023 · BMC Medical Education
  26. Beautiful is Good and Good is Reputable: Multiple-Attribute Charity Website Evaluation andInitialPerceptions ofReputationUnder the Halo Effect
    Dong-Heon Kwak, Keshavamurthy Ramamurthy, Derek L. Nazareth · 2019 · Journal of the Association for Information Systems
  27. An Experimental Comparison of Rater Performance on an SP-Based Clinical Skills Exam
    Jae Jin Han, Clarence Dennis Kreiter, Hyesook Park, Kristi J. Ferguson · 2006 · Teaching and Learning in Medicine
  28. Exploring the Influence of Range Restrictions on Connectivity in Sparse Assessment Networks: An Illustration and Exploration Within the Context of Classroom Observations
    Stefanie A. Wind, Eli A. Jones · 2018 · Journal of Educational Measurement
  29. School adjustment and health in Russian adolescents
    Gennady G. Knyazev, Helena R. Slobodskaya, Margarita V. Safronova, Irina A. Kinsht · 2002 · Psychology Health & Medicine
  30. Behavior Checklists and Rating Forms
    Timothy K. Daugherty, Steven K. Shapiro · 1994 · Issues in clinical child psychology
  31. The Economic Value of Ratings in App Market
    Hao Lin, Xiaofei Li, Yong Tan, Lei Xu · 2011 · SSRN Electronic Journal
  32. Administrators' Uses of Teacher Observation Protocol in Different Rating Contexts
    Yi Qi, Courtney A. Bell, Nathan D. Jones, Jennifer M. Lewis, Margaret W. Witherspoon, Amanda Redash · 2018 · ETS Research Report Series
  33. We agreed to measure agreement - Redefining reliability de-justifies Krippendorff’s alpha
    Xinshu Zhao, Guangchao Charles Feng, Jun S. Liu, Ke Deng · 2018 · HKBU Institutional Repository (Hong Kong Baptist University)
  34. Halo Error: A Field Study Comparison of Self -and Subordinate Evaluations of Leadership Process and Leader Effectiveness
    Michael R. Frone, Jerome Adams, Robert W. Rice, Debra Instone · 1986 · Personality and Social Psychology Bulletin
  35. Replication and Extension of Models of Supervisory Job Performance Ratings
    Charles E. Lance, Winston R. Bennett · 2000 · Human Performance
  36. A Field Study of Rating Conditions and Leniency in Performance Appraisal
    David A. Waldman, George C. Thornton · 1988 · Psychological Reports
  37. How Do Indonesia Firms Encounter Covid-19 Pandemic? An Evidence of Transformation of the Roots of Competitive Advantage from EMDE Country
    2023 · Quality - Access to Success
  38. An Analysis of Rater Severity and Leniency in Open-ended Mathematic Questions Rated Through Standard Rubrics and Rubrics Based on the SOLO Taxonomy
    Bayram Çetin, Mustafa İlhan · 2017 · TED EĞİTİM VE BİLİM
  39. Validity And Reliability Rubric Of Performance Assessment Geometry Study In Junior High School Using The Many Facet Rasch Model Approach
    Rivo Panji Yudha · 2020 · Eduma Mathematics Education Learning and Teaching
  40. Self-organization in online collaborative work settings
    Ioanna Lykourentzou, Federica Lucia Vinella, Faez Ahmed, Costas Papastathis, Konstantinos Papangelis, Vassilis-Javed Khan, Judith Masthoff · 2022 · Collective Intelligence
  41. On the Relationship Between Job Performance and Ratings of Job Performance: What Do We Really Know?
    David J. Woehr · 2008 · Industrial and Organizational Psychology
  42. Operationalisierung
    Nicola M. Döring, Jürgen Bortz · 2015 · Springer-Lehrbuch
  43. Effects of Dextroamphetamine on the Cognitive and Social Play of a Preschooler with ADHD
    Steven M. Alessandri, Kathleen Schramm · 1991 · Journal of the American Academy of Child & Adolescent Psychiatry
  44. Has the problem of judgment in utility analysis been solved?
    Michael K. Judiesch, Frank L. Schmidt, John E. Hunter · 1993 · Journal of Applied Psychology
  45. Individualized feedback to raters in language assessment: Impacts on rater effects
    Jing Huang, Gaowei Chen · 2022 · Assessing Writing
  46. Robot-Led Vision Language Model Wellbeing Assessment of Children
    Nida Itrat Abbasi, Fethiye Irmak Doğan, Guy Laban, Joanna Anderson, Tamsin Jane Ford, Peter Brian Jones, Hatice Güneş · 2025 · IEEE RO-MAN
  47. Test theories, educational priorities and reliability of public examinations in England
    Jo‐Anne Baird, Paul Black · 2013 · Research Papers in Education
  48. Evaluating Rater Accuracy in Rater-Mediated Assessments Using an Unfolding Model
    Jue Wang, George Engelhard, Edward W. Wolfe · 2015 · Educational and Psychological Measurement
  49. Evaluating the Construct Validity of an Automated Writing Evaluation System with a Randomization Algorithm
    Matthew C. Myers, Joshua Wilson · 2022 · International Journal of Artificial Intelligence in Education
  50. Rating Errors and Rating Accuracy: A Field Experiment
    Ronen Kasten, Zisu Weintraub · 1999 · Human Performance
  51. Investigating Central Tendency in Competency Assessment of Design Electronic Circuit: Analysis Using Many Facet Rasch Measurement (MFRM)
    Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja Batu Pahat Johor, Malaysia, Azmanirah Ab Rahman, Jamil Ahmad, Ruhizan Mohammad Yasin, Nurfirdawati Muhamad Hanafi · 2016 · International Journal of Information and Education Technology
  52. Prompt and rater effects in second language writing performance assessment
    Gad S. Lim · 2009 · Deep Blue (University of Michigan)
  53. Assessing Treatment Fidelity in a Cultural Adaptation of Motivational Interviewing
    Christina S. Lee, Tonya Tavares, Ami Popat-Jain, Pamela Naab · 2015 · Journal of Ethnicity in Substance Abuse
  54. Influence of prior acquaintance with the ratee on rater accuracy and halo
    James Robert Van Scotter, Karen South Moustafa, Jennifer R. Burnett, Paul G. Michael · 2007 · Journal of Management Development
  55. Uncertainty-tolerant design: Evaluating task performance and drag-and-link information gathering for a news-writing task
    Simon Attfield, ANN E. BLANDFORD, J.D. Dowell, Paul A. Cairns · 2008 · International Journal of Human-Computer Studies
  56. Beurteilerübereinstimmung und Beurteilerstrenge
    Thomas Eckes · 2004 · Diagnostica
  57. Construct validation of assessment center final dimension ratings: Definition and measurement issues
    Robert G. Jones · 1992 · Human Resource Management Review
  58. AN EVALUATION OF ALTERNATE SCORING METHODS FOR THE MIXED STANDARD SCALE
    Garry L. Hughes, Erich P. Prien · 1986 · Personnel Psychology
  59. Measuring treatment motivation and treatment engagement in forensic psychiatric outpatient treatment: development of two instruments
    Klaus Heinrich Drieschner · 2005 · Radboud Repository (Radboud University)
  60. Measuring and correcting staff variability in large-scale OSCEs
    Skerdi Haviari, Christian de Tymowski, Nelly Burnichon, Cédric Lemogne, Martin Flamant, Philippe B Ruszniewski, Saja Bensaadi, Grégory Mercier, Hasséne Hamaoui, Université Paris Cité OSCE study group et al. · 2024 · BMC Medical Education
  61. Automated low‐contrast pattern recognition algorithm for magnetic resonance image quality assessment
    Morgan O. Ehman, Zhonghao Bao, Scott O. Stiving, Mallikarjunarao Kasam, Dianna Lanners, Teresa Peterson, Renee Jonsgaard, Rickey E. Carter, Kiaran P. McGee · 2017 · Medical Physics
  62. Evaluating the Bookmark Judgments of Standard-Setting Panelists
    George Engelhard · 2011 · Educational and Psychological Measurement
  63. Establishing clinician reliability using the therapy outcome measure for the purpose of benchmarking services
    Alexandra John, Anthony Hughes, Pam Enderby · 2002 · International Journal of Speech-Language Pathology
  64. Congeneric Modeling of Reliability Using Censored Variables
    Roger Brown · 1989 · Applied Psychological Measurement
  65. EXPLORING THE EXISTENCE OF GRADER BIAS IN BEEF GRADING
    Ju Won Jang, Ariun Ishdorj, David P. Anderson, Tsengeg Purevjav, Garland R. Dahlke · 2017 · Journal of Agricultural and Applied Economics
  66. Crowdsourcing for quantifying transcripts: An exploratory study
    Tarek Azzam, Elena Harman · 2015 · Evaluation and Program Planning
  67. Comparison of Automatic and Expert Teachers’ Rating of Computerized English Listening-Speaking Test
    Cao Linlin · 2019 · English Language Teaching
  68. Reconsidering the Equivalence of Multisource Performance Ratings: Evidence for the Importance and Meaning of Rater Factors
    Bethany H. Bynum, Brian James Hoffman, Adam W. Meade, William A. Gentry · 2012 · Journal of Business and Psychology
  69. Monitoring Reader Performance and DRIFT in the AP English Literature and Composition Examination Using Benchmark Essays
    Jonathan R. Manalo, Carol M. Myford, Edward W. Wolfe, George Engelhard · 2007
  70. Examining severity and centrality effects in TestDaF writing and speaking assessments: An extended Bayesian many-facet Rasch analysis
    Eckes, Thomas, Jin, Kuan-Yu · 2021 · International Journal of Testing
  71. Bibliometric and Content Analysis of Sustainable Education in Biology for Promoting Sustainability at Primary and Secondary Schools and in Teacher Education
    Eila Kyllikki Jeronen, Juha Jeronen · 2026 · Education Sciences
  72. A LLTM approach to the examination of teachers' ratings of classroom assessment tasks
    Karen Draney, Mark R. Wilson · 2008 · Psychology science
  73. Exploring the Relationship Between Interrater Correlations and Validity of Peer Ratings
    Ronen Kasten, Baruch Nevo · 2008 · Human Performance
  74. Maximum Likelihood Item Easiness Models for Test Theory Without an Answer Key
    Stephen L. France, William H. Batchelder · 2014 · Educational and Psychological Measurement
  75. Quantifying social skill deficits and strengths profiles in autistic youth
    Jacquelyn A Gates, Alan S. Gerber, C. Miller, Matthew D. Lerner · 2022 · Child Development
  76. Theories, Models, and Standard Systems of Measurement
    Marion S. Aftanas · 1988 · Applied Psychological Measurement
  77. AN EXAMINATION OF THE LINK BETWEEN RATER CALIBRATION PERFORMANCE AND SUBSEQUENT SCORING ACCURACY IN GRADUATE RECORD EXAMINATIONS® (GRE®) WRITING
    Kathryn L. Ricker‐Pedley · 2011 · ETS Research Report Series
  78. Examining Rater Effects in a WDCT Pragmatics Test
    Jianda Liu, Lijun Xie · 2014
  79. Annotating affective dimensions in user-generated content: Comparing the reliability of best–worst scaling, pairwise comparison and rating scales for annotating valence, arousal and dominance
    Luna De Bruyne, Orphée De Clercq, Véronique Hoste · 2021 · Language Resources and Evaluation
  80. Confounding among Measures of Leniency and Halo
    George M. Alliger, Kevin J. Williams · 1989 · Educational and Psychological Measurement
  81. Supervisory Performance Ratings
    David J. Woehr, Sylvia G. Roch · 2012 · Oxford University Press eBooks
  82. EFFECTS OF STANDARD EXTREMITY ON MIXED STANDARD SCALE PERFORMANCE RATINGS1
    Janet L. Barnes-Farrell, Howard M. Weiss · 1984 · Personnel Psychology
  83. An analysis of the uncertainty factors affecting the sustainable supply of rice production in Thailand
    Phatcharee Toghaw Thongrattana · 2012 · Research Online (University of Wollongong)
  84. Design and implementation of an intelligent assessment technology for elementary school students’ scientific argumentation ability
    LI-JIE TIAN, Yunan Ding, Xuewei Tian, Yuqing Chen, Jingying Wang · 2025 · Assessment in Education Principles Policy and Practice
  85. High Stakes Tests with Self-Selected Essay Questions: Addressing Issues of Fairness
    Iasonas Lamprianou · 2008 · International Journal of Testing
  86. Unsupervised consensus analysis for on-line review and questionnaire data
    Stephen L. France, William H. Batchelder · 2014 · Information Sciences
  87. Examining reliability and validity of job analysis survey data.
    Ning Wang · 2003 · PubMed
  88. Permutation Statistical Methods
    Kenneth J. Berry, Janis E. Johnston, Paul W. Mielke · 2018 · Springer eBooks
  89. Investigation of Rater Effects Using Social Network Analysis and Exponential Random Graph Models
    Iasonas Lamprianou · 2017 · Educational and Psychological Measurement
  90. Leniency Bias in Subjective Performance Evaluation: Contextual Uncertainty and Prior Employee Performance
    Na Gong, Wai Fong Boh, Anne Wu, Tsuilin Kuo · 2019 · Emerging Markets Finance and Trade
  91. Gathering evidence on e-rubrics: Perspectives and many facet Rasch analysis of rating behavior
    Inan Deniz Erguvan, Beyza Aksu Dünya · 2021 · International Journal of Assessment Tools in Education
  92. Psychometrics
    Mark R. Wilson, Perman Gochyyev · 2013 · SensePublishers eBooks
  93. Detecting Rater Centrality Effects in Performance Assessments: A Model-Based Comparison of Centrality Indices
    Kuan‐Yu Jin, Thomas Eckes · 2022 · Measurement Interdisciplinary Research and Perspectives
  94. Examination of map reading skills with orienteering activity: An example of Many Facet Rasch Model
    Şeyma Uyar, Onur Yayla, Hidayet ZÜNBER · 2022 · International Journal of Assessment Tools in Education
  95. Evaluation von Beurteilungen:
    Thomas Eckes · 2005 · Zeitschrift für Psychologie
  96. Estimating the Consistency of Third-Party Evaluator Scoring of Organizational Self-Assessments
    Garry D. Coleman, C. Patrick Koelling · 1998 · Quality Management Journal
  97. Interrater Reliability of the Devereux Behavior Rating Scale-School Form: the Influence of Teacher Frame of Reference
    Amanda B. Nickerson, Richard J. Nagle · 2001 · Journal of Psychoeducational Assessment
  98. Development and evaluation of validity scales for the Neuropsychology Behavior and Affect Profile: A dissembling study.
    Paul Satz, Steven G. Holston, Craig Lyons Uchiyama, Grace Shimahara, Maura Mitrushina, David L. Forney, Ken Zaucha, Roger K. Light, Robert F. Asarnow, Charles E. Drebing et al. · 1996 · Psychological Assessment
  99. Interpersonal Behavior, Range of Ratings, and Personal Security
    John R. Hurley · 1986 · Psychological Reports
  100. The Structure of Instruction Rating Scale (SIRS): Development and Technical Characteristics.
    Stanley L. Deno · 1983

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.