Rating the ratings: Assessing the psychometric quality of rating data.

Saal, Frank E., Downey, Ronald G., Lahey, Mary A.·1980·Psychological Bulletin

Cited by (567)

Items 501–567 of 567

  1. Evaluating indicators of job performance: Distributions and types of analyses
    Chambers, Jean-Christophe M. Richard · 2016
  2. The Role of Communication Channel, Feedback Valence and Cultural Differences in Performance Appraisal Outcomes
    James C. Matchen · 2007 · IRL - University of Missouri, St. Louis (University of Missouri–St. Louis)
  3. Psychometric Characteristics of the Behavioral Observation Scale
    Gregg Allen Bolt · 1984
  4. Eötvös Loránd Tudományegyetem.
    Enrico Geva · 2017 · Biographies | European Encyclopedia of Law
  5. Effectiveness of the Marine Corps' Junior Enlisted Performance Evaluation System: An Evaluation of Proficiency and Conduct Marks
    Richard Jr B Larger · 2017 · Calhoun: The Naval Postgraduate School Institutional Archive (Naval Postgraduate School)
  6. Analysing Severity Data When Assessors Differ in Their Usage of the Categories
    Timothy Paul Hutchinson · 1985 · Journal of the Royal Statistical Society Series D (The Statistician)
  7. Reliability assessment of torque measures during a unilateral pressing movement.
    Brett Harris · 2018 · Otago University Research Archive (University of Otago)
  8. An investigation of a method for validating individual raters of performance and its implications for a generalized rating ability
    Jamie J. Carlyle · 1982 · VTechWorks (Virginia Tech)
  9. Tecnologías de la información para medir socialmente la creatividad
    Carlos Guillem Aldave · 2018 · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
  10. Graduate Student Self and Adviser Ratings on Professional Competencies
    Bret Arnold, Alison J. Kerr, Bradley James Brummel, Michael Keller · 2020 · ASEE Annual Conference & Exposition
  11. Evaluating assessment in a School of Economics
    Steven Barrett, Francisco Ben · 2018 · Journal of Applied Learning & Teaching
  12. The mixed method for development of preliminary DanceSport judging scale
    EungJoon Kim, Jinoh Jeong · 2008 · The Korean Journal of Measurement and Evaluation in Physical Education and Sports Science
  13. Validation of 360-Degree Feedback Assessments
    Jana Mahlke · 2019 · Refubium (Universitätsbibliothek der Freien Universität Berlin)
  14. Monte Carlo examination of common approaches for detecting response styles
    Qianqian Fan · 2019
  15. Sidecar: Augmenting Word Embedding Models with Expert Knowledge
    Mathieu Lemay, Daniel Shapiro, Mary Kate MacPherson, Kieran Yee, Hamza Qassoud, Miodrag Bolić · 2020 · Advances in intelligent systems and computing
  16. An Analysis of Teacher Decision-Making in Grading 10th Grade Student Writing in English Language Arts
    Guy Swenson · 2020 · Journal of International Crisis and Risk Communication Research
  17. Supervisor competence as an assessor of medical trainees: evaluating the validity and quality of supervisor assessments
    McGill Darryl Andrew · 2019 · Research Publications (Maastricht University)
  18. Modeling Rater Effects and Complex Learning Progressions using Item Response Models
    Hyo Jeong Shin · 2015 · eScholarship (California Digital Library)
  19. Studies of Rater and Item Effects in Rater Models
    Yihan Zhao · 2020
  20. How Performance Ratings are Influenced by Objective and Subjective Measures: Evidence from Public Enterprises in Korea
    Tae Sik Ahn, Myung‐In Kim · 2006 · SSRN Electronic Journal
  21. Employee Performance Appraisal Satisfaction
    Dao Hung · 2019 · Seoul National University Open Repository (Seoul National University)
  22. The Analysis of Frequency Terms Used in Rating Scales
    Seock-Ho Kim1, Hyo Jin Eom2 · 2021 · American Journal of Educational Research and Reviews
  23. Observer Ratings
    John Richard Wittenborn · 1984 · Springer eBooks
  24. Nominal-Level Variables, I
    Kenneth J. Berry, Janis E. Johnston, Paul W. Mielke · 2018 · Springer eBooks
  25. Randomized Designs: Ordinal Data, I
    Kenneth J. Berry, Paul W. Mielke, Janis E. Johnston · 2016 · Springer eBooks
  26. Introduction
    Kenneth J. Berry, Paul W. Mielke, Janis E. Johnston · 2016 · Springer eBooks
  27. Randomized Block Designs: Nominal Data
    Kenneth J. Berry, Paul W. Mielke, Janis E. Johnston · 2016 · Springer eBooks
  28. REFERENCES
    1994 · Elsevier eBooks
  29. Obtaining Data
    Francesco Chiappelli · 2013 · Springer eBooks
  30. The Analysis of Frequency Terms Used in Rating Scales
    Seock‐Ho Kim · 2019 · American Educational Research Association Annual Meeting (AERA)
  31. การพัฒนาแบบประเมินรายงานกรณีศึกษาผู้ป่วยและครอบครัวที่มีปัญหาสุขภาพซับซ้อนที่พบในเวชปฏิบัติปฐมภูมิ
    จิรวดี ชุมพล, ศศิธร กาญจนสุวรรณ, ธนภรณ์ แดงใจ, ปณิธี พูนเพชรรัตน์ · 2022 · Ramathibodi Medical Journal
  32. การวิเคราะห์ฉันทามติและการทำหน้าที่ต่างกันระหว่างผู้ประเมินในการวิเคราะห์ความสอดคล้องในแนวเดียวกัน: การประยุกต์ใช้โมเดลการวิเคราะห์ฉันทามติเชิงวัฒนธรรม
    ศิธรา จุฑารัตน์ · 2019 · Chula Digiverse
  33. Statistical Methods
    Kenneth J. Berry, Janis E. Johnston · 2023 · Springer eBooks
  34. A Feedforward Interactive Network to Explain Complex Information Processing Phenomena Including the Halo Effect and Its Generalization
    Irwin D. Nahinsky · 2023 · The American Journal of Psychology
  35. A Review of the Halo Effect and Its Applications at the Corporate Level
    Yuanhang Yao · 2023 · Advances in Economics Management and Political Sciences
  36. Reciprocity in Peer Assessments
    Lunzheng Li, Philippos Louis, Zacharias Maniadis, Dimitrios Xefteris · 2026 · AEA Randomized Controlled Trials
  37. Reassigning sources of misophonic trigger sounds to change their unpleasantness: Testing alternative mechanisms with a new set of movies, paintings, and words
    Laurie M. Heller, Urszula Oszczapinska, Jessica M. Smith, Megan M. Julien · 2025 · OSF Preprints (OSF Preprints)
  38. Permutation Statistical Methods
    Kenneth J. Berry, Janis E. Johnston, Michael A. Long, Paul B. Stretesky, Michael J. Lynch · 2025 · Society, environment and statistics
  39. การเปรียบเทียบความเที่ยงของแบบวัดทักษะการสื่อสารทางคณิตศาสตร์ด้านการเขียนของนักเรียน ชั้นประถมศึกษาปีที่ 6 : การประยุกต์ใช้ทฤษฎีการสรุปอ้างอิงความน่าเชื่อถือของผลการวัด
    มณชญา สักกามาศ, กมลทิพย์ ศรีหาเศษ, ดารุณี ทิพยกุลไพโรจน์ · 2025 · Interdisciplinary Academic and Research Journal
  40. References
    2000 · Emerald Group Publishing Limited eBooks
  41. Overview and Evaluation of Supervisory Performance Ratings
    Sylvia G. Roch, David J. Woehr · 2025 · Oxford University Press eBooks
  42. The Problem of Subjective Intensity
    Lennart Sjöberg · 1984 · Advances in psychology
  43. The Effect of Mindfulness-Based Programmes on Work Performance
    Maris Vainre · 2023 · Apollo (University of Cambridge)
  44. Additional file 2 of A study of validity and usability evidence for non-technical skills assessment tools in simulated adult resuscitation scenarios
    Higham, Helen, Paul Robert Greig, Crabtree, Nick, George Hadjipavlou, Young, Duncan, Charles A. Vincent · 2024 · Figshare
  45. Additional file 3 of A study of validity and usability evidence for non-technical skills assessment tools in simulated adult resuscitation scenarios
    Higham, Helen, Greig, Paul, Crabtree, Nick, George Hadjipavlou, Young, Duncan, Charles A. Vincent · 2024 · Figshare
  46. Additional file 1 of A study of validity and usability evidence for non-technical skills assessment tools in simulated adult resuscitation scenarios
    Higham, Helen, Paul Robert Greig, Crabtree, Nick, George Hadjipavlou, Young, Duncan, Charles A. Vincent · 2024 · Figshare
  47. Thesis examining the accuracy of witness statements for real world criminal events
    Lilley, Alexandra · 2024 · University of Birmingham Institutional Research Archive (University of Birmingham)
  48. A Monte Carlo Evaluation of Growth Mixture Models: Effects of Varying Distributional Parameters on Grouping Outcomes
    Tiffany M. Shader · 2019 · OhioLink ETD Center (Ohio Library and Information Network)
  49. The effect of raters' prior knowledge of students' proficiency levels on their assessment during oral interviews
    Fatma TANRIVERDİ-KÖKSAL · 2013 · Bilkent University Institutional Repository (Bilkent University)
  50. The Software Adoption Process Seen Through System Logs
    Áslaug Eiríksdóttir · 2014 · Skemman
  51. Interobserver variability in assessing anterior cruciate ligament anatomy: comparing gross and microanatomical observations
    Elizabeth Reschenberg, Aidan A. Ruth · 2026 · Surgical and Radiologic Anatomy
  52. Motivations for Teaching: Extension of the FIT-Choice Scale and Validation in Cuba
    Martina King, Anna M. Eder-Jahn · 2026 · Journal of Latinos and Education
  53. Explainable Artificial Intelligence in Conceptual Design: An Large Language Model–Powered Framework to Enhance Originality and Credibility
    Iván Felipe Velasco Medina, Tamotsu MURAKAMI · 2026 · Journal of Mechanical Design
  54. References
    2026 · Emerald Group Publishing Limited eBooks
  55. Rater-Mediated Assessments with Explanatory Rasch Models
    George Engelhard, Stefanie A. Wind · 2026 · Springer eBooks
  56. Bibliometric and Content Analysis of Sustainability Education in Geography for Promoting Sustainable Development at Primary and Secondary Schools and in Teacher Education
    Eila Kyllikki Jeronen, Juha Jeronen · 2026 · International Journal of Educational Studies
  57. The Relationships Between Rating Adoption Rates, Rater Severity, and Raters’ Characteristics: A Multilevel Analysis
    Jiaqi Yang, Yingbin Zhang, Yehui Wang, Yuze Deng, Yimei Zhang · 2026 · SAGE Open
  58. When Cut Scores Impact Human Ratings: An Extended Many‐Facet Rasch Model for Category‐Specific Rater Severity Shifts
    Kuan‐Yu Jin, Thomas Eckes · 2026 · Journal of Educational Measurement
  59. Evaluating AI-Based Automated Essay Scoring Through Signal Detection Theory: Beyond Aggregate Agreement Metrics
    Xiaoliang Zhou · 2026 · Applied Psychological Measurement
  60. Matematik Dersindeki Açık Uçlu Soruların Puanlanmasında Puanlayıcı Eğitiminin Puanlayıcı Davranışlarına Etkisi
    Umur Öç, İsmail Karakaya · 2026 · Türk Eğitim Bilimleri Dergisi
  61. Exploring the Impact of AI-powered Creativity Support Tools on Professional Creative Workflows
    Caterina Moruzzi, Charlotte Bird, Laura Mariah Herman · 2026 · Proceedings of the ACM on Human-Computer Interaction
  62. RATER SEVERITY IN MEASURING TEACHERS’ COMPETENCY IN CLASSROOM ASSESSMENT
    Rosyafinaz Mohamat, Harris Shah Abd Hamid, Bambang Sumintono · 2022 · Journal of Islamic Social Economics and Development
  63. What do willingness and involvement ratings actually measure? Evidence from L2 speech and writing assessment
    Yiran Du · 2026 · Applied Corpus Linguistics
  64. Do Spanish Affective Picture Norms Transfer to Russian Speakers? Valence and Arousal Ratings of a Low-Level Matched Subset of the EmoMadrid Database
    Alexander Kirsanov, Anton Karapuzov, Elena Levchenko, Taisia Loginova, Daria Marochko, Evgeny Blagoveshchensky · 2026 · Applied Sciences
  65. When Is Inference About the Relationship Between Self–Other Agreement and Leader–Member Exchange Warranted? Rating Discriminability as a Boundary Condition on Inference
    Takashi Hongo, Dai Senoo · 2026 · Administrative Sciences
  66. Advancing Language and Literacy Assessments through Technological Integrations
    Melissa R. Hunte · 2024 · TSpace (University of Toronto)
  67. Modele analizy efektu oceniającego w pomiarze edukacyjnym
    Filip Kulon · 2014 · Jagiellonian Digital Library (Jagiellonian University)

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.