OBJECTIVE: To measure the reliability and preliminary validity of a grading instrument for editors to evaluate the quality of peer reviews. DESIGN: The consecutive sample design included 53 reviews of 23 manuscripts. Reviews were systematically assigned to interrater reliability (n = 41; power gr…