PORTAAL: A Classroom Observation Tool Assessing Evidence-Based Teaching Practices for Active Learning in Large Science, Technology, Engineering, and Mathematics Classes

“For example, when asked to define active learning, faculty members in one study offered a range of answers, including simply “using clickers” or “group work” to “emphasizing higher-order skills” (Freeman et al., 2014).”

“In addition to the amount of practice, the quality and distribution of the practice is also important (Ericsson et al., 1993).”

“The resulting ICCs were all > 0.8, and thus all were in the “excellent” range (Supplemental Table 2; Cicchetti 1994).”

“The resulting ICCs were all > 0.8, and thus all were in the “excellent” range (Supplemental Table 2; Cicchetti 1994).”

“One of the most popular definitions of active learning comes from Hake (1998): “those [practices] designed at least in part to promote conceptual understanding through interactive engagement of students in heads-on (always) and hands-on (usually) activities which yield immediate feedback through…”

“To provide students with opportunities to practice their logic development, it is necessary for instructors to formulate questions that require a higher level of thinking (PORTAAL element logic development 1, L1; Morris et al., 1977; Ericsson et al.,1993; Jensen et al., 2014).”

“At the college level, an intervention that had students reflect on a situation in their own life in which hard work helped them improve also increased their achievement in their classes (Aronson et al., 2002).”

“Praise that focuses on effort encourages a mastery goal and can increase student willingness to participate in challenging tasks (R6; Mueller and Dweck, 1998).”

“However, in the same studies, instructors were rarely observed to use error framing [1,13].”

“This rubric seems very helpful for understanding the faculty perspective in the classroom but lacks the connection of student behaviors to instructor actions (17).”

“…(Classroom Observation Protocol for Undergraduate STEM; Smith et al., 2013), the PORTAAL (Practical Observation Rubric to Assess Active Learning; Eddy et al., 2015) and the RTOP (Reformed Teaching Observation Protocol; Sawada et al., 2000), each of which are summarized along with other…”

“Eddy et al. (2015) identified several barriers to adoption, including clarity, the overwhelming number of aspects that must be considered, and the time required for implementation.”

“Within undergraduate STEM education alone, protocols include DART (6), RTOP (7), RIOT (8), PORTAAL (9), TA-IOP (10), and COPUS (11).”

“For example, an instructor who unknowingly asks more questions of students sitting on the middle-right side of the classroom may increase students’ apprehension and decrease the utility of that practice for those students (Cooper et al., 2018; Eddy et al., 2015; England et al., 2017).”

“One example is the Practical Observation Rubric to Assess Active Learning [5].”

“The PORTAAL (Practical Observation Rubric To Assess Active Learning) tool was designed based on a review of education research literature to identify best practices in active learning [8].”

“Research giving students more opportunities to practice improves student learning (provided that the practice activities are related to tasks that the students are expected to perform)[5].”

“In recent years, many scholars have developed new instruments, especially classroom observation protocols (e.g., Eddy et al., 2015; Hora, 2015; Hora & Ferrare, 2013; M. K. Smith et al., 2013) and faculty surveys (e.g., Landrum et al., 2017; Walter et al., 2016), with the intent to probe instructor…”

“Particularly in group work, other studies have shown that identities, including race and ethnicity, religion, nationality, political values, belonging to the LGBTQIA +community, and gender, may affect how students perceive active learning (77–80).”

“Existing classroom instruments useful for observing teaching practices are not designed to measure instructor humor (48–51).”

“They may also require time outside of the class period; for example, developers of the PORTAAL recommend that the class be videotaped and scored later for more reliable results (Eddy et al., 2015).”

“The perceived variation on how active learning was applied during classes was not unique to this study nor to the remote emergent teaching period [24, 28, 29].”

“Eddy, Converse, and Wenderoth (2015), who worked with STEM students, consider learning from failure a key element of active learning.”

“Furthermore, Eddy, Converse, and Wenderoth [8] discuss how these strategies developed to encourage active learning need to acknowledge the day-to-day life of faculty as well as potential barriers described in the literature, such as the limited effort to train faculty members on teaching methods.”

“Each element is “supported by at least one published peer-reviewed article that demonstrates its impact on a relevant student outcome” (Eddy et al. 2015, p. 3).”

“…the initial BIS Framework—drawing on research and existing frameworks for research-based and equity-centered practice (e.g., Beane et al., 2019; Eddy et al., 2015; Kober, 2015; Kuh, 2008; Labov et al., 2009; National Research Council [NRC], 2012; Sathy & Hogan, 2022; Sawada et al., 2002;…”

“In biology education, assessment tools appeared to play an important role in evaluating the effectiveness of three-dimensional learning reforms [25], while also assisting teachers in identifying strengths and areas for development in teaching practice [26].”

“Each of the 10 PORTAAL practices when used alone is documented in the literature to support student learning [41].”

“Also, Eddy et al. (2015) observed that students perceive genetics as highly relevant when it is framed within societal and technological contexts.”

“Wenderoth (2015) posit that hands-on learning brings laboratory tasks to classrooms and vice versa, where students are involved in using, testing materials and observing the working modules.”

“Most commonly, classroom observations [6], [7], [8] or retrospective surveys [9], [10], are used to evaluate instructional practices.”

“…et al. 2015), Finally, providing students with an in-class practice that is similar in rigor to course exams appears to improve student achievement (Eddy, Converse, and Wenderoth 2015; Jensen et al. 2014), and it would speak to Instructor 1’s worry of students that do minimal practice on their own…”

“Random call (RC) is a specific practice of calling on individual students who have not volunteered (Eddy et al., 2015; Hood et al., 2021).”

“Error framing can occur when an instructor is engaged with a student or as part of the introduction to an activity [6].”

“Yet defining the critical components of a particular active-learning practice is often not straightforward (Eddy et al., 2015).”

“Practical Observation Rubric to Assess Active Learning (pORTAAL) was used to evaluate the promoting effect of teacher behavior on students' active learning in STEM (Science, Technology, Engineering, Mathematics) classroom in higher education [12].”

“113 Classroom observation protocols 114 An array of tools have been developed over the past two decades to measure active 115 teaching pedagogies, especially in STEM courses (Eddy et al., 2015; Owens et al., 2017; Sawada 116 et al., 2002; Smith et al., 2013).”

“Active learning in STEM instruction Recent evidence for the effectiveness of active learning interventions has instigated widespread interest in implementing active learning in STEM courses, instead of passive lectures (Eddy et al., 2015).”

“In engineering education, additional data from instruction observation on active learning and student-center practices can be used, such as Classroom Observation Protocol for Undergraduate STEM (COPUS) (Smith et al., 2013), Reformed Teaching Observation Protocol (RTOP) (Piburn et al., 2000), Teaching Dimensions Observ…”