PORTAAL: A Classroom Observation Tool Assessing Evidence-Based Teaching Practices for Active Learning in Large Science, Technology, Engineering, and Mathematics Classes

“For example, when asked to define active learning, faculty members in one study offered a range of answers, including simply “using clickers” or “group work” to “emphasizing higher-order skills” (Freeman et al., 2014).”

“In addition to the amount of practice, the quality and distribution of the practice is also important (Ericsson et al., 1993).”

“The resulting ICCs were all > 0.8, and thus all were in the “excellent” range (Supplemental Table 2; Cicchetti 1994).”

“The resulting ICCs were all > 0.8, and thus all were in the “excellent” range (Supplemental Table 2; Cicchetti 1994).”

“One of the most popular definitions of active learning comes from Hake (1998): “those [practices] designed at least in part to promote conceptual understanding through interactive engagement of students in heads-on (always) and hands-on (usually) activities which yield immediate feedback through…”

“To provide students with opportunities to practice their logic development, it is necessary for instructors to formulate questions that require a higher level of thinking (PORTAAL element logic development 1, L1; Morris et al., 1977; Ericsson et al.,1993; Jensen et al., 2014).”

“At the college level, an intervention that had students reflect on a situation in their own life in which hard work helped them improve also increased their achievement in their classes (Aronson et al., 2002).”

“Praise that focuses on effort encourages a mastery goal and can increase student willingness to participate in challenging tasks (R6; Mueller and Dweck, 1998).”

“Examples include the Reformed Teaching Observation Protocol (RTOP; Sawada et al., 2002), the Teaching Dimensions Observation Protocol (Hora et al., 2013), the Classroom Observation Protocol for Undergraduate STEM (COPUS; Smith et al., 2013), the Practical Observation Rubric to Assess Active Learning (Eddy et al., 2015…”

“There are many different ways to enact active learning, but typically active learning includes students working with other students during class and more frequent assessment of student learning in the classroom (Eddy et al. 2015b; Freeman et al. 2014).”

“…can reduce bias with regard to who is called on in college science courses; in biology courses in which random call is not practiced and students volunteer to answer questions, only 40% of female voices are heard, even when females make up 60% of the class (Eddy et al., 2014, 2015; Tanner, 2013).”

“Second, collaborations between researchers and instructors could help increase the implementation of evidence-based teaching practices by instructors/faculty who are not education researchers and would benefit from support while shifting their instructional approaches (Eddy et al., 2015).”

“…a growing interest in understanding how undergraduate anxiety may impact motivation, performance, and persistence, particularly for demographic subsets of students (Bledsoe and Baskin, 2014; Eddy et al., 2014, 2015a,b; Broeckelman-Post et al., 2016; Cooper and Brownell, 2016; Cooper et al., 2018).”

“…engagement between two instructors; paired with additional classroom observation data about instructor habits (for example from a tool like PORTAAL [Eddy et al., 2015] or COPUS [Smith et al., 2013]), instructors may be able to enhance their own soft skills to increase engagement in their own…”

“…definition for the BER community; the fact that active learning is a complex process encompassing both teaching methods and student learning (Eddy et al., 2015); the dichotomous nature of the term as both a theory and as a set of pedagogical strategies (Drew and Mackie, 2011); the…”

“…Protocol (TDOP), Reformed Teaching Observation Protocol (RTOP), Classroom Observation Protocol for Undergraduate STEM (COPUS), Practical Observation Rubric To Assess Active Learning (PORTAAL)] (11 – 14) require trained human observers and are not feasible for addressing questions at this scale.”

“A literature review revealed that about half of the instructional practices demonstrated to improve student achievement in active-learning instruction in large undergraduate STEM courses focused on student motivation (Eddy et al. 2015).”

“These studies should include inspection of finer-grain analyses of student intersectionality/identity, as well as a better understanding of the diverse relationships within working groups (Eddy et al., 2015).”

“There are also recent observation protocols that would indicate whether the GTA PD is being taught using current best practices in science teaching (e.g., Smith et al., 2013; Eddy et al., 2015).”

“A study that recently investigated the teaching strategies of the classrooms used in our study documented that instructional practices varied between instructors, ranging from almost entirely traditional lecture to more student-centered interactive methods (Eddy et al., 2015).”

“…to relevant literature to inform our thinking about conceptual relationships among codes, including literature on active-learning instruction (e.g., Eddy et al., 2015), student motivation (e.g., Glynn et al., 2009), cognitive science (e.g., Vermunt, 1996), and course climate (e.g., Ambrose et al.,…”

“In particular, such Instructor Talk sampling would complement information being collected with currently available classroom observation tools (e.g., Reformed Teaching Observation Protocol, Teaching Dimensions Observation Protocol, Classroom Observation Protocol for Undergraduate STEM, Practical Observation Rubric To…”

“…the focus of their pedagogies from preparing students to pass exams to preparing them to be life-long learners who are able to think critically, collaborate, solve problems, and communicate effectively (Conference Board of Mathematical Sciences, 2016; Eddy et al., 2015; Maron, 2016; Pienta, 2015).”

“These results corroborate the improvement in science education through active methodologies already described by many authors in the last few decades [41,42].”

“…using additional approaches, such as analyzing the instructor talk exhibited (Seidel et al., 2015), the 21 specific practices documented in the PORTAAL (Eddy et al., 2015), or the participation patterns in classroom discourse provided by Equity Quantified in Participation (Reinholz and Shah, 2018).”

“In the past decade, the biology education research community has identified a number of evidence-based instructional practices (Freeman et al., 2007, 2011; Handelsman et al., 2007; Bradforth et al., 2015; Eddy et al., 2015; James and Singer, 2016; Jordt et al., 2017).”

“The Practical Observation Rubric to Assess Active Learning (PORTAAL) is used by observers to document the frequency and duration of class activities that employ specific active-learning techniques documented to improve student learning (Eddy et al., 2015).”

“Although there are many different aspects of active learning, important components with demonstrated positive effects on student learning have recently been reviewed (Eddy, Converse, and Wenderoth 2015).”

“The Ignored event could have been mitigated if the instructor had been cautious with praise and not judged student responses (Aronson et al., 2002; Tanner, 2013; Eddy et al., 2015).”

“We adapted our definitions from the biology education literature (e.g., Freeman et al., 2014; Eddy, Brownell, et al., 2015; Eddy et al., 2015).”

“These efforts have led to the development of multiple classroom observation instruments (for a review, see Hora and Ferrare, 2012), such as the Reformed Teaching Observation Protocol (i.e., RTOP; Sawada et al., 2002), Practical Observation Rubric to Assess Active Learning (Eddy et al., 2015), Teaching Dimensions Obser…”

“, 2015) and the Faculty Practice/Faculty Support rubric (D’Avanzo, 2013; Smith et al., 2013; Wieman and Gilbert, 2014; Eddy et al., 2015).”

“First, it is possible that additional or other approaches to active learning would have been more effective than those used in this study; future research should be directed at determining what types of activities are most effective in eliciting active learning (Andrews et al., 2011; Freeman et al., 2014; Eddy et al.,…”

“Several classroom observation tools have been designed to measure student (and instructor) participation and behavior in the classroom (Sawada et al., 2002; Hora and Ferrare, 2010; Smith et al., 2013; Eddy et al., 2015; Lane and Harris, 2015).”

“The changes in course design that occurred over the three experiments (Table 1) are backed by data from a classroom observation tool [11] (S3 Text).”

“, teaching practices inventories) could be used to give faculty the tools to support student buy-in of evidence-based teaching approaches and to be successful in implementing these teaching models (Wieman and Gilbert, 2014; Eddy et al., 2015).”

“random to participate in a multiple-choice test on the course topic provided on the online platform Moodle, to incorporate a typical active learning assessment-component into the online teaching concept (Freeman et al., 2014; Eddy et al., 2015).”

“Other tools could also be useful for investigating the implementation and fidelity of evidence-based practices as a result of co-teaching experiences (e.g., Crowe et al., 2008; Eddy et al., 2015; Tekkumru‐Kisa et al., 2015; Arneson and Offerdahl, 2018; Kranzfelder et al., 2019).”

“…method [26], the Teaching Dimensions Ob-servation Protocol (TDOP) [12], the Practical Observation Rubric To Assess Active Learning (PORTAAL) [7], the Classroom Assessment Scoring System (CLASS) [29], and the Classroom Observation Protocol for Undergraduate STEM (COPUS) [34]; where observers…”

“Eddy et al. (2015) found that students explaining their answers to peers, hearing other students describe their logic, and participating in activities that require group work are all elements of best practices in AL.”

“The possibility of biased PRT has been stressed and can be countered by conducting several independent class observations or by involving multiple reviewers (Eddy et al., 2015).”

“In either case, the large difference in the fractions of more engaged students in these two classes calls for the use of more observation codes when active learning occurs such as the 21 elements used in the PORTAAL classroom observation tool (Eddy et al. 2015).”

“Other disadvantages include lack of collaboration and teamwork (key International Engineering Alliance and Engineering Council of South Africa qualification standard graduate attributes [9,14]) as well as the risk of plagiarism during the assessment, which may misrepresent the benefits of virtual experimentation [8].”

“The approach was purely directed to student self-directed learning (SDL) with an all-inclusive scheme to direct the online activities to student academic improvement (Darmawan & Hidayah, 2017; Eddy et al., 2015).”

“Many factors contribute to students’ success in their coursework, including but not limited to scaffolding of course material, homework assignments, students’ motivation, and family responsibilities (Eddy et al., 2015a).”

“Obviously, the study conducted by Eddy, Converse, and Wenderoth (2015) emphasized that learners’ participation can be taken to a higher level by feeling the sense of responsibility during grasping of their own knowledge and skills.”

“…necessary to combine these resources with those of the students: previous school experience and their own personality (dotted arrows) (modi fi ed from Eddy et al. 2015). for molecular biology students (Laneuville & Sikora 2015), suggesting that this kind of learning activity is not…”

“Eddy, Converse, and Wenderoth (2015) created an observational tool for capturing the presence, absence, and frequency of best practices for active learning, although it is specific to large ALCs (i.e., 50–700 students) in STEM domains.”

“Error framing is proposed in the literature as a pedagogical tool to reduce student apprehension (Eddy et al., 2015) and anxiety (Cooper et al.”

“Although other instruments exist, such as the Classroom Observation Protocol for Undergraduate STEM (COPUS) [43] and the Practice Observation Rubric to Assess Active Learning (PORTAAL), [44] the TPI provides a relatively simple, frequency-based system for characterizing research-based teaching practices with little to…”

“The training guide provided by COPUS and PORTAAL (Smith et al., 2013; Eddy et al., 2015) was also reviewed.”

“While activity within lectures is varied by lecturer, the majority of lecture time is predominantly the lecturer talking (Smith et al., 2013b; Eddy et al., 2015; Kinnear et al., 2021).”

“…the volume and variance of classroom audio recordings to estimate the percentage of time spent in active learning (56); and the Practical Observation Rubric To Assess Active Learning, which measures the minutes students could spend in thinking or discussion as a component of its score (57).”

“This increase in interaction, based on the evidence-based active learning strategy [44,45], was accompanied by the requirement to turn on the students’ cameras to enable a more personal learning atmosphere.”

“For example, Eddy et al. (2015) proposed that active learning typically involves students collaborating with peers in the classroom and engaging in mutual knowledge evaluation.”

“…tools are the Teaching Dimen - sions Observation Protocol (TDOP) [ 6 ], Practical Observa - tion Rubric To Assess Active Learning (PORTAAL) [7], the Classroom Assessment Scoring System (CLASS) [8], and the Classroom Observation Protocol for Undergradu - ate STEM (COPUS) [9] where observers…”

“Other tools, such as the Practical Observation Rubric to Assess Active Learning (PORTAAL) (Eddy et al., 2015) and the Teaching Dimensions Observation Protocol (TDOP) (Hora, 2015; Hora & Ferrare, 2014) were designed to measure the types and prevalence of active learning instructional strategies in…”

“Traditionally, knowing how class time is spent requires bringing in a trained observer to hand-annotate during lecture, for example, using one of the popular classroom annotation schemes (Smith et al. 2013; Sawada et al. 2002; Velasco et al. 2016; Eddy, Converse, and Wenderoth 2015).”

“…(e.g., Dasgupta et al., 2014), surveys (e.g., Glynn et al., 2011; Hanauer and Dolan, 2014), concept inventories (e.g., Anderson et al., 2002; Price et al., 2014; Couch et al., 2015), or observation protocols (e.g., Smith et al., 2013; Eddy et al., 2015), for use by the biology education community.”

“…in specific disciplines for traditional face to face classroom, for example, the Practical Observation Rubric to Assess Active Learning (PORTAAL; Eddy et al., 2015), The Classroom Observation Protocol for Undergraduate STEM (COPUS; Smith et al., 2013), Peer Review of Teaching (PRoT) program for…”

“In recent years, “Active Learning (AL),” the teaching method which puts more emphasis on students’ active participation in class and their abilities to discover problems and solve them, has been coming under the spotlight worldwide [1][2][3][4][5][6].”

“It is probable that students would actively participate in their studies, contribute to debates, and work collaboratively with peers when they feel respected, encouraged, and included (Eddy et al., 2020).”