Emergency remote teaching and students’ academic performance in higher education during the COVID-19 pandemic: A case study

May combine two claims.

There was a relatively improved performance associated with more advanced courses, but the improvement was not significant.

“Fig. 12 (left) shows a relatively improved performance associated with more advanced courses; however, the analysis finds that this improvement is not significant.”

May combine two claims.

Nearly 60% of coordinators and instructors chose synchronous teaching, mostly through videoconferencing tools; this rose to two thirds when only core courses are considered.

“nearly 60 percent of coordinators and instructors chose to continue their classes using synchronous teaching, mostly through videoconferencing tools –this percentage rose to two thirds of the courses when only core courses are considered”

Classified as probably not a substantial claim.

Instructors had barely a week to prepare the move to online teaching.

“instructors barely had a week to prepare the move to online teaching”

May combine two claims.

First-semester courses, all face-to-face, had stable passing rates around 80% with no statistical differences across the three years.

“The analysis (Fig. 9 ) shows stability in passing rates (around 80 percent) and no statistical differences across all three years”

May combine two claims.

All courses used the institutional LMS (Moodle) as support for content delivery and assignments; two courses did not use any tools other than Moodle.

“all the courses used the institutional LMS (Moodle) as support for content delivery and assignments; two courses did not use any tools other than Moodle”

Probably not a substantial claim: may not pass S3 (load-bearing).

The transition to emergency remote teaching posed challenges in scaling the LMS to ensure quality of service for higher numbers of concurrent users.

“even though the transition to emergency remote teaching posed some challenges in scaling the system to ensure quality of service for higher number of concurrent users”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

There is a balance between small-medium and large courses, and more instructors chose synchronous teaching than asynchronous.

“there is a balance between small-medium and large courses, and a slight difference in the delivery mode, with more instructors choosing to adopt synchronous teaching”

May combine two claims.

The School has a strong technical orientation; most instructors are technology experts and use synchronous/asynchronous communication tools and learning virtual spaces daily.

“its strong technical orientation; because the School specializes in information and communication technologies and systems, most instructors are technology experts and use synchronous/asynchronous communication tools and learning virtual spaces on a daily basis”

Probably not a substantial claim: may not pass S3 (load-bearing).

Elective courses are typically small to medium in class size.

“another characteristic of elective courses is that they typically may be considered small to medium regarding class size”

May combine two claims.

The degree of diversification in activities and processes was relatively low: clients and services remained the same, only technologies changed.

“the degree of diversification in the activities and processes has been relatively low: the clients served (students) and services delivered (teaching) remained the same, while only the technologies employed changed”

May combine two claims.

The School's Board of Directors established necessary informal communication channels in early March in anticipation for the crisis, helping prepare possible responses.

“flexibility was further enabled by the School's Board of Directors, which established the necessary informal communication channels in early March in anticipation for the crisis, and therefore helped prepare possible responses”

May combine two claims.

The number of students with slightly above average or outstanding performance did not change significantly, but the number with very good performance (7 to 10 points excluding outstanding) did vary significantly.

“the number of students that had slightly above average or outstanding performance did not change significatively, but the number of students that had a very good performance (from 7 to 10 points out of 10, excluding outstanding students) did vary significantly”

May combine two claims. The quote only partly states this claim.

Proctoring was restricted due to privacy issues, despite efforts to prevent cheating and use of plagiarism detection software.

“proctoring was restricted due to privacy issues”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A fully functional instance of Moodle has been in place for several years, and most courses regularly use their Moodle virtual spaces at least as educational content repository and asynchronous communication channel.

“a fully functional instance of Moodle has been in place for several years now, and most courses regularly use their Moodle virtual spaces at least as educational content repository and asynchronous communication channel with students via message boards”

May combine two claims.

Differences in variations of passing grades cannot be attributed to the choice of software application, except for Webex, where differences are significant.

“The results of the analyses suggest that differences in variations of passing grades cannot be attributed to the choice of one software application or another, except in the case of Webex, where the differences are significant.”

May combine two claims.

When electiveness is accounted for, there were no significant differences across elective courses, but differences were sustained across non-elective core courses.

“the analysis reveals no significant differences across elective courses, while the differences are sustained across non-elective (core) courses”

The quote only partly states this claim.

In the second semester of 2019-20, all elective courses except two had a 100% pass rate.

“From Fig. 2, every elective course except for two achieve a 100 percent pass rate”

Probably not a substantial claim: may not pass S3 (load-bearing).

Instructors perceive that students find it useful to be able to revise the content of the session at a later moment.

“Instructors also perceive that students find it useful to be able to revise the content of the session at a later moment”

Probably not a substantial claim: may not pass S3 (load-bearing).

Coordinators find value in online learning, especially from the students' perspective.

“The coordinators find value in online learning, especially from the students' perspective.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Three coordinators stated that online learning cannot replace face-to-face instruction.

“and 3 coordinators stated that online learning cannot be a replacement for face-to-face instruction.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Organic organizations based on informal relations and decentralized communication are better able to cope with changes.

“A final important aspect to consider is the ability of an organization to foster the existence of an organic organization, based on informal relations and where communication is diffused among members in a non-centralized way, as such kind of organizations have higher capabilities to cope with changes (Knowles & Saxberg, 1988).”

• cites Knowles & Saxberg, 1988Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For higher education institutions during the COVID-19 outbreak, the first condition is satisfied, while the second depends on instructors' decisions.

“In the case of higher education institutions and the COVID-19 outbreak, the first condition is met, whereas the second depends on the decisions made by the instructors.”

May combine two claims.

Some options such as breadth or modality are forced by the pandemic and cannot be chosen, but instructors retain some freedom in deciding the remaining options.

“Admittedly, some of these options, such as breadth (whole program, course, portion of course, brief episode) or modality (blended, semi-blended) are imposed by the pandemic situation and cannot be really chosen, but instructors do have some degree of freedom in their decision about the remaining ones (e.g. online communication synchrony, student-instructor ratio, role of summative assessments, etc”

Probably not a substantial claim: may not pass S3 (load-bearing).

There is a consensus that a distinction exists between courses with 35 or fewer students and courses with more than 35 students.

“Therefore, there seems to be a consensus in that there is a difference between courses with 35 students of fewer, and courses with over 35 students.”

• cites Hoyt and Lee, 2002Not checked yet

  Not checked in the free run.

• cites Means et al., 2014Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Organizations experience structural inertia, which restricts their capacity to adapt quickly to changes.

“The main idea of the ecological model of organizations is that organizations are subject to structural inertia, which limits their ability to rapidly adapt to changes (Haveman, 1992).”

• cites Haveman, 1992 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Most or at least some findings from prior online learning research may not apply to emergency remote teaching.

“If we observe the changes in teaching and learning caused by COVID-19 under the lens of emergency remote teaching, it could be argued that most, or at least part, of the findings from existing research on online learning might not be applicable to this situation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Online learning increases instructors' workload compared with traditional instruction.

“Even though the results of the different studies under analysis do not indicate differences in academic performance, they do agree in that online learning increases the instructor's workload”

• cites Tomei, 2004; 2006Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In emergency remote teaching, instructors tend to use tools they already know and had in place before the crisis because urgency leaves little time for testing.

“However, in the case of emergency remote teaching, there is little room for testing due to the urgency of moving online in a very short span of time, and therefore instructors tend to turn to what they already know and the tools they have in place before the crisis (Dill, Fischer, McMurtrie, & Supiano, 2020).”

• cites Dill et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Emergency remote teaching is a temporary move to an alternate remote delivery mode in response to a crisis, using fully remote solutions for instruction that would otherwise be face-to-face, blended or hybrid.

“emergency remote teaching emerges as a response to a crisis and entails a temporary shift of instructional delivery to an alternate delivery mode that involves the use of fully remote solutions for instruction that would otherwise be delivered using face-to face, blended or hybrid courses.”

• cites Hodges et al., 2020Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The choice of adaptation strategy falls to individuals, in this case instructors who must quickly choose among multiple digital tools while managing their workload.

“Therefore, the decision on which adaptation strategy to make is left to individuals; in this case, instructors who have to swiftly select among multiple digital tools with different capabilities to support teaching while balancing their workload.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Asynchronous distance education produced substantially better outcomes than traditional teaching, though with higher drop-out rates.

“Bernard et al. (2004) conducted a meta-review of 232 studies and concluded that, when compared to traditional teaching, asynchronous distance education rendered substantial better outcomes, even though at an expense of higher drop-out rates.”

• cites Bernard et al., 2004 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In sudden environmental changes, merely repeating previous practices may result in failure.

“Haveman (1992) further emphasizes that under sudden environmental changes simply reproducing previous ways of doing things may lead to failure.”

• cites Haveman, 1992 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The extent of diversification in an organization's response affects performance: the closer new activities are to previous ones, the higher the chance of success.

“Additionally, the degree of diversification in how the organization reacts –changes across three dimensions: the clients it serves, the goods and services it produces, and the technologies it employs– affects performance in a way that the more closely the new activities are related to the previous ones, the greater the probability of success.”

• cites Haveman, 1992 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Existing meta-analyses generally indicate either that academic performance in online learning is higher than in face-to-face courses or that there are no significant differences between the two.

“Existing meta-analyses tend to support the idea that either the academic performance of students –as final course grades– in online learning is higher than in face-to-face courses, or there are no significant differences between both.”

• cites Shachar and Neumann, 2003Not checked yet

  Not checked in the free run.

• cites Zhao et al., 2005 sourceNot checked yet

  Not checked in the free run.

• cites Means et al., 2013 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Bureaucratic systems implement planned change effectively but are generally inefficient in responding to unplanned or punctuational changes.

“While bureaucratic systems are effective in implementing planned change (Knowles & Saxberg, 1988), they are generally not efficient in responding to unplanned or punctuational changes because of the numerous rules under which they operate and their natural tendency to resist to innovation (Haveman, 1992; Knowles & Saxberg, 1988).”

• cites Knowles & Saxberg, 1988Not checked yet

  Not checked in the free run.

• cites Haveman, 1992 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

More recent studies find differences in higher order thinking skills developed through student social constructivism, favoring asynchronous teaching.

“More recent studies observe differences in higher order thinking skills developed via student social constructivism, in favor of asynchronous teaching (Brierton, Wilson, Kistler, Flowers, & Jones, 2016).”

• cites Brierton et al., 2016Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Organizational change can be beneficial when two conditions hold: the changes respond to dramatic environmental shifts, and they build on established routines and competences.

“First, organizational change may be beneficial if two conditions are met: that the changes occur in response to dramatic environmental shifts (punctuational change), and that they build on established routines and competences (Haveman, 1992).”

• cites Haveman, 1992 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Online learning is the product of deliberate instructional design and planning and needs a whole ecosystem of learner supports that takes time to build.

“Hodges et al. (2020) indicate that the main difference between online learning and emergency remote teaching lies in that online learning results from careful instructional design and planning, requiring an investment in a whole ecosystem of learner supports that takes time to build”

• cites Hodges et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Later studies found no differences between synchronous and asynchronous teaching delivery modes, although students prefer synchronous course sessions.

“However, subsequent studies showed no differences between synchronous and asynchronous teaching delivery modes (Roblyer et al., 2007), pointing out that both types are effective in delivering online teaching, even though students show preference for synchronous course sessions (Skylar, 2009).”

• cites Roblyer et al., 2007 sourceNot checked yet

  Not checked in the free run.

• cites Skylar, 2009Not checked yet

  Not checked in the free run.

May combine two claims.

Recent research supports higher achievement in blended learning than in face-to-face learning, with moderating effects from variables such as computer support, interaction treatments, and STEM versus non-STEM disciplines.

“More recently, the focus has shifted to the comparison of blended and face-to-face learning, with results supporting higher achievement in blended learning (Bernard, Borokhovski, Schmid, Tamim, & Abrami, 2014; Vo, Zhu, & Diep, 2017) and a moderating effect of different variables, such as the kind of computer support used, interaction treatments or whether the courses belong to STEM or non-STEM”

• cites Bernard et al., 2014 sourceNot checked yet

  Not checked in the free run.

• cites Vo et al., 2017Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Adapting a typical course to online teaching, including planning, preparation and development, is estimated to take six to nine months.

“it is estimated that adapting a typical course to online teaching (including planning, preparation and development) takes between six and nine months (Hodges et al., 2020).”

• cites Hodges et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A review of COVID-19 responses by US and South African universities found that two main types of online tools were widely adopted: learning management systems and video conferencing platforms.

“In his review of the response to COVID-19 by US and South African universities, Chaka (2020) finds that there are two main types of online tools and resources that have been widely adopted across all institutions: learning management systems (LMS) and video conferencing platforms.”

• cites Chaka, 2020Not checked yet

  Not checked in the free run.

May combine two claims.

Previous literature supports that students perceive cheating to be easier and more prevalent in online courses, and that unproctored remote exams include more cheating behaviors than proctored ones.

“Previous literature supports the idea that students perceive cheating to be easier and more prevalent in online courses, and that unproctored remote exams include more cheating behaviors than proctored ones (Clark, Callam, Paul, Stoltzfus, & Turner, 2020)”

• cites Clark, Callam, Paul, Stoltzfus, & Turner, 2020 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Zoom is the most used video conferencing tool, followed by collaborative options such as Blackboard Collaborate, Microsoft Teams or WebEx.

“Regarding video conferencing tools, Zoom stands out as the most used tool, followed by other options with collaborative approaches, such as Blackboard Collaborate, Microsoft Teams or WebEx.”

• cites Chaka, 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Procrastination and anxiety are detrimental variables to academic achievement in online learning, with procrastination affecting anxiety.

“Procrastination and anxiety are well-known detrimental variables to academic achievement in online learning, with the former two affecting the latter”

• cites Cormack, Eagle, & Davies, 2020Not checked yet

  Not checked in the free run.

• cites Frazier, Gabriel, Merians, & Lust, 2019Not checked yet

  Not checked in the free run.

• cites Kim & Nembhard, 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The cause of the grade distribution change might be an overall shift in individual grades moving the normal distribution toward higher marks.

“The cause of this result might be attributed to an overall shift in individual grades that would cause the usual normal distribution to move toward higher marks.”

May combine two claims. The quote only partly states this claim.

Students had higher flexibility to take self-paced learning and were in a better position to organize their own study time and pace.

“Therefore, students have found themselves in a better position to organize their own study time and pace, including adjusting for the mix of synchronous and asynchronous delivery of the sessions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Informal channels and an organic structure help successfully coping with changes.

“According to Knowles and Saxberg (1988), these informal channels and an organic structure help successfully coping with changes.”

• cites Knowles and Saxberg (1988)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study did not test students' attitudes and behaviors, so the potential negative impact of procrastination and anxiety cannot be assessed.

“We have not tested students’ attitudes and behaviors in this study, and therefore we cannot assess the potential negative impact of these variables”

Probably not a substantial claim: may not pass S3 (load-bearing).

Being a digital native does not directly equate to being a digital learner.

“being a digital native does not directly equate to being a digital learner”

• cites Gallardo-Echenique, Marqués-Molías, Bullen, & Strijbos, 2015Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Instructors likely turned to videoconferencing tools they were more familiar with, such as Zoom, Skype, Teams or Webex, because Blackboard Collaborate was relatively new.

“it is highly possible that instructors have turned to videoconferencing tools with which they were more familiar, such as Zoom, Skype, Teams or Webex”

Probably not a substantial claim: may not pass S3 (load-bearing).

There is prior evidence of a positive relationship between digital skills and academic performance.

“there is prior evidence of the positive relationship between digital skills and academic performance (Kim, Hong, & Song, 2019; Soleymani, 2014)”

• cites Kim, Hong, & Song, 2019Not checked yet

  Not checked in the free run.

• cites Soleymani, 2014 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The choice of digital tools, delivery methods or class size does not have any relevance in students' outcomes in remote learning.

“the results would also support that the choice of digital tools, delivery methods or class size does not have any relevance whatsoever in students’ outcomes in remote learning”

May combine two claims.

Flexible structures and rich informal information flows, together with decent technical infrastructure and staff's technical literacy and innovativeness, may help succeed in facing a crisis such as the COVID-19 pandemic.

“the results suggest that flexible structures and rich informal information flows, together with a decent technical infrastructure and staff's technical literacy and innovativeness, may help succeed in facing a moment of crisis such as the COVID-19 pandemic”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Among learning management systems, Canvas and Blackboard were most used in the US, while Moodle was predominant in South Africa.

“Among LMS, Canvas and Blackboard were the most used online tools in the US, and Moodle was predominant in South Africa”

• cites Chaka, 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Instructors' digital skills may have been a contributing factor to an effective and rapid deployment of emergency remote teaching.

“it is reasonable to think that it may have been a contributing factor to an effective and rapid deployment of emergency remote teaching”

May combine two claims.

When facing punctual changes such as the pandemic, instructors tend to resort to digital tools they are most familiar with and instructional methods that resemble current practices.

“This result seems to confirm that, when facing punctual changes such as those caused by the pandemic, instructors seem to resort to digital tools that they are most familiar with –i.e., what ‘already works’– and with instructional methods that most easily resemble current practices”

Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

The rapid development of virtual simulation environments would not have been possible without instructors' digital skills.

“something that would have never been possible without said digital skills”

Probably not a substantial claim: may not pass S3 (load-bearing).

The more related activities are to previous ones, the higher the probability of success.

“the more related the activities to the previous ones, the higher the probability of success (Haveman, 1992)”

• cites Haveman, 1992 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Multi-choice tests may have been perceived as easier by students and are better suited to assess knowledge rather than skill, making comparison with previous years problematic.

“These tests may have been perceived as easier by students, and are better suited to assess knowledge rather than skill (Hettiarachchi, Balasooriya, Mor, & Huertas, 2016), which makes it somewhat problematic to compare the results obtained under lockdown with those of previous years.”

• cites Hettiarachchi, Balasooriya, Mor, & Huertas, 2016Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Previous literature suggests that a mix of synchronous and asynchronous approaches (bichronous online learning) works better than either alone because it combines their benefits.

“previous literature (Moallem, 2015; Oztok, Zingaro, Brett, & Hewitt, 2013; Xie, Liu, Bhairma, & Shim, 2018) suggests that a mix of both approaches –or bichronous online learning (Martin, Polly, & Ritzhaupt, 2020)– works better because it combines the benefits of both delivery modes”

• cites Moallem, 2015Not checked yet

  Not checked in the free run.

• cites Oztok, Zingaro, Brett, & Hewitt, 2013Not checked yet

  Not checked in the free run.

• cites Xie, Liu, Bhairma, & Shim, 2018Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The choice of institution and subject by convenience has a non-negligible effect on organizational aspects and therefore on the results.

“The choice of the institution and the subject was made by convenience, and we acknowledge that its effect on organizational aspects –availability of technical infrastructure, organizational structure and processes, students' and instructors' digital skills, equipment and general positive attitudes towards the use of educational technology–, and therefore on the results, is not negligible in the le”

May combine two claims.

The study considered only delivery mode and supporting technologies, not the type of learning activity, and focused only on professional and knowledge skills among student outcomes.

“we only considered delivery mode and supporting technologies rather than the type of learning activity performed–; even regarding student outcomes we just focused on a single aspect: the development of professional and knowledge skills.”

May combine two claims.

The results of this study are specific to one institution (one engineering school), one degree program (Telecommunication Engineering), and the strict lockdown conditions experienced.

“As a case study, the results are specific of one institution –and in particular, of one engineering school– and one subject –a bachelor's degree in Telecommunication Engineering-, and the same applies to the conditions experienced during the course –the strictest lockdown among all European countries.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In general, online learning tends to work better with small or medium class sizes.

“In our literature review, we showed that online learning generally benefits from small- or medium-class sizes”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Genuine digital transformation requires incorporating Industry 4.0 technologies and rethinking how teaching and learning are organized.

“a true digital transformation calls for integration of Industry 4.0 tools (artificial intelligence, robots, internet of things, educational data analytics) and rethinking of the teaching-learning process itself (Bonfield, Salter, Longmuir, Benson, & Adachi, 2020; Koul & Nayar, 2020).”

• cites Bonfield, Salter, Longmuir, Benson, & Adachi, 2020Not checked yet

  Not checked in the free run.

• cites Koul & Nayar, 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Every study carried out during the pandemic represents a temporary reaction by instructors and institutions.

“all studies being conducted during the pandemic reflect a temporary response from instructors and institutions.”

May combine two claims.

The combination of contextual factors might largely bias the results, so the findings lack universal validity.

“The combination of such factors might largely bias the results found in this study; therefore, we do not dare to claim universal validity of our findings, but rather present this case to allow for comparisons with other studies framed in contexts different than the one in this research.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The data indicate a possible student preference for active involvement during synchronous sessions.

“which suggests that students may prefer to engage actively in synchronous sessions.”