The effect of using Kahoot! for learning – A literature review

In the paper

“The results showed many positive effects for the gamified approach, including improved class attendance, less late arrivals to class, higher downloads of course material, improved classroom dynamics, and higher final grade (61% for gamified vs. 53% for traditional).”

What the sources say

• Supported

  Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites Fotaris, Mastoras, Leinfellner, & Rosunally, 2016

In the paper

“In one study where Kahoot! was used for teaching Chinese, using Kahoot!had a significant improvement in the competences of reading and speaking, but not for listening nor vocabulary (X.Liu & Wang, 2017).”

What the sources say

• Supported

  The Procedures section says Kahoot was used daily in the Chinese class. The Results section reports significant pre–post improvements in reading and speaking, but no significant difference in listening or vocabulary. The Discussion clarifies that listening improved numerically, though not significantly. (LLM, conf=0.93)

  cites X.Liu & Wang, 2017

In the paper

“Similarly, a study among 65 Preschool teacher students in Turkey evaluated Kahoot!very positively, including its ability to increase students' interest in the lesson, increase students' motivation, helping students' to understand the lesson better, and that it encourages learners (Bicen & Kocakoyun, 2018).”

What the sources say

• Partly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites Bicen & Kocakoyun, 2018 sourceExperimental — accuracy not yet measured

In the paper

“The study showed that teachers' motivation increased significantly after starting to use Kahoot!(p ¼ .001)and that Kahoot!enhanced their teaching, was entertaining, resulted in better teaching, and increased attention and concentration.”

What the sources say

• Partly supported

  The motivation-scale findings report a significant increase for 15 preservice biology teachers, and Table 3 gives p = .001. The positive-aspects table reports entertainment, attention and concentration. But the section on future use presents more effective teaching as the preservice teachers’ expectation, not a measured improvement in their teaching. (LLM, conf=0.88)

  cites Yapıcı & Karakoyun, 2017 sourceExperimental — accuracy not yet measured

In the paper

“Negative comments were related to persons failing the quiz and lack of technical infrastructure.”

What the sources say

• Partly supported

  Table 4 (Suggestions; page unavailable) lists failing the quiz and technological insufficiency as the two negative views. The accompanying text and Conclusion and Discussion specify that the latter meant students’ lack of technological skills, not a lack of technical infrastructure. (LLM, conf=0.88)

  cites Yapıcı & Karakoyun, 2017 sourceExperimental — accuracy not yet measured

In the paper

“The articles on Kahoot and student anxiety that did not include any statistical analysis include findings such as 56% of students following traditional teaching were afraid to ask questions in class compared to 37% for those who played Kahoot! in class (Budiati, 2017); students rated the statements”

What the sources say

• ?Not checked — no full text of the sourcecites Budiati, 2017
• Partly supported

  Table 3 (section 6) reports a positive rating for the statement that Kahoot builds students’ courage to participate (mean 4.46, SD 0.63). The paper’s Data analysis section (3.5) describes means, standard deviations, frequencies and percentages, rather than inferential tests. I found no result on low stress; the 56% versus 37% comparison is attributed to Budiati (2017) in the citing sentence, not to this paper. (LLM, conf=0.84)

  cites Bicen & Kocakoyun, 2018 sourceExperimental — accuracy not yet measured
• Partly supported

  Section 3.1 reports a rating for the statement that gamification makes learning less stressful. The cited paper does not report reduced fear of asking questions or positive ratings of courage. Moreover, sections 2.2 and 3 explicitly describe descriptive statistical analysis, so it is inaccurate to characterize this paper as having no statistical analysis. The 56% and 37% figures are attributed to Budiati, not this source. (LLM, conf=0.86)

  cites M.Ismail et al., 2018 sourceExperimental — accuracy not yet measured

In the paper

“The anxiety score for Socrative was 2.238, and for Kahoot! it was 1.871, where a lower value represents less anxiety.”

What the sources say

• Supported

  The study-group passage identifies Kahoot! as the game-based system and Socrative as the non-game-based system. Table 6 reports mean test-anxiety scores of 1.871 and 2.238, respectively, and the test-anxiety results section explicitly describes the lower Kahoot! group score as lower anxiety. (LLM, conf=0.95)

  cites Turan & Meral, 2018 source

In the paper

“Further studies report that students found Kahoot! to be helpful to prepare for the exam (Iwamoto et al., 2017), that Kahoot!increased their interest in learning more about what they had learned and that they told others about it (Budiati, 2017), that the students motivation and confidence increased, and they felt more secure and safe in class (Tsymbal, 2018), and that students felt comfortable”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Iwamoto et al., 2017
• ?Not checked — no full text of the sourcecites Budiati, 2017
• Partly supported

  Split into 3 atoms — 0 confirmed, 3 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites Tsymbal, 2018 sourceExperimental — accuracy not yet measured

In the paper

“In the search for literature, four literature reviews where Kahoot! was mentioned were found.”

In the paper

“The results showed that the use of Kahoot! had a significant improvement from pre-to post-test for all topics (p ¼ .001,effect size large), but compared to the traditional approach, the Kahoot!group, performed only significantly better on theoretical topics (p ¼ .0001,effect size small) and not on the more practical topics.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Ranieri, Raffaghelli, & Bruni, 2018

In the paper

“Thirty-four of the articles in our literature review explicitly describe how Kahoot!affects the classroom dynamics, but only four of these contain any analysis for statistical significance.”

In the paper

“Both groups performed the same on the pre-test, but the technology-supported learning group performed significantly better on the post-test (scored 90%) compared to those who used paper (scored 75%).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Filologiczna, 2016

In the paper

“A quasi-experiment in the USA with 49 undergrad students investigated the impact on the final grade of learning of lectures with group discussions vs. lectures with Kahoot!where it was found that Kahoot! made a significant impact on the exam score (p ¼ .008)(Iwamoto et al., 2017).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Iwamoto et al., 2017

In the paper

“The results showed that the control group got an average score of 46.05, and the experimental group an average score of 63.15 (p ¼ .035).”

What the sources say

• ?Not checked — no full text of the sourcecites Asmalı, 2018

In the paper

“Further, 20% more students passed the exam with less frequent use of Kahoot! than traditional teaching and 29% with frequent use.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Ares, Bernal, Nozal, S anchez, & Bernal, 2018

In the paper

“However, there are also a few studies where Kahoot! did not result in improved learning.”

• Also stated in: § 5. Conclusions · plain

  “However, there are also few studies where Kahoot! did not have a significant positive effect on learning performance.”

In the paper

“Of the thirty-six studies investigating the learning outcome of only using Kahoot!for teaching, six are in the K-12 context (Primary, Secondary, and High School), while the rest are with university students.”

In the paper

“However, there are some exceptions.”

In the paper

“A case study from Portugal with 324 university students showed that with Kahoot! the average grade was improved 6.4% compared to traditional teaching, that weaker students improved their grades with 12%, and fewer failed the course (Esteves, Pereira, Veiga, Vasco, & Veiga, 2017).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Esteves, Pereira, Veiga, Vasco, & Veiga, 2017

In the paper

“The results showed a significant improvement in the final grade from traditional teaching to the use of Kahoot! (both groups), and that the group using Kahoot!frequently improved more (þ38.4%)than the other (þ32.0%).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Ares, Bernal, Nozal, S anchez, & Bernal, 2018

In the paper

“The results showed that including these Kahoot!sessions had a significant impact on the final examination scores (p ¼ .005).”

What the sources say

• ?Not checked — no full text of the sourcecites Kinder & Kurz, 2018 source

In the paper

“Studies also found that Kahoot!positively affected the excitement, engagement, learning experience and efficiency of the students”

In the paper

“There are only four articles that include statistical significance tests and two that report effect sizes, so more experiments are necessary for a solid conclusion.”

In the paper

“Seventy percent of the articles focusing on student anxiety describe studies that report a reduction of student anxiety from playing Kahoot! in class, but there were only two studies with tests for statistical significance.”

In the paper

“The challenges or negative issues related to Kahoot! mentioned by teachers in these studies included the challenge of getting the difficulty of questions and answer time right, problems related to network connectivity, scoring based on how quickly the students answer can reduce student reflection and cause some students to guess without thinking, some students can have”

In the paper

“Most of the articles found in our literature review investigate how students perceive the use of Kahoot! in the classroom, and eightytwo articles reported findings from the students' point of view.”

In the paper

“The main challenges reported by students included technical problems such as unreliable internet connections, hard to read questions and answers on projected screen, not being able to change answer after submission, stressful time-pressure for giving answers, not enough time to answer, afraid of losing, and hard to catch up if an incorrect answer had been given.”

In the paper

“Our literature review found fourteen studies where student anxiety is mentioned, where two tested for statistical significance.”

In the paper

“The challenges mentioned by most articles were unreliable and slow internet connections.”

In the paper

“However, several articles were left out because they were behind a paywall where most of them were published in the proceedings for the International Conference on Technology, Education and Development (INTED).”

In the paper

“The main challenges reported include technical challenges, hard to read questions and answers on a projected screen, time pressure, afraid of losing, and hard to catch up if an incorrect answer had been given.”

In the paper

“Fourteen articles were found that contain results related to student anxiety, and ten articles reported that Kahoot!reduced student anxiety related to asking questions, reduce stress and tension, encourages participation without being judged, add humor to class, and enables shy students to get involved.”

In the paper

“Two studies showed a statistically significant reduction of student anxiety, and one study reported that Kahoot! could produce agitation.”

In the paper

“Reviews | 22 (11%) | Reviews or comparisons from a single person”

In the paper

“Our literature review shows that there have been conducted several experiments on the learning effect of using Kahoot!but there is still room for empirical studies, especially on classroom dynamics, student anxiety, and perceptions of students and teachers.”

In the paper

“This article has presented a literature review on the effect of using Kahoot!for learning, where 93 studies were included.”

In the paper

“Of the accepted articles, 97% present mainly positive results related to Kahoot! and 8% include challenges and problems.”

In the paper

“Rejected studies | 44 (21%) | Lack of description, analysis, no explicit presentation of results, too limited scope, invalid results due to research design or”

• Also stated in: § Appendix D. Uses of a Game-based Student Response System · plain

  “execution, conclusions not based on data or analysis”

In the paper

“Positive findings include higher motivation of teachers, ease of use, a motivating tool, support teachers' instructive work in class, can assess students' knowledge in real-time, increases student motivation, stimulate students to speak their point of view in class, can wake up students, increase class participation, and reduce teacher's workload.”

In the paper

“Several studies reported the importance for students to be anonymous, which can result in reduced stress and a safer environment to fail”

In the paper

“The main challenges were found to be technical issues and challenges, get the questions and answers right and scoring based on timing.”

In the paper

“Presentations | 14 (7%) | Only presentation and not a study”

In the paper

“The literature review included four studies that reported statistically significant results related to classroom dynamics, where one showed that the effect on classroom dynamics was reduced from first-time use to after frequent use over five months.”

In the paper

“Total | 1232 | 206”

In the paper

“One challenge mentioned by four articles was stressful time-pressure for giving answers and the tendency that some students simply guessed the answer in order to try to get a high score”

In the paper

“The main difference between all the systems mentioned above is that Kahoot!focuses more on engagement through a competitive gaming experience.”

In the paper

“Boredom in a computer learning environment can cause inferior learning and problem behavior (Baker, D'Mello, Rodrigo, & Graesser, 2010).”

What the sources say

• ?Not checked — no full text of the sourcecites Baker, D'Mello, Rodrigo, & Graesser, 2010 source

In the paper

“Experiments with the early prototypes showed that Lecture Quiz increased student motivation, engagement, and perceived learning through entertaining social learning activities (Wang, Øfsdal, & Mørch-Storstein, 2008;Wu, Wang, Børresen, & Tidemann, 2011).”

What the sources say

• ?Not checked — no full text of the sourcecites Wang, Øfsdal, & Mørch-Storstein, 2008 source
• ?Not checked — no full text of the sourcecites Wu, Wang, Børresen, & Tidemann, 2011 source

In the paper

“Further, there is extensive evidence that student engagement in lectures improves understanding and academic results (Prince, 2004).”

What the sources say

• ?Not checked — no full text of the sourcecites Prince, 2004 source

In the paper

“Games can be beneficial for academic achievement, motivation, and classroom dynamics (Sharples, 2000).”

What the sources say

• ?Not checked — no full text of the sourcecites Sharples, 2000 source

In the paper

“Some teachers are afraid that the competitive gaming elements in Kahoot! will create anxiety for the students that will ruin their motivation to learn (Zarzycka-Piskorz, 2016).”

What the sources say

• ?Not checked — no full text of the sourcecites Zarzycka-Piskorz, 2016

In the paper

“Lack of motivation can result in a reduction of learning outcomes and a negative atmosphere in the classroom (O.L. Liu, Bridgeman, & Adler, 2012).”

What the sources say

• ?Not checked — no full text of the sourcecites O.L. Liu, Bridgeman, & Adler, 2012 source

In the paper

“The identified feedback practices not fully supported in Kahoot! were 1) Ability to assist in clarifying what good performance is; 2) Ability to deliver high-quality information to students about their learning; and 3) Ability to encourage teacher and peer dialogue around learning.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Omar, 2017

In the paper

“Finally, there is a pilot study from South Korea with 51 university students that found that Kahoot!improves vocabulary retention while fostering a positive learning environment and a meaningful learning experience (Taylor & Reynolds, 2018).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Taylor & Reynolds, 2018

In the paper

“The results showed that the use of audio and music in Kahoot! had a significant positive impact on classroom dynamics, which resulted in an improved atmosphere for oral questions and discussions in class.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Wang & Lieberoth, 2016

In the paper

“The main finding was that the learner as leader strategy contributed to enhancing discussions in groups, especially for the leading group, and benefited those who acted as leaders.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Y.-H.Wang, 2017a
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Y.-H.Wang, 2017b

In the paper

“One study comparing using a paper quiz, a Clicker student response system and Kahoot! for reviewing knowledge, revealed significant improvement in motivation (small effect size), engagement (medium effect size), enjoyment (small effect size), and concentration (medium effect size) for the gamified approach, but not significant learning improvement (Wang et al., 2016).”

What the sources say

• ?Not checked — no full text of the sourcecites Wang et al., 2016

In the paper

“in a comparison of Clicker and Kahoot!-the engagement was significantly better with Kahoot! and answering carefully was significantly better with Clicker (Jones, Harden, Rassias, & Abourashchi, 2018)”

What the sources say

• ?Not checked — no full text of the sourcecites Jones et al., 2018

In the paper

“One result from these articles shows that playing Kahoot!frequently over five months did not affect the students' engagement, motivation, concentration, and perceived learning negatively compared to the first time it was played, but the classroom dynamics were affected (Wang, 2015).”

What the sources say

• ?Not checked — no full text of the sourcecites Wang, 2015 source

In the paper

“In a survey, using Likert's scale among 149 secondary education teachers in Greece who were introduced to Kahoot! the most positive findings were that Kahoot! was easy to use (4.81 of 5), it was a motivating tool for learning (4.63 of 5), support teachers' instructive work in class (4.01 of 5), assess students' knowledge (4.64 of 5), and is an exciting tool for teaching (4.45 of 5).”

In the paper

“The studies above show that for the majority of studies, Kahoot! had a significant positive effect on motivation, concentration, perceived learningbut not for all studies.”

In the paper

“Another study found that the use of audio and points in Kahoot!significantly affects (large effect size) the students' concentration, engagement, enjoyment, motivation, perceived learning, and classroom dynamics and that using Kahoot!without points and audio gave the worst result (Wang & Lieberoth, 2016).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Wang & Lieberoth, 2016

In the paper

“and Kahoot! had a more positive impact on students' engagement than Quizizzbut not significant (G€ oksün & Gürsoy, 2019).”

What the sources say

• ?Not checked — no full text of the sourcecites Göksün & Gürsoy, 2019 source

In the paper

“Both involved in the critical appraisal process agreed on what articles to reject, although the reasoning for rejecting the two articles was different.”

In the paper

“However, the thirty-three other papers all reported on improved classroom dynamics from using Kahoot! in the classroom, including the improved teacher-student interaction and student-student interaction, more favorable to actively participate in class, improved classroom atmosphere, and easier to answer questions in class.”

In the paper

“To our knowledge, all of the rejected articles failed to meet these requirements, and lacked a description of the research context, lacked a description of results, analysis of data, having conclusions not based on data, or had significant flaws in set-up or execution of the study.”

In the paper

“The literature review presented in this article was carried out according to the review method presented in (Dybå & Dingsøyr, 2008) to ensure maximum literature coverage, include all relevant studies and collect and synthesize the data from these studies in a meaningful way.”

What the sources say

• ?Not checked — no full text of the sourcecites Dybå & Dingsøyr, 2008 source

In the paper

“The main conclusion is that students have a very positive perception of the use of Kahoot! in learning, and these results include a positive effect on motivation, engagement, concentration, perceived learning, attention, enjoyment, satisfaction, and confidence.”

In the paper

“It was found that Kahoot! had a positive effect on learning both for K-12 and higher education, as well as for language learning, technical and engineering fields, science, math, business, and nursing.”

In the paper

“In our case, the search string was simple, and should not be a source for leaving out articles.”

In the paper

“Finally, the generalization of the results of this review is limited to Kahoot! and not to game-based learning or any GSRS in general, and for the context of K-12 and higher education.”

In the paper

“This process requires students to think critically (Bryant et al., 2018).”

What the sources say

• ?Not checked — no full text of the sourcecites Bryant et al., 2018 source

In the paper

“The filter used was to check if the articles described a study, and then articles with studies were checked for relevance (scope), rigor, and credibility.”

In the paper

“The main conclusion of our literature review is that Kahoot! has a positive effect on learning, but there are challenges and room for improvement.”

In the paper

“This process was carried out independently by the two authors of this article.”

In the paper

“There might be some studies we did not find in our literature search, but we believe the by searching Google Scholar, Science Direct, Wiley InterScience, Web of Science, and Scopus, the coverage should be satisfactory.”