CoChat

GPTeach: Interactive TA Training with GPT-based Students

Open access
Markel, Julia M., Opferman, Steven G., Landay, James A., Piech, Chris · 2023 · Proceedings of the Tenth ACM Conference on Learning @ Scale

Published version of a preprint

Cited by 147

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

Interactive and realistic teacher training is hard to scale. This is a key issue for learning at scale, as inadequate preparation can negatively impact both students and teachers. What if we could make the teacher training experience more engaging and, as a downstream effect, reduce the potential for harm that teachers-in-training could inflict on students? We present GPTeach, an interactive chat-based teacher training tool that allows novice teachers to practice with simulated students. We performed two studies to evaluate GPTeach: one think-aloud study and one A/B test between our tool and a baseline. Participants took the role of a teaching assistant conducting office hours with two GPT-simulated students. We found that our tool provides the opportunity for teachers to get valuable teaching practice without the pressures of affecting real students, allowing them to iterate their responses both during and across sessions. Additionally, participants enjoyed flexibility in tailoring their responses according to the varied personas, needs, and learning goals. In this paper, we provide quantitative results and qualitative observations to inform future work in this area. We conclude with a discussion of actionable design ideas for such systems, as well as other ways to use this tool for evaluating teachers and students. GPTeach has recently been deployed into the teacher training component of an online course with over 800 novice teachers.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3573051.3593393

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Research works printed but not linked: 23

  1. Arora, S., Narayan, A., Chen, M. F., Orr, L. J., Guha, N., Bhatia, K., Chami, I., Sala, F., and Ré, C. Ask me anything: A simple strategy for prompting language models. arXiv preprint arXiv:2210.02441 (2022). · arXiv
  2. 10.64152/10125/73488 · DOI · Paper page
  3. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J. Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D. E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Kohd, P. W., Krass, M., Krishna, R., Kuditipudi, R., Kumar, A., Ladhak, F., Lee, M., Lee, T., Leskovec, J., Levent, I., Li, X. L., Li, X., Ma, T., Malik, A., Manning, C. D., Mirchandani, S., Mitchell, E., Munyikwa, Z., Nair, S., Narayan, A., Narayanan, D., Newman, B., Nie, A., Niebles, J. C., Nilforoshan, H., Nyarko, J., Ogut, G., Orr, L., Papadimitriou, I., Park, J. S., Piech, C., Portelance, E., Potts, C., Raghunathan, A., Reich, R., Ren, H., Rong, F., Roohani, Y., Ruiz, C., Ryan, J., Ré, C., Sadigh, D., Sagawa, S., Santhanam, K., Shih, A., Srinivasan, K., Tamkin, A., Taori, R., Thomas, A. W., Tramèr, F., Wang, R. E., Wang, W., Wu, B., Wu, J., Wu, Y., Xie, S. M., Yasunaga, M., You, J., Zaharia, M., Zhang, M., Zhang, T., Zhang, X., Zhang, Y., Zheng, L., Zhou, K., and Liang, P. On the Opportunities and Risks of Foundation Models. Tech. rep., Aug. 2021.
  4. 10.1016/j.compedu.2021.104158 · DOI · Paper page
  5. Cotton, D. R., Cotton, P. A., and Shipway, J. R. Chatting and cheating: Ensuring academic integrity in the era of chatgpt. Preprint. https://doi. org/10.35542/osf. io/mrz8h (2023). · doi:10.35542/osf
  6. Demszky, D., Liu, J., Hill, H. C., Jurafsky, D., and Piech, C. Can automated feedback improve teachers' uptake of student ideas? evidence from a randomized controlled trial in a large-scale online course. edworkingpaper no. 21--483. Annenberg Institute for School Reform at Brown University (2021).
  7. 10.18653/v1/2021.acl-long.130 · DOI · Paper page
  8. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (Minneapolis, Minnesota, June 2019), Association for Computational Linguistics, pp. 4171--4186.
  9. Dzikovska, M. O., Moore, J. D., Steinhauser, N., Campbell, G., Farrow, E., and Callaway, C. B. Beetle ii: a system for tutoring and computational linguistics experimentation. In Proceedings of the ACL 2010 System Demonstrations (2010), pp. 13--18.
  10. Goe, L., Bell, C., and Little, O. Approaches to Evaluating Teacher Effectiveness: A Research Synthesis. National Comprehensive Center for Teacher Quality, June 2008.
  11. Graesser, A. C., Lu, S., Jackson, G. T., Mitchell, H. H., Ventura, M., Olney, A., and Louwerse, M. M. Autotutor: A tutor with dialogue in natural language. Behavior Research Methods, Instruments, & Computers 36, 2 (2004), 180--192.
  12. Jääskeläinen, R. Think-aloud protocol. Handbook of translation studies 1 (2010), 371--374.
  13. 10.1145/3408877.3432533 · DOI
  14. 10.18653/v1/D17-2014 · DOI · Paper page
  15. 10.1080/13803610500392236 · DOI · Paper page
  16. 10.1007/s40593-014-0029-5 · DOI · Paper page
  17. OpenAI. GPT-4 technical report, 2023.
  18. 10.1145/3408877.3432562 · DOI
  19. 10.1145/3386527.3405915 · DOI
  20. 10.1145/3290605.3300587 · DOI
  21. Ruan, S., Nie, A., Steenbergen, W., He, J., Zhang, J., Guo, M., Liu, Y., Nguyen, K. D.,Wang, C. Y., Ying, R., et al. Reinforcement learning tutor better supported lower performers in a math task. arXiv preprint arXiv:2304.04933 (2023). · arXiv
  22. Tack, A., and Piech, C. The AI teacher test: Measuring the pedagogical ability of blender and GPT-3 in educational dialogues. In Proceedings of the 15th International Conference on Educational Data Mining (Durham, United Kingdom, July 2022), A. Mitrovic and N. Bosch, Eds., International Educational Data Mining Society, pp. 522--529.
  23. Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., and Drachsler, H. AreWe There Yet? - A Systematic Literature Review on Chatbots in Education. Frontiers in Artificial Intelligence 4 (July 2021), 654924

Cited by

See all 147

Metadata

Authors
Markel, J. M., Opferman, S. G., Landay, J. A., & Piech, C.
Venue
Proceedings of the Tenth ACM Conference on Learning @ Scale
Publisher
ACM
Published
2023-07-20
Pages
226-236
OpenAlex
W4384659608
Cited by
147
References
12
FWCI
27.60
Type
conference paper
Language
en
Open access
bronze
Funders
Human Centered Artificial Intelligence (203116)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: GPTeach: Interactive TA Training with GPT-based Students. https://app.cochat.ai/papers/10.1145/3573051.3593393
The paper
Markel, J. M., Opferman, S. G., Landay, J. A., & Piech, C. (2023). GPTeach: Interactive TA Training with GPT-based Students. Proceedings of the Tenth ACM Conference on Learning @ Scale. https://doi.org/10.1145/3573051.3593393

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.