New Approach for Conversational Agent Definition by Non-Programmers: A Visual Domain-Specific Language
Open accessAbstract
Intelligent tutors and conversational agents (CAs) have proven to be useful learning tools. They have potential not only as stand-alone devices but also as integrable components to enrich and complement other educational resources. For this, new authoring approaches and platforms are required. They should be accessible to non-programmers (such as most teachers) and they should be integrable into current web-based educational platforms. This paper proposes a new approach to define such agents through a visual domain-specific language based on Google Blockly (a scratch-like language). It also develops a web-based integrable authoring platform to serve as a prototype, describing the requirements and architecture. To evaluate whether this novel approach is effective, a multi-stage experiment was conducted. First, participants learned to use the prototype authoring platform through an interactive tutorial. Second, they created a CA with a specific domain model. Times and performance were measured. Finally, they answered a standardized usability questionnaire (UMUX) and a purpose-specific survey. Results show that participants were able to learn to use the domain-specific language in a short time. Moreover, the purpose-specific survey indicates that their perception of the approach (and its potential) is positive. The standardized questionnaire indicates that even in its prototype stage, its usability is satisfactory.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Chatbot development can be made accessible to non-programmers, for example through visual programming approaches such as Blockly.
“without requiring chatbot developers to have in-depth software engineering knowledge—as exemplified in using visual programming approaches such as Blockly to chatbot development [89]”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [89]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989 · see citing works
An Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · see citing works
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · see citing works
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,765 · see citing works
The Scratch Programming Language and Environment
Maloney, J. H., Resnick, M., Rusk, N., Silverman, B., & Eastmond, E. (2010) · ACM Transactions on Computing Education · doi:10.1145/1868358.1868363 · Cited by 1,423 · see citing works
Hands-on, simulated, and remote laboratories
Ma, J., & Nickerson, J. V. (2006) · ACM Computing Surveys · doi:10.1145/1132960.1132961 · Cited by 986 · see citing works
The Behavior of Tutoring Systems
VanLehn, K. A. (2006) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2006-16(3)02 · Cited by 821 · see citing works
The Usability Metric for User Experience
Finstad, K. (2010) · Interacting with Computers · doi:10.1016/j.intcom.2010.04.004 · Cited by 604 · see citing works
Embodiment in conversational interfaces
Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L. W., Chang, K., Vilhjálmsson, H. H., & Yan, H. (1999) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/302979.303150 · Cited by 569 · see citing works
Sáez-López, J.-M., Román-González, M., & Vázquez-Cano, E. (2016) · Comput. Educ. · doi:10.1016/j.compedu.2016.03.003 · Cited by 522 · see citing works
Five Major Shifts in 100 Years of Engineering Education
Froyd, J. E., Wankat, P. C., & Smith, K. A. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2190167 · Cited by 510 · see citing works
A Tablet Computer for Young Children? Exploring its Viability for Early Childhood Education
Couse, L. J., & Chen, D. W. (2010) · Journal of Research on Technology in Education · doi:10.1080/15391523.2010.10782562 · Cited by 477 · see citing works
A new way of teaching programming skills to K-12 students: Code.org
Kalelıoğlu, F. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.05.047 · Cited by 437 · see citing works
A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application
Kopp, S., Gesellensetter, L., Krämer, N. C., & Wachsmuth, I. (2005) · Lecture notes in computer science · doi:10.1007/11550617_28 · Cited by 400 · see citing works
PhET: Simulations That Enhance Learning
Wieman, C., Adams, W. K., & Perkins, K. K. (2008) · Science · doi:10.1126/science.1161948 · Cited by 397 · see citing works
Embodied conversational interface agents
Cassell, J. (2000) · Communications of the ACM · doi:10.1145/332051.332075 · Cited by 388 · see citing works
CONTROL LEARNING: PRESENT AND FUTURE
Bencomo, S. D. (2002) · IFAC Proceedings Volumes · doi:10.3182/20020721-6-es-1901.01641 · Cited by 382 · see citing works
Innovations in STEM education: the Go-Lab federation of online labs
de Jong, T., Sotiriou, S. A., & Gillet, D. (2014) · Smart Learning Environments · doi:10.1186/s40561-014-0003-6 · Cited by 311 · see citing works
AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring
Nye, B. D., Graesser, A. C., & Hu, X. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0029-5 · Cited by 286 · see citing works
A New Paradigm for Intelligent Tutoring Systems: Example-Tracing Tutors
Aleven, V., McLaren, B. M., Sewall, J., & Koedinger, K. R. (2009) · doi:10.5555/1734243.1734245 · Cited by 279 · see citing works
Control learning: present and future
Dormido, S. (2004) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2003.12.002 · Cited by 279 · see citing works
The Use of Mobile Learning in Science: A Systematic Review
Crompton, H., Burke, D., Gregory, K., & Gräbe, C. (2016) · Journal of Science Education and Technology · doi:10.1007/s10956-015-9597-x · Cited by 228 · see citing works
Johnson, G. W., & Jennings, R. M. (1994) · Cited by 223 · see citing works
The Cognitive Tutor Authoring Tools (CTAT): Preliminary Evaluation of Efficiency Gains
Aleven, V., McLaren, B. M., Sewall, J., & Koedinger, K. R. (2006) · Lecture notes in computer science · doi:10.1007/11774303_7 · Cited by 218 · see citing works
Mobile educational features in authoring tools for personalised tutoring
Virvou, M. K., & ALEPIS, E. (2004) · Computers & Education · doi:10.1016/j.compedu.2003.12.020 · Cited by 205 · see citing works
Şad, S. N., & Göktaş, Ö. (2013) · British Journal of Educational Technology · doi:10.1111/bjet.12064 · Cited by 183 · see citing works
Tawfik, M., Sancristobal, E., Martín, S., et al. (2013) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2012.20 · Cited by 183 · see citing works
Architecture for Building Conversational Agents That Support Collaborative Learning
Kumar, R., & Rosé, C. P. (2011) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2010.41 · Cited by 160 · see citing works
Opening the Door to Non-programmers: Authoring Intelligent Tutor Behavior by Demonstration
Koedinger, K. R., Aleven, V., Heffernan, N. T., McLaren, B. M., & Hockenberry, M. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30139-4_16 · Cited by 154 · see citing works
Bickmore, T. W., Utami, D., Matsuyama, R., & Paasche-Orlow, M. K. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5239 · Cited by 153 · see citing works
Evolving Remote Laboratory Architectures to Leverage Emerging Internet Technologies
Lowe, D. B., Murray, S., Lindsay, E. D., et al. (2009) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2009.33 · Cited by 152 · see citing works
The Generalized Intelligent Framework for tutoring (GIFT)
Sottilare, R. A., Brawner, K., Goldberg, B., & Holden, H. K. (2017) · CRC Press eBooks · doi:10.1201/9781315583655-20 · Cited by 146 · see citing works
Example-Tracing Tutors: Intelligent Tutor Development for Non-programmers
Aleven, V., McLaren, B. M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., Ringenberg, M. A., & Koedinger, K. R. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0088-2 · Cited by 139 · see citing works
Artificially intelligent conversational agents in libraries
Rubin, V. L., Chen, Y., & Thorimbert, L. M. (2010) · Library Hi Tech · doi:10.1108/07378831011096196 · Cited by 113 · see citing works
Conversational Agents in E-Learning
Kerry, A., Ellis, R., & Susan Bull, A. (2009) · Springer eBooks · doi:10.1007/978-1-84882-215-3_13 · Cited by 99 · see citing works
Teaching the Teacher: Tutoring SimStudent Leads to More Effective Cognitive Tutor Authoring
Matsuda, N., Cohen, W. W., & Koedinger, K. R. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0020-1 · Cited by 84 · see citing works
A Flexible and Configurable Architecture for Automatic Control Remote Laboratories
Kalúz, M., García-Zubía, J., Fikar, M., et al. (2015) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2015.2389251 · Cited by 82 · see citing works
Re-assessing the usability metric for user experience (UMUX) scale
Berkman, M. İ., & Karahoca, D. (2016) · Journal of Usability Studies archive · doi:10.5555/2993219.2993221 · Cited by 71 · see citing works
Towards an Online Lab Portal for Inquiry-Based STEM Learning at School
Govaerts, S., Cao, Y., Vozniuk, A., et al. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41175-5_25 · Cited by 69 · see citing works
Towards New Multiplatform Hybrid Online Laboratory Models
Rodríguez-Gil, L., García-Zubía, J., Orduña, P., et al. (2016) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2016.2591953 · Cited by 64 · see citing works
EJS, JIL Server, and LabVIEW: An Architecture for Rapid Development of Remote Labs
Chacón, J., Vargas, H., Farías, G., et al. (2015) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2015.2389245 · Cited by 63 · see citing works
Personalised learning spaces and federated online labs for STEM Education at School
Gillet, D., de Jong, T., Sotirou, S., et al. (2013) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon.2013.6530194 · Cited by 60 · see citing works
Blockly Language Creation and Applications
Trower, J., & Gray, J. (2015) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/2676723.2691871 · Cited by 49 · see citing works
A Programming by Demonstration Authoring Tool for Model-Tracing Tutors
Blessing, S. B. (2003) · Springer eBooks · doi:10.1007/978-94-017-0819-7_4 · Cited by 44 · see citing works
Digital playgrounds for early computing education
Kumar, D. (2014) · ACM Inroads · doi:10.1145/2568195.2568200 · Cited by 41 · see citing works
A visual programming framework for wireless sensor networks in smart home applications
Serna, M. Á., Sreenan, C. J., & Fedor, S. (2015) · IEEE International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP) · doi:10.1109/issnip.2015.7106946 · Cited by 38 · see citing works
A Self-Adaptive Multi-Agent System Approach for Collaborative Mobile Learning
de la Iglesia, D. G., Calderón, J. F., Weyns, D., Milrad, M., & Nussbaum, M. (2014) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2014.2367493 · Cited by 35 · see citing works
Kuan, W.-H., Tseng, C.-H., Chen, S., et al. (2016) · Journal of Science Education and Technology · doi:10.1007/s10956-016-9603-y · Cited by 30 · see citing works
Authoring Tools for Collaborative Intelligent Tutoring System Environments
Olsen, J. K., Belenky, D. M., Aleven, V., Rummel, N., Sewall, J., & Ringenberg, M. A. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07221-0_66 · Cited by 26 · see citing works
WebTalk: mining Websites for automatically building dialog systems
Feng, J., Bangalore, S., & Rahim, M. A. (2004) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2003.1318423 · Cited by 23 · see citing works
Rodríguez‐Triana, M. J., Govaerts, S., Halimi, W., et al. (2014) · doi:10.1109/icwoal.2014.7009219 · Cited by 20 · see citing works
A mobile robot platform for open learning based on serious games and remote laboratories
Iturrate, I., Martin, G., García-Zubía, J., Angulo, I., Dziabenko, O., Orduña, P., Alves, G. R., & Fidalgo, A. V. D. S. (2013) · International Conference of the Portuguese Society for Engineering Education (CISPEE) · doi:10.1109/cispee.2013.6701970 · Cited by 19 · see citing works
Intelligent Tutoring Systems for Collaborative Learning: Enhancements to Authoring Tools
Olsen, J. K., Belenky, D. M., Aleven, V., & Rummel, N. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_141 · Cited by 16 · see citing works
Making It Personal: End-User Authoring of Health Narratives Delivered by Virtual Agents
Bickmore, T., & Ring, L. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15892-6_43 · Cited by 15 · see citing works
Situated Pedagogical Authoring: Authoring Intelligent Tutors from a Student’s Perspective
Lane, H. C., Core, M. G., Hays, M. J., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_20 · Cited by 10 · see citing works
Bringing Non-programmer Authoring of Intelligent Tutors to MOOCs
Aleven, V., Baker, R. S., Wang, Y., Sewall, J., & Popescu, O. (2016) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2876034.2893442 · Cited by 6 · see citing works
A Chatbot as a Novel Corpus Visualization Tool
Shawar, B. A., & Atwell, E. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/59r7uzupnexz · Cited by 6 · see citing works
Ray, C., & Gilbert, S. B. (2013) · Iowa State University Digital Repository (Iowa State University) · Cited by 5 · see citing works
A visual editor for language-independent scripting for BPMN modeling
Wiriyakul, J., & Senivongse, T. (2015) · International Joint Conference on Computer Science and Software Engineering · doi:10.1109/jcsse.2015.7219788 · Cited by 3 · see citing works
OpenSocial Application Builder and Customizer for School Teachers
Rodríguez-Gil, L., Latorre, M., Orduña, P., et al. (2014) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2014.20 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- V. Aleven, J. Sewall, O. Popescu, M. van Velsen, S. Demi, and B. Leber, ''Reflecting on twelve years of ITS authoring tools research with CTAT,'' in Design Recommendations for Intelligent Tutoring Systems. Orlando, FL, USA: U.S. Army Research Laboratory, 2015, pp. 263-283.
- J. Artif. Intell. Educ., vol. 25, no. 1, pp. 1-34, 2015.
- DIEGO LÓPEZ-DE-IPIÑA received the Ph.D. degree from the University of Cambridge in 2002. Responsible for several modules in the B.Sc. and M.Sc. degrees in computer engineering, he is interested in pervasive computing, IoT, semantic service middleware, open linked data, and social data mining. He is an Associate Professor and a P.R. of the MORElab Group and the Direc- tor of the DeustoTech Internet Unit, and of the Ph.D. Program within the Faculty of Engineer- ing, University of Deusto. He is taking and has taken part in several big consortium-based research European (IES CITIES, MUGGES, SONOPA, CBDP, GO-LAB, and LifeWear) and Spanish projects, and has more than 70 publications in relevant international conference and journals, including more than 25 JCR-indexed articles.
Cited by
Sorted by citations
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247 · see citing works
- Authors
- Rodriguez-Gil, L., Garcia-Zubia, J., Orduna, P., Villar-Martinez, A., & Lopez-De-Ipina, D.
- ORCID
- Rodriguez-Gil, Luis
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 5262-5276
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2906696915
- Cited by
- 20
- References
- 60
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 4 · 2022: 4 · 2021: 6 · 2020: 2 · 2019: 1
- FWCI
- 1.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- chatbots, intelligent tutoring systems, visual programming, domain-specific languages, Blockly, non-programmers, web-based educational platform, usability evaluation, agent definition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
