Evaluating the Efficiency of Voice Control as Human Machine Interface in Production
Abstract
Cars, mobile phones, and smart home devices already provide automatic speech recognition (ASR) by default. However, human machine interfaces (HMI) in industrial settings, as opposed to consumer settings, operate under different conditions and thus, present different design challenges. Voice control, arguably the most natural form of communication, has the potential to shorten complex command sequences and menu structures in order to directly execute a final command. Therefore, this contribution explored how differing HMI scenarios could possibly be optimized, by either replacing or complementing existing touch control interactions with voice control. Typical commands from CNC milling machines and industrial robots were categorized by their complexity, quantified by menu level and the necessary number of interactions. The collected interaction data showed that voice control can already provide a time efficiency advantage at either one additional menu level or three touchscreen interactions. For complex commands, such as those needing five menu levels and seven interactions on the touchscreen, the time efficiency advantage of voice control can reach up to 67 %. Furthermore, the study shows the possibility of reducing machine operator training times when using voice control by significantly lower interaction times for the first repetition of the participantsNote to Practitioners—Several publications investigate the ergonomics, usability, and cognitive load of classic mouse and keyboard control, button control, touch control, gesture control, gaze control, or voice control in specific interaction scenarios. All publications state that these factors need to be considered for the development of modern human machine interfaces (HMI). Due by the complexity of these factors, it is difficult to develop general guidelines to build efficient HMIs independent from the machine or process. A lack of efficiency guidelines potentially hampers the development of new HMIs, currently necessary to address the new challenges in the digital production hall like increasingly complex machines, processes that become more individual as well as multiple machine operation. In order to inform HMI development, voice and touch control alternatives were empirically measured. Based on the collected data complexity time equivalents for each menu level and number of interactions were calculated. These time equivalents provide the opportunity for machine and programmable logic controller (PLC) manufacturers to evaluate their production processes and the related interaction processes regarding the potential efficiency benefits of voice control as a complement or substitute for the conventional HMI system. Using this model, the efficiency advantage of voice control can be estimated without implementing and testing a voice control on a real production machine. Thus, the potential benefit of implementing voice control can be assessed directly, avoiding expensive test runs.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
TLMs enable workers to operate machinery using natural language commands, reducing the need for manual input.
“TLMs enable workers to operate machinery using natural language commands, reducing the need for manual input [148].”
From Tiny Language Models for Automation and Control: Overview, Potential Applications, and Future Research Directions (2025) · cites this paper as [148]
Cites
Fitting Linear Mixed-Effects Models Using <b>lme4</b>
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835 · see citing works
performance: An R Package for Assessment, Comparison and Testing of Statistical Models
Lüdecke, D., Ben-Shachar, M., Patil, I., Waggoner, P., & Makowski, D. (2021) · Journal of Open Source Software · doi:10.21105/joss.03139 · Cited by 5,836 · see citing works
A time delay neural network architecture for efficient modeling of long temporal contexts
Peddinti, V., Povey, D., & Khudanpur, S. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-647 · Cited by 986 · see citing works
Purely Sequence-Trained Neural Networks for ASR Based on Lattice-Free MMI
Povey, D., Peddinti, V., Galvez, D., Ghahremani, P., Manohar, V., Na, X., Wang, Y., & Khudanpur, S. P. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-595 · Cited by 746 · see citing works
A first course in design and analysis of experiments
Oehlert, G. W. (2000) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 387 · see citing works
Research Methods in Human-Computer Interaction
Landauer, T. (1988) · Elsevier eBooks · doi:10.1016/b978-0-444-70536-5.50047-6 · Cited by 384 · see citing works
An empirical methodology for writing user-friendly natural language computer applications
Kelley, J. F. (1983) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/800045.801609 · Cited by 264 · see citing works
Natural user interfaces for human-drone multi-modal interaction
Fernández, R. A. S., Sánchez-López, J. L., Sampedro, C., Bavle, H., Molina, M., & Campoy, P. (2016) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas.2016.7502665 · Cited by 182 · see citing works
Lasi, H., Fettke, P., Kemper, H.-G., Feld, T., & Hoffmann, M. (2014) · WIRTSCHAFTSINFORMATIK · doi:10.1007/s11576-014-0424-4 · Cited by 174 · see citing works
Bingöl, M. C., & Aydoğmuş, Ö. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103903 · Cited by 89 · see citing works
Interaction With Gaze, Gesture, and Speech in a Flexibly Configurable Augmented Reality System
Wang, Z., Wang, H., Yu, H., & Lu, F. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3097973 · Cited by 74 · see citing works
The Fourth Singularity and the Future of Jobs
Higgins, J. M. (2013) · World Futures Review · doi:10.1177/1946756712473437 · Cited by 65 · see citing works
Longo, F., & Padovano, A. (2020) · Manufacturing Letters · doi:10.1016/j.mfglet.2020.09.001 · Cited by 51 · see citing works
Comparing human and automatic speech recognition in simple and complex acoustic scenes
Spille, C., Kollmeier, B., & Meyer, B. T. (2018) · Computer Speech & Language · doi:10.1016/j.csl.2018.04.003 · Cited by 47 · see citing works
Guerin, C., Rauffet, P., Chauvin, C., & Martin, E. (2019) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2019.12.111 · Cited by 43 · see citing works
Programming cobots by voice: A human-centered, web-based approach
Ionescu, T. B., & Schlund, S. (2021) · Procedia CIRP · doi:10.1016/j.procir.2020.05.213 · Cited by 32 · see citing works
Voice-Based Intelligent Virtual Agents (VIVA) to Support Construction Worker Productivity
Linares-Garcia, D. A., Roofigari‐Esfahan, N., Pratt, K., & Jeon, M. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104554 · Cited by 31 · see citing works
3-D CNN Models for Far-Field Multi-Channel Speech Recognition
Ganapathy, S., & Peddinti, V. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461580 · Cited by 31 · see citing works
Lodgaard, E., & Dransfeld, S. (2020) · Procedia CIRP · doi:10.1016/j.procir.2020.05.039 · Cited by 30 · see citing works
Mertes, J., Lindenschmitt, D., Amirrezai, M., Tashakor, N., Glatt, M. F., Schellenberger, C., Shah, S. M., Karnoub, A., Hobelsberger, C., Yi, L., et al. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.08.009 · Cited by 30 · see citing works
A General Methodology for Adapting Industrial HMIs to Human Operators
Villani, V., Sabattini, L., Loch, F., Vogel‐Heuser, B., & Fantuzzi, C. (2019) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2019.2941541 · Cited by 23 · see citing works
A Concept for Integration of Voice Assistant and Modular Cyber-Physical Production System
Afanasev, M. Y., Fedosov, Y. V., Andreev, Y., Krylova, A. A., Shorokhov, S. A., Zimenko, K. V., & Kolesnikov, M. V. (2019) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin41052.2019.8972015 · Cited by 15 · see citing works
Measurement of efficiency of auditory vs visual communication in HMI: A cognitive load approach
Kumar, N., V, & Kumar, J. (2016) · doi:10.1109/hmi.2016.7449168 · Cited by 11 · see citing works
Analysis of voice control of a collaborative robot
Janíček, M., Ružarovský, R., Velíšek, K., & Holubek, R. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1781/1/012025 · Cited by 11 · see citing works
Design and experimental analysis of continuous location tracking techniques for Wizard of Oz testing
Yang, L., Welbourne, E., & Landay, J. A. (2006) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1124772.1124924 · Cited by 8 · see citing works
Beyond errors: measuring reliability for error-prone interaction devices
Feng, J. H., & Sears, A. (2009) · Behaviour and Information Technology · doi:10.1080/01449290802196596 · Cited by 7 · see citing works
Realization of Contactless Elevator Control Panel System Based on Voice Interaction Technology
Liu, Y., Wang, W., & Li, Y. (2021) · International Conference on Control Systems, Mathematical Modeling, Automation and Energy Efficiency (SUMMA) · doi:10.1109/summa53307.2021.9632098 · Cited by 7 · see citing works
Analysis of Human Machine Interaction Design Perspective - A Comprehensive Literature Review
Ahamed, M. M. (2017) · International Journal on Contemporary Computer Research · Cited by 5 · see citing works
Google Assistant Controlled Home Automation with Voice Recognition
Yadav, S. S., Kumar, P., Kumar, S., & Singh, S. (2021) · International Conference on Advances in Computing, Communication Control and Networking (ICAC3N) · doi:10.1109/icac3n53548.2021.9725502 · Cited by 5 · see citing works
Voice Command Recognition for Movement Control of a 4-DoF Robot Arm
Rendyansyah, R., Prasetyo, A. P. P., & Sembiring, S. (2022) · ELKHA · doi:10.26418/elkha.v14i2.57556 · Cited by 4 · see citing works
Improving the performance of speech-gesture multimodal interface in non-ideal environments
Ambadar, F. F., & Martinez, J. J. L. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2022.12.173 · Cited by 3 · see citing works
Voice frequency impact for voice control of collaborative robots
Janíček, M., Holubek, R., Ružarovský, R., & Velíšek, K. (2021) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/1009/1/012025 · Cited by 2 · see citing works
Voice Controlled Robot Manipulator for Industrial Applications
Priyadarshana, V. P. S. C., Yaparathne, Y. M. P. G. R. R. Y., Jayasooriya, D. B. S., Thennakoon, T. M. T. S. D. B., Jayasekara, A. G. B. P., & Chandima, D. P. (2022) · IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) · doi:10.1109/iemcon56893.2022.9946545 · Cited by 2 · see citing works
Nzuva, S. M. (2021) · International Journal of Information Technology Control and Automation · doi:10.5121/ijitca.2021.11401 · Cited by 1 · see citing works
Flagged first, then by citations
1 of the 35 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Lamaakal, I., Maleh, Y., El Makkaoui, K., Ouahbi, I., Pławiak, P., Alfarraj, O., Almousa, M., & Abd El-Latif, A. A. (2025) · Sensors · doi:10.3390/s25051318 · Cited by 60 · see citing works
Welihinda, D. V. D. S., Gunarathne, L. K. P., Herath, H. M. K. K. M. B., Yasakethu, S. L. P., Madusanka, N., & Lee, B.-I. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e27777 · Cited by 23 · see citing works
Augmented intelligence with voice assistance and automated machine learning in Industry 5.0
Bousdekis, A., Foosherian, M., Fikardos, M., Wellsandt, S., Lepenioti, K., Bosani, E., Mentzas, G. N., & Thoben, K. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1538840 · Cited by 10 · see citing works
Robust Self-Localization of Wireless Acoustic Sensor Networks
Wang, X., Hu, D., Liu, R., & Bao, F. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3548616 · Cited by 4 · see citing works
Distributed-Robust Source Localization in Wireless Acoustic Sensor Networks
Wang, X., Hu, D., & Si, Q. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887576 · Cited by 2 · see citing works
Accelerating the Development of Machine Voice User Interfaces with Immersive Environments
Häfner, P., Eisenlohr, F., Michels, F. L., Karande, A., & Grethler, M. (2025) · 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw66409.2025.00390 · Cited by 3 · see citing works
A LLM-based voice user interface for voice dialogues between user and industrial machines
Mukherjee, A., Karande, A., Häfner, P., Poonia, M. D., Kimmig, A., Kreuzwieser, S., Vlas, R., Klar, M., Sykora, T., & Grethler, M. (2025) · Procedia CIRP · doi:10.1016/j.procir.2025.02.138 · Cited by 2 · see citing works
Automatic Speech Recognition for Human-Robot Interaction on The Humanoid Robot Barelang 7
Mawaddah, N., Tamba, C. R., Hutagalung, D. P., Nainggolan, F., Lubis, E. M., & Jamzuri, E. R. (2024) · International Conference on Applied Engineering (ICAE) · doi:10.4108/eai.7-11-2023.2342940 · Cited by 1 · see citing works
Schmidt, M., & Meitinger, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60412-6_13 · Cited by 1 · see citing works
How to Manipulate a High-Freedom Space Robotic Arm: Finding an Optimal Interaction Method
Zhang, Y. L., Huang, Y., Zhu, C., Chen, S., & Li, Z. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2626808 · Cited by 1 · see citing works
Khan, A., & Thurow, K. (2026) · Italian National Conference on Sensors · doi:10.3390/s26061840 · Cited by 1 · see citing works
Ohlenbusch, M., Rollwage, C., & Doclo, S. (2025) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-025-00418-1 · Cited by 1 · see citing works
Norda, M., Appell, J.-E., Lange, C. S., & Hahn, A. (2024) · wt Werkstattstechnik online · doi:10.37544/1436-4980-2024-03-38
Self-citationThe Interface of Minds Human-Machine Interaction in the Digital Age
Vanisri, K., & Padhy, P. C. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-2015-0.ch002
Dynamic Animation Scaling: Design and Development of Adaptive Character Animations for Varying Sizes
Vidisha, Viranjan, R., Shweta, & Raturi, V. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-5231-7_18
Enhancing Automatic Speech Recognition Accuracy Using a Gaussian Mixture Model (GMM)Preprint
Soundarya, M., Anusuya, S., & Narayanan, L. K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5089158
Wei, J., Lo, C., & Liu, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93739-2_19
Intelligent Multi-channel Fan Control System for Data Center Power Containers Using Siemens PLC
Wu, J., Kadoch, M., & Yao, Y. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-09694-4_2
Centimeter Differences in Wrist Electrode Placement Significantly Impact Myoelectric Performance
Olsen, C. D., Lewis, S. R., Gubler, J. D., Coleman, M. K., Davis, T. S., & George, J. A. (2025) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc58623.2025.11253120
Voice-Controlled Cnc Machine Operations Using Natural Language Processing
Paéz, I., Mora, R., Martinez, M., Molina, M., Arévalo, J., Pilco, A., & Quito, A. (2025) · IEEE Ecuador Technical Chapters Meeting (ETCM) · doi:10.1109/etcm67548.2025.11304525
Li, J., Bin, J., & Wu, X. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2610762
Hao, P., Wang, H., Mao, Y., Zhong, M., & Wang, J. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2631677
Calories365: An Innovative Approach to Calorie Tracking with Voice Input
Katerynych, L. O., Kubichka, M., & Zhereb, K. (2025) · doi:10.55056/ceur-ws.org/vol-4053/s_24_katerynych_kubichka_zhereb.pdf
Mukherjee, A., Vlas, R., Häfner, P., Kimmig, A., Wagner, M., & Aurich, J. C. (2026) · Procedia CIRP · doi:10.1016/j.procir.2026.03.190
A Voice Command-Based Digital Microfluidic Platform for Programmable Hands-Free Droplet Manipulation
Park, D., Kim, H., Kim, D., & Chung, S. K. (2026) · Applied Sciences · doi:10.3390/app16189232
Gahlawat, V., Aidibe, A., & Yacout, S. (2026) · Intelligent Methods, Systems, and Applications (IMSA) · doi:10.1109/imsa70415.2026.11700361
- Authors
- Norda, M., Engel, C., Rennies, J., Appell, J.-E., Lange, S. C., & Hahn, A.
- Affiliations
- Fraunhofer Institute for Digital Media Technology (DE)
- University of Applied Sciences Emden Leer (DE)
- Carl von Ossietzky Universität Oldenburg (DE)
- Venue
- IEEE Transactions on Automation Science and Engineering
- Published
- 2024-07
- Volume
- 21
- Issue
- 3
- Pages
- 4817-4828
- ISSN
- 1545-5955 (ISSN-L)
- 1558-3783
- OpenAlex
- W4386175710
- Cited by
- 26
- References
- 35
- Citations by year
- 2026: 7 · 2025: 13 · 2024: 6
- FWCI
- 4.50
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Automation Science and Engineering (publishedVersion)
- elib (German Aerospace Center) (acceptedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Funders
- Fraunhofer Institute for Digital Media Technology, Oldenburg Branch Hearing, Speech, and Audio Technology (HSA) through the Program ≫Vorab≪
- Lower Saxony Ministry of Science and Culture
- Volkswagen Foundation for its further development (76251-99-2/19 (ZN3812))
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- voice control, human-machine interaction, industrial automation, CNC milling machine, industrial robots, touch control, interaction time
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
