Human Versus Artificial Intelligence: A Data-Driven Approach to Real-Time Process Management During Complex Engineering Design
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Managing the design process of teams has been shown to considerably improve problem-solving behaviors and resulting final outcomes. Automating this activity presents significant opportunities in delivering interventions that dynamically adapt to the state of a team in order to reap the most impact. In this work, an artificial intelligence (AI) agent is created to manage the design process of engineering teams in real time, tracking features of teams’ actions and communications during a complex design and path-planning task in multidisciplinary teams. Teams are also placed under the guidance of human process managers for comparison. Regarding outcomes, teams perform equally as well under both types of management, with trends toward even superior performance from the AI-managed teams. The managers’ intervention strategies and team perceptions of those strategies are also explored, illuminating some intriguing similarities. Both the AI and human process managers focus largely on communication-based interventions, though differences start to emerge in the distribution of interventions across team roles. Furthermore, team members perceive the interventions from both the AI and human manager as equally relevant and helpful, and believe the AI agent to be just as sensitive to the needs of the team. Thus, the overall results show that the AI manager agent introduced in this work is able to match the capabilities of humans, showing potential in automating the management of a complex design process.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This design methodology is applicable to scenarios such as collaborative manufacturing (determining what production metrics robots share) [34], autonomous vehicle coordination (controlling trajectory data broadcast), and distributed energy management (modulating demand forecasts between grid resources).”
From Adaptive Information Modulation: Designing Governance Mechanisms for Multi-Agent Artificial Intelligence Systems (2026) · cites this paper as [34]
Citations
Cites
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880
The Effects of Team Diversity on Team Outcomes: A Meta-Analytic Review of Team Demography
Horwitz, S. K., & Horwitz, I. B. (2007) · Journal of Management · doi:10.1177/0149206307308587 · Cited by 1,513
Team Diversity and Information Use
Dahlin, K. B., Weingart, L. R., & Hinds, P. J. (2005) · Academy of Management Journal · doi:10.5465/amj.2005.19573112 · Cited by 784
Branquinho, L. C., Carrião, M. S., Costa, A. S., Zufelato, N., Sousa, M. H., Miotto, R., Ivkov, R., & Bakuzis, A. F. (2014) · Scientific Reports · doi:10.1038/srep03637 · Cited by 491
Gardner, H. K. (2012) · Administrative Science Quarterly · doi:10.1177/0001839212446454 · Cited by 222
Collaborative Creativity—Group Creativity and Team Innovation
Paulus, P. B., Dzindolet, M., & Kohn, N. W. (2012) · Handbook of Organizational Creativity · doi:10.1016/b978-0-12-374714-3.00014-8 · Cited by 189
A Document Analysis Method for Characterizing Design Team Performance
Dong, A., Hill, A. W., & Agogino, A. M. (2004) · Journal of Mechanical Design · doi:10.1115/1.1711818 · Cited by 187
Relationships between Project Complexity and Communication
Senescu, R. R., Aranda‐Mena, G., & Haymaker, J. R. (2012) · Journal of Management in Engineering · doi:10.1061/(asce)me.1943-5479.0000121 · Cited by 139
The Strength of the Strongest Ties in Collaborative Problem Solving
de Montjoye, Y.-A., Stopczynski, A., Shmueli, E., Pentland, A., & Lehmann, S. (2014) · Scientific Reports · doi:10.1038/srep05277 · Cited by 128
A cautionary tale about the impact of AI on human design teams
Zhang, G., Raina, A., Cagan, J., & McComb, C. (2021) · Design Studies · doi:10.1016/j.destud.2021.100990 · Cited by 117
Self-citation
Research works printed but not linked: 1
- Cohen, G. P. , 1993, “The Virtual Design Team: An Information-Processing Model of Design Team Management,” Doctoral dissertation, Stanford University.
Cited by
Who is better in project planning?Generative artificial intelligence or project managers?
Barcauí, A., & Monat, A. (2023) · Project Leadership and Society · doi:10.1016/j.plas.2023.100101 · Cited by 245
Zhang, G., Chong, L., Kotovsky, K., & Cagan, J. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107536 · Cited by 147
Self-citationBartlett, K. A., & Camba, J. D. (2024) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2024.02.006 · Cited by 107
“…collaboration has been investigated in various stages of the design process, including early ideation and concept evaluation [14], later-stage ideation [15], management of the design team [16], aiding teams in design problem-solving [17], and aiding teams in the design of complex systems [18].”
BackgroundBiologically Inspired Design Concept Generation Using Generative Pre-Trained Transformers
Zhu, Q., Zhang, X., & Luo, J. (2023) · Journal of Mechanical Design · doi:10.1115/1.4056598 · Cited by 83
Opportunities for large language models and discourse in engineering design
Göpfert, J., Weinand, J. M., Kuckertz, P., & Stolten, D. (2024) · Energy and AI · doi:10.1016/j.egyai.2024.100383 · Cited by 53
“Gyory et al. [33, 34] analyze real time data of design teams to suggest measures from a predefined list if the communication or action frequency appears to be too low.”
BackgroundFOCUS AND MODALITY: DEFINING A ROADMAP TO FUTURE AI-HUMAN TEAMING IN DESIGN
McComb, C., Boatwright, P., & Cagan, J. (2023) · Proceedings of the Design Society · doi:10.1017/pds.2023.191 · Cited by 26
Self-citationHuman-Controllable AI: Meaningful Human Control
Liu, C., & Xu, W. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_58-1 · Cited by 26
Xu, Z., Hong, C. S., Zurita, N. F. S., Gyory, J. T., Stump, G. M., Nolte, H., Cagan, J., & McComb, C. (2024) · Journal of Mechanical Design · doi:10.1115/1.4064490 · Cited by 22
Self-citationChong, L., Kotovsky, K., & Cagan, J. (2024) · Journal of Mechanical Design · doi:10.1115/1.4064565 · Cited by 17
Self-citationEvaluating Designer Learning and Performance in Interactive Deep Generative Design
Chaudhari, A. M., & Selva, D. (2022) · Journal of Mechanical Design · doi:10.1115/1.4056374 · Cited by 13
Enhancing Supply Chain Management Through Artificial Intelligence: A Case Study of JD Logistics
Pan, Y., Wang, X., & Ye, Q. (2024) · Advances in Economics Management and Political Sciences · doi:10.54254/2754-1169/109/2024bj0127 · Cited by 12
Generative AI and empirical research methods in operations management
Shalpegin, T., Browning, T. R., Kumar, A., Shang, G., Thatcher, J. B., Fransoo, J. C., Holweg, M., & Lawson, B. (2025) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.1002/joom.1371
Gyory, J. T., Kotovsky, K., McComb, C., & Cagan, J. (2022) · Journal of Mechanical Design · doi:10.1115/1.4054723 · Cited by 12
Self-citationAI VS. HUMAN: THE PUBLIC'S PERCEPTIONS OF THE DESIGN ABILITIES OF ARTIFICIAL INTELLIGENCE
Chong, L., & Yang, M. C. (2023) · Proceedings of the Design Society · doi:10.1017/pds.2023.50 · Cited by 12
“…to these AIs that help in the early stages of the design process, there are many others that assist throughout or at the later stages of the design process, such as data-driven topology optimization, simulation, or process management AIs (Gyory et al., 2022; Jang et al., 2022; Pan et al., 2022).”
BackgroundTopcu, T. G., Husain, M., Ofsa, M., & Wach, P. (2025) · Systems Engineering · doi:10.1002/sys.21810 · Cited by 17
McGee, E. S., Cagan, J., & McComb, C. (2025) · Journal of Mechanical Design · doi:10.1115/1.4068909 · Cited by 9
Self-citationChong, L., Kotovsky, K., & Cagan, J. (2022) · Proceedings of the Design Society · doi:10.1017/pds.2022.155 · Cited by 9
“Therefore, AI agents are progressively deployed in human-AI design teams to achieve better joint performance, especially by assisting human designers to make better design decisions (Gyory et al., 2022; Zhang et al., 2021).”
BackgroundSelf-citationNavigating the intellectual terrain of AI–HRM integration: a bibliometric review (2000–2024)
Usman, M. B., Naveed, S., Nazeer, S., Siddique, N., & Rana, A. H. (2026) · Journal of Organizational Effectiveness People and Performance · doi:10.1108/joepp-08-2025-0670 · Cited by 10
A Systematic Review on the Implementation of Agility in Sustainable Design Development
Zhao, Z., Alli, H., Me, R. C., & Wang, S. (2023) · Designs · doi:10.3390/designs7050111 · Cited by 8
Lai, X., & Rau, P. P. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2545310 · Cited by 6
“Gyory et al. (2021) compared team design processes under the guidance and intervention of AI and human managers and observed equivalent team performance under the management types, thereby indicating that AI managers possessed similar capabilities to those of human managers.”
Wijaya, E. F. A., & Qamari, I. N. (2024) · International Research Journal of Multidisciplinary Scope · doi:10.47857/irjms.2024.v05i02.0383 · Cited by 8
“Artificial intelligence also plays a role in attracting high-potential individuals and evaluating them rapidly, investigating their eligibility according to job descriptions, designations, and estimates of additional value that can be provided to organizations (15).”
BackgroundThe Positive Effects of Employee AI Dependence on Voice Behavior—Based on Power Dependence Theory
Liu, J., Huang, M., Cui, M., Tian, G., & Li, X. (2025) · Behavioral Sciences · doi:10.3390/bs15121709 · Cited by 9
“By supporting employees in managing and executing work in ways comparable to human leaders (Gyory et al., 2022), AI reduces their reliance on supervisors for direction, expertise, and feedback.”
Drivers and constraints of agentic AI in supply chain management: an exploratory study
Trifone, N., Dallari, F., & Brintrup, A. M. (2026) · International Journal of Production Research · doi:10.1080/00207543.2026.2669878 · Cited by 8
“In this regard, authors have proposed frameworks that classify the roles of AI and formalise its introduction with humans based on the tasks performed (Chenna 2024; Gyory et al. 2022; McComb, Boatwright, and Cagan 2023; Vann Yaroson, Abadie, and Roux 2025).”
Large Language Models and the Enhancement of Human Cognition: Some Theoretical Insights
Diržytė, A. (2025) · Filosofija Sociologija · doi:10.6001/fil-soc.2025.36.1.2 · Cited by 5
Data on the Human Versus artificial intelligence process management experiment
Zurita, N. F. S., Gyory, J. T., Balon, C., Martin, J. D., Kotovsky, K., Cagan, J., & McComb, C. (2022) · Data in Brief · doi:10.1016/j.dib.2022.107917 · Cited by 6
Self-citationMeta-X design: from tools to partners in human–machine co–creation for industry 5.0
Yu, C., Yan, T., & Chen, Y. (2025) · Journal of Engineering Design · doi:10.1080/09544828.2025.2571005 · Cited by 4
Comparing Strategies for Visualizing the High-Dimensional Exploration Behavior of CPS Design Agents
Agrawal, A. D., & McComb, C. (2022) · IEEE Workshop on Design Automation for CPS and IoT (DESTION) · doi:10.1109/destion56136.2022.00017 · Cited by 9
“In that context, communication between human and AI is known to be critical [4], [5].”
BackgroundSelf-citationChen, Q., Ilami, S., Lore, N., & Heydari, B. (2026) · Journal of Mechanical Design · doi:10.1115/1.4070755 · Cited by 3
Gyory, J. T., Zurita, N. F. S., Cagan, J., & McComb, C. (2023) · Journal of Mechanical Design · doi:10.1115/1.4056597 · Cited by 3
Self-citationMirsch, M., Knoth, N., Schultz, B., Bata, K., Schleiß, J., & Leicht-Scholten, C. (2026) · European Journal of Engineering Education · doi:10.1080/03043797.2026.2620482 · Cited by 3
“Across domains, AI supports complex decision-making and system optimisation (Yüksel et al. 2023) and enhances collaborative workflows (Gyory et al. 2022).”
Estes, J., & Pandey, V. (2023) · Mathematics · doi:10.3390/math11214533 · Cited by 2
It Takes Two to Trust: Mediating Human-AI Trust for Resilience and Reliability
Zerick, J. N., Kaufman, Z., Ott, J., Kuber, J., Chow, E., Shah, S., & Lewis, G. F. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00145 · Cited by 2
“…over 2,000 studies was conducted to systematically evaluate the factors influencing human-human trust (primarily in teams and organizations), concluding that the reputation of the trustee and the shared closeness between trustor and trustee as pivotal predictors of trustworthiness [21].”
BackgroundHUVER: The HyForm Uncrewed Vehicle Engineering Repository
Karri, A., Stump, G. M., McComb, C., & Song, B. (2025) · Journal of Mechanical Design · doi:10.1115/1.4067711 · Cited by 5
Self-citationGyory, J. T., Kotovsky, K., & Cagan, J. (2022) · Proceedings of the Design Society · doi:10.1017/pds.2022.160 · Cited by 2
“These results are interesting because when comparing the same questions by the actual team conditions (human-managed teams versus AI-managed teams), no differences emerged (Gyory et al., 2022) showing that the belief that the management was human rather than the reality is what mattered.”
BackgroundMethodResultInfluentialSelf-citationЛатыпова, В. А. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-3-031-82494-4_25 · Cited by 1
Kim, Y. J., & Heo, I. (2023) · International Journal of Evaluation and Research in Education (IJERE) · doi:10.11591/ijere.v12i2.23786 · Cited by 4
Patel, A., & Summers, J. D. (2023) · Journal of Engineering Design · doi:10.1080/09544828.2023.2271772 · Cited by 1
Bartlett, K. A., Young, A., & Camba, J. D. (2024) · Journal of Integrated Design and Process Science · doi:10.3233/jid-240033
Advancements in Behavioral AI: A Bibliometric Analysis
Rad, D. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9158-7.ch002
Testing AI Models: The Human Factor in Ensuring Accuracy, Fairness, and Transparency
Choubey, A. (2025) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit251112238
Performance Optimization of Hybrid Azure AD Join Across Multi-Forest Deployments
Gannavarapu, P. (2025) · Journal of Information Systems Engineering & Management · doi:10.52783/jisem.v10i45s.8897
“AI and ML can be trained to identify and resolve conflicts in real-time, eliminating the need for human intervention (Gyory et al., 2022).”
Lv, J. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251380813
RetractedXu, Z., Martelaro, N., & McComb, C. (2026) · Journal of Mechanical Design · doi:10.1115/1.4072720
Self-citationMajumder, S., & Dey, N. (2025) · Studies in computational intelligence · doi:10.1007/978-3-032-15735-5_2 · Cited by 1
Müller, W., Wichmann, J., Liebetrau, D., Leyer, M., & Engin, A. (2026) · The Bottom Line Managing Library Finances · doi:10.1108/bl-02-2025-0030
Abubakar, A. M., Ferhoune, I., Ndaba, S., Bilal, M. T., & Abdul, A. Z. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-36412-9.00010-8
Neuro-Optimization and Revolutionizing Product and Ad Design Using AI
Rao, T. V. N., Reddy, G. H. V., Gowda, K. S. K., & Kumar, B. S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1152-4.ch009
The Art of Facilitation: Building Trust and Psychological Safety in Design Teams
Witherell, C. J., Maar, A., Dougherty, P., & Menold, J. (2026) · Journal of Mechanical Design · doi:10.1115/1.4071514
Dehan, M. F. Z., Disha, J. F., & Anzum, K. M. T. (2026) · International Journal of Lean Six Sigma · doi:10.1108/ijlss-09-2025-0251 · Cited by 4
Giovenali, N., Bruno, G., Chiabert, P., & Segonds, F. (2026) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-026-17892-2
A Methodology for Objective Design Fixation Analysis in Constrained Design Tools
Hunter, J. G., Schonlau, S. L., Jain, N., & Reid, T. (2026) · Journal of Mechanical Design · doi:10.1115/1.4071923
Human-Controllable AI: Meaningful Human Control
Liu, C., & Xu, W. (2026) · Springer eBooks · doi:10.1007/978-981-95-3658-0_58
Design Management in Industry 5.0: Synergy of AI, Humans, and Machines
Nezhad, A. M. A., Amirkhizi, P. J., Pedrammehr, S., Aghdam, Z. H., & Soleimanzadeh, M. (2026) · Administrative Sciences · doi:10.3390/admsci16070333
“Recent advances in AI, machine learning, and digital twin technologies have further expanded opportunities for optimizing resource use, reducing waste, and improving sustainability performance across design and manufacturing processes (Denuwara et al., 2022; Gyory et al., 2022; Zhang et al., 2023).”
How Creative Design Teams Make Sense of Contrasting AI Personas
Ma, K., Won, D., Zeng, Y., Chung, J., & Goucher-Lambert, K. (2026) · Creativity and Cognition · doi:10.1145/3803784.3816826
“Studies of AI facilitation suggest that process-managing agents can guide teams through a design task comparable to human facilitators [13] and conversational cues can help steer real-time brainstorming [32].”
Ammersdörfer, T., Tartler, D., Kauffeld, S., & Inkermann, D. (2026) · Gruppe Interaktion Organisation Zeitschrift für Angewandte Organisationspsychologie (GIO) · doi:10.1007/s11612-026-00887-8 · Cited by 1
“So können KI-basierte Interventionen beispielsweise Kommunikations-schwächen aufdecken, kognitive Verzerrungen bei der Entscheidungsfindung hervorheben und strukturierte Feed-backschleifen anbieten, die die Selbstreflexion in Teams verbessern (Gyory et al. 2022).”
The Tale of the Broken Clock: How LLMs Evaluate Requirement Quality and Where They Fail
Shefa, J., Iyer, A., & Topcu, T. G. (2026) · INCOSE International Symposium · doi:10.1002/iis2.70245
“Today, significant research is being conducted on both human-AI dyads (Singh & Szajn-farber, 2025) and teams of humans with AI advisors (Gyory et al., 2021; Song et al., 2022).”
Mapping the Human-AI Collaboration Trends in Design Research: A Systematic Review
Figoli, F. A., & Rampino, L. (2026) · Interaction design & architecture(s)/ID&A Interaction design & architecture(s) · doi:10.55612/s-5002-069-012
Teamflow und Entscheidungen: Empirische Analyse in kollaborativen Konstruktionsprozessen
Henkel, M., Kull, P., Musawi, A., van Remmen, J., Förster, F., Götz, S., Bursac, N., Wartzack, S. J., & Ritzer, K. (2026) · Symposium Design for X (DFX) · doi:10.35199/dfx2026.24
Innovation Machine: Human-LLM Teaming for Aeronautics Research and Discovery
Meadows, M. S., Shyam, V., Villarrubia, J., Miller, A., Mahota, T., Chin, J., Wheeler, K. R., Moore, G., Brubaker, E. R., Kinch, S. A., et al. (2025) · doi:10.64631/gelp3414
Artificial Intelligence In Product Development: Perspectives From Academia And Industry
Henke, J., Berger, E., Weippert, J., & Paetzold-Byhain, K. (2026) · doi:10.35199/norddesign2026.38
Metadata
- Authors
- Gyory, J. T., Soria Zurita, N. F., Martin, J., Balon, C., McComb, C., Kotovsky, K., & Cagan, J.
- Affiliations
- Carnegie Mellon University (US)
- Pennsylvania State University School of Engineering Design, Technology, and Professional Programs, , University Park, State College, PA 16802 ;
- Applied Research Laboratory at Penn State (US)
- Venue
- Journal of Mechanical Design
- Publisher
- ASME International
- Published
- 2021-10-14
- Volume
- 144
- Issue
- 2
- ISSN
- 0161-8458 (ISSN-L)
- 1050-0472
- 1528-9001
- OpenAlex
- W3200185945
- Cited by
- 56
- References
- 42
- Citations by year
- 2026: 17 · 2025: 13 · 2024: 9 · 2023: 10 · 2022: 7
- FWCI
- 6.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Mechanical Design (publishedVersion)
- Funders
- Air Force Office of Scientific Research (FA9550-18-1-0088)
- Defense Advanced Research Projects Agency (N66001-17-1-4064)
- Topics
- Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- AI agents, design process management, engineering teams, multidisciplinary team, team communication, adaptive interventions, team performance, team member perceptions, team roles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.