Decoding the agility of artificial intelligence-assisted human design teams
Open accessAbstract
Although necessary for complex problem solving, such as engineering design, team agility is often difficult to achieve in practice. The evolution of Artificial Intelligence (AI) affords unique opportunities for supporting team problem solving. While integrating assistive AI agents into human teams has at times improved team performance, it is still unclear if, how, and why AI affects team agility. A large-scale human experiment answers these questions, revealing that, with appropriately interfaced AIs, AI-assisted human teams enjoy improved coordination and communications, leading to better performance and adaptations to team disruptions, while devoting more effort to information handling and exploring the solution space more broadly. In sum, working with AI enables human team members to think more and act less.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI enhances students' adaptability, fosters reflection, and inspires more effective actions.
“AI also enhances students' adaptability, fosters reflection, and inspires more effective actions (Song et al., 2022).”
From Facilitator or hindrance? The impact of AI on university students' higher-order thinking skills in complex problem solving (2025) · cites this paper as Song et al., 2022
The Participants section identifies the participants as university students, and the Conclusions report that AI-assisted design teams were more agile, thoughtful, and effective in their actions. The findings concern teams in a design experiment; they do not establish improved adaptability or reflection as individual student outcomes. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Psychological Safety and Learning Behavior in Work Teams
Edmondson, A. (1999) · Administrative Science Quarterly · doi:10.2307/2666999 · Cited by 12,026 · see citing works
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880 · see citing works
Baum, L. E., Petrie, T., Soules, G., & Weiss, N. (1970) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697196 · Cited by 4,299 · see citing works
Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making
Jarrahi, M. H. (2018) · Business Horizons · doi:10.1016/j.bushor.2018.03.007 · Cited by 2,772 · see citing works
USING THE CORRECT STATISTICAL TEST FOR THE EQUALITY OF REGRESSION COEFFICIENTS
Paternoster, R., Brame, R. W., Mazerolle, P., & Piquero, A. R. (1998) · Criminology · doi:10.1111/j.1745-9125.1998.tb01268.x · Cited by 2,755 · see citing works
Enhancing the Effectiveness of Work Groups and Teams
Kozlowski, S. W. J., & Ilgen, D. R. (2006) · Psychological Science in the Public Interest · doi:10.1111/j.1529-1006.2006.00030.x · Cited by 2,623 · see citing works
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550 · see citing works
Mind Children: The Future of Robot and Human Intelligence
Truck, F., & Moravec, H. (1991) · Leonardo · doi:10.2307/1575314 · Cited by 1,280 · see citing works
Yusuf, Y. Y., Sarhadi, M., & Gunasekaran, A. (1999) · International Journal of Production Economics · doi:10.1016/s0925-5273(98)00219-9 · Cited by 1,148 · see citing works
Jansson, D. G., & Smith, S. M. (1991) · Design Studies · doi:10.1016/0142-694x(91)90003-f · Cited by 1,087 · see citing works
A methodology for achieving agility in manufacturing organisations : An introduction
Sharifi, H., & Zhang, Z. (1999) · International Journal of Production Economics · doi:10.1016/s0925-5273(98)00217-5 · Cited by 904 · see citing works
Dellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00595-2 · Cited by 826 · see citing works
Agile manufacturing: A framework for research and development
Gunasekaran, A. (1999) · International Journal of Production Economics · doi:10.1016/s0925-5273(98)00222-9 · Cited by 809 · see citing works
A review of enterprise agility: Concepts, frameworks, and attributes
Sherehiy, B., Karwowski, W., & Layer, J. K. (2007) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2007.01.007 · Cited by 767 · see citing works
Artificial life meets entertainment: lifelike autonomous agents
Maes, P. (1995) · CACM · doi:10.1145/219717.219808 · Cited by 606 · see citing works
Park, Y., Sawy, O. A. E., & Fiss, P. C. (2017) · Journal of the Association for Information Systems · doi:10.17705/1jais.00001 · Cited by 488 · see citing works
Natural Language Processing and Text Mining
Kao, A., & Poteet, S. R. (2007) · Springer eBooks · doi:10.1007/978-1-84628-754-1 · Cited by 476 · see citing works
Bernardes, E. S., & Hanna, M. (2009) · International Journal of Operations & Production Management · doi:10.1108/01443570910925352 · Cited by 390 · see citing works
Tallon, P. P. (2007) · Information Technology and Management · doi:10.1007/s10799-007-0024-8 · Cited by 350 · see citing works
The future of radiology augmented with Artificial Intelligence: A strategy for success
Liew, C. J. Y. (2018) · European Journal of Radiology · doi:10.1016/j.ejrad.2018.03.019 · Cited by 258 · see citing works
The agility construct on project management theory
Conforto, E. C., Amaral, D. C., da Silva, S. L., Felippo, A. D., & Kamikawachi, D. S. L. (2016) · International Journal of Project Management · doi:10.1016/j.ijproman.2016.01.007 · Cited by 257 · see citing works
Fixation or inspiration? A meta-analytic review of the role of examples on design processes
Sio, U. N., Kotovsky, K., & Cagan, J. (2015) · Design Studies · doi:10.1016/j.destud.2015.04.004 · Cited by 191 · see citing works
Self-citationA Document Analysis Method for Characterizing Design Team Performance
Dong, A., Hill, A. W., & Agogino, A. M. (2003) · Journal of Mechanical Design · doi:10.1115/1.1711818 · Cited by 188 · see citing works
Vickery, S. K., Droge, C., Setia, P., & Sambamurthy, V. (2010) · International Journal of Production Research · doi:10.1080/00207540903348353 · Cited by 174 · see citing works
The challenges of organizational agility (part 1)
Appelbaum, S. H., Calla, R., Desautels, D., et al. (2017) · Industrial and Commercial Training · doi:10.1108/ict-05-2016-0027 · Cited by 174 · see citing works
Business agility and diffusion of information technology
Mathiassen, L., & Pries‐Heje, J. (2006) · European Journal of Information Systems · doi:10.1057/palgrave.ejis.3000610 · Cited by 171 · see citing works
Military Team Research: 10 Years of Progress
Eduardo Salas, A., Bowers, C. A., & Cannon‐Bowers, J. A. (1995) · Military Psychology · doi:10.1207/s15327876mp0702_2 · Cited by 149 · see citing works
Relationships between Project Complexity and Communication
Senescu, R. R., Aranda‐Mena, G., & Haymaker, J. R. (2012) · Journal of Management in Engineering · doi:10.1061/(asce)me.1943-5479.0000121 · Cited by 139 · see citing works
A cautionary tale about the impact of AI on human design teams
Zhang, G., Raina, A., Cagan, J., & McComb, C. (2021) · Design Studies · doi:10.1016/j.destud.2021.100990 · Cited by 127 · see citing works
Self-citationTeam Structure and Team Building Improve Human–Machine Teaming With Autonomous Agents
Walliser, J. C., de Visser, E., Wiese, E., & Shaw, T. H. (2019) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343419867563 · Cited by 105 · see citing works
Stewart, G. L., & Barrick, M. R. (2000) · Academy of Management Journal · doi:10.5465/1556372 · Cited by 102 · see citing works
Team Coordination and Effectiveness in Human-Autonomy Teaming
Demir, M., Likens, A. D., Cooke, N. J., Amazeen, P. G., & McNeese, N. J. (2018) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2018.2877482 · Cited by 95 · see citing works
Williams, G., Meisel, N. A., Simpson, T. W., & McComb, C. (2019) · Journal of Mechanical Design · doi:10.1115/1.4044199 · Cited by 92 · see citing works
Self-citationThe influence of open goals on the acquisition of problem-relevant information.
Moss, J., Kotovsky, K., & Cagan, J. (2007) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.33.5.876 · Cited by 80 · see citing works
Self-citationWorkforce agility in operations management
Qin, R., & Nembhard, D. A. (2015) · Surveys in Operations Research and Management Science · doi:10.1016/j.sorms.2015.11.001 · Cited by 78 · see citing works
A hybrid approach to achieve organizational agility
Nejatian, M., Zarei, M. H., Nejati, M., & Zanjirchi, S. M. (2018) · Benchmarking An International Journal · doi:10.1108/bij-09-2016-0147 · Cited by 76 · see citing works
Exploring team performance in high-tech industries: Future trends of building up teamwork
Liu, M.-L., Liu, N., Ding, C. G., & Lin, C. (2014) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2014.03.014 · Cited by 75 · see citing works
Design Team Communication and Design Task Complexity: The Preference for Dialogues
den Otter, A., & Emmitt, S. (2008) · Architectural Engineering and Design Management · doi:10.3763/aedm.2008.0072 · Cited by 67 · see citing works
Idea generation with Technology Semantic Network
Sarica, S., Song, B., Luo, J., et al. (2021) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060421000020 · Cited by 67 · see citing works
Self-citationA conceptual model of team dynamical behaviors and performance in human-autonomy teaming
Demir, M., Cooke, N. J., & Amazeen, P. G. (2018) · Cognitive Systems Research · doi:10.1016/j.cogsys.2018.07.029 · Cited by 65 · see citing works
Explaining the emergence of team agility: a complex adaptive systems perspective
Werder, K., & Maedche, A. (2018) · Information Technology and People · doi:10.1108/itp-04-2017-0125 · Cited by 62 · see citing works
Mining Process Heuristics From Designer Action Data via Hidden Markov Models
McComb, C., Cagan, J., & Kotovsky, K. (2017) · Journal of Mechanical Design · doi:10.1115/1.4037308 · Cited by 53 · see citing works
Self-citationKABOOM: an agent-based model for simulating cognitive style in team problem solving
Lapp, S., Jablokow, K., & McComb, C. (2019) · Design Science · doi:10.1017/dsj.2019.12 · Cited by 50 · see citing works
Self-citationEvaluation of Latent Semantic Analysis-Based Measures of Team Communications Content
Gorman, J. C., Foltz, P. W., Kiekel, P. A., et al. (2003) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120304700337 · Cited by 48 · see citing works
Collective hybrid intelligence: towards a conceptual framework
Moradi, M., Moradi, M., Bayat, F., & Toosi, A. N. (2019) · International Journal of Crowd Science · doi:10.1108/ijcs-03-2019-0012 · Cited by 41 · see citing works
Schulz, A. P., & Fricke, E. (2003) · AIAA/IEEE Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc.1999.863677 · Cited by 38 · see citing works
Song, B., Zurita, N. F. S., Zhang, G., Stump, G. M., Balon, C., Miller, S. W., Yukish, M. A., Cagan, J., & McComb, C. (2020) · Proceedings of the Design Society DESIGN Conference · doi:10.1017/dsd.2020.68 · Cited by 38 · see citing works
Self-citationHybrid workplaces of the future
Kamar, E. S. (2016) · XRDS Crossroads The ACM Magazine for Students · doi:10.1145/3013488 · Cited by 34 · see citing works
Song, B., Zurita, N. F. S., Nolte, H., Singh, H., Cagan, J., & McComb, C. (2021) · Journal of Mechanical Design · doi:10.1115/1.4051871 · Cited by 33 · see citing works
Self-citationSocial learning in design teams: The importance of direct and indirect communications
Singh, V., Dong, A., & Gero, J. S. (2013) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060413000061 · Cited by 32 · see citing works
Exploring Biases Between Human and Machine Generated Designs
Lopez, C. E., Miller, S. R., & Tucker, C. S. (2018) · Journal of Mechanical Design · doi:10.1115/1.4041857 · Cited by 30 · see citing works
Self-citationVanLehn, K. A., Burkhardt, H., Cheema, S., Kang, S., Pead, D., Schoenfeld, A. H., & Wetzel, J. (2019) · Interactive Learning Environments · doi:10.1080/10494820.2019.1616567 · Cited by 30 · see citing works
Transferring Design Strategies From Human to Computer and Across Design Problems
Raina, A., Cagan, J., & McComb, C. (2019) · Journal of Mechanical Design · doi:10.1115/1.4044258 · Cited by 30 · see citing works
Self-citationAGILITY FACTORS AND THEIR IMPACT ON PRODUCT DEVELOPMENT PERFORMANCE
Rebentisch, E., Conforto, E. C., Schuh, G., Riesener, M., Kantelberg, J., Amaral, D. C., & Januszek, S. (2018) · Proceedings of the ... International Design Conference/Design ... · doi:10.21278/idc.2018.0236 · Cited by 24 · see citing works
Spatial Grammar-Based Recurrent Neural Network for Design Form and Behavior Optimization
Stump, G. M., Miller, S. W., Yukish, M. A., Simpson, T. W., & Tucker, C. S. (2019) · Journal of Mechanical Design · doi:10.1115/1.4044398 · Cited by 20 · see citing works
Self-citationThe Role of Design Team Interaction Structure on Individual and Shared Mental Models
Wood, M. D., Chen, P., Fu, K. K., Cagan, J., & Kotovsky, K. (2014) · Springer eBooks · doi:10.1007/978-94-017-9112-0_12 · Cited by 17 · see citing works
Self-citationData on the design and operation of drones by both individuals and teams
Zhang, G., Zurita, N. F. S., Stump, G. M., Song, B., Cagan, J., & McComb, C. (2021) · Data in Brief · doi:10.1016/j.dib.2021.107008 · Cited by 12 · see citing works
Self-citationDigital Supply Chains and the Human Factor
R, C. (2021) · Lecture notes in logistics · doi:10.1007/978-3-030-58430-6 · Cited by 10 · see citing works
Side-by-Side Human–Computer Design Using a Tangible User Interface
Law, M. V., Dhawan, N., Bang, H., Yoon, S., Selva, D., & Hoffman, G. (2019) · Springer eBooks · doi:10.1007/978-3-030-05363-5_9 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Bird, S., Klein, E., & Loper, E. (2009). Natural Language processing with Python: Analyzing text with the Natural Language toolkit. O'Reilly Media Inc.
- Dellermann, D., Ebel, P., S€ ollner, M., & Leimeister, J. M. (2019). Hybrid intelli- gence. Business and Information Systems Engineering, 61(5), 637e643.
- Dove, R. (1995). Measuring agility: The toll of turmoil. Production, 107(1), 16e18.
- Fliedner, G., & Vokurka, R. J. (1997). Agility: Competitive weapon of the 1990s and beyond? Production and Inventory Management Journal, 38(3), 19.
- Wilson, H. J., & Daugherty, P. R. (2018). Collaborative intelligence: Humans and AI are joining forces. Harvard Business Review, 96(4), 114e123.
- Yaghoubi, N.-M., Kazemi, M., Dahmardeh, M. R., & Arhami, F. (2011). Orga- nizational agility A way to import the advantageous aspects of competitive market (an analytical-comparative approach on agility models). Interdisci- plinary Journal of Contemporary Research in Business, 2(12), 766e783.
Cited by
Sorted by citations
Yin, H., Zhang, Z., & Liu, Y. (2023) · Systems · doi:10.3390/systems11120566 · Cited by 131 · see citing works
When and How to Use AI in the Design Process? Implications for Human-AI Design Collaboration
Lee, S.-Y., Law, M., & Hoffman, G. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2353451 · Cited by 97 · see citing works
Du, X., Du, M., Zhou, Z., & Bai, Y. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00534-0 · Cited by 44 · see citing works
“AI also enhances students’ adaptability, fosters reflection, and inspires more effective actions (Song et al., 2022).”
FOCUS AND MODALITY: DEFINING A ROADMAP TO FUTURE AI-HUMAN TEAMING IN DESIGN
McComb, C., Boatwright, P., & Cagan, J. (2023) · Proceedings of the Design Society · doi:10.1017/pds.2023.191 · Cited by 26 · see citing works
Self-citation
- Authors
- Song, B., Gyory, J. T., Zhang, G., Soria Zurita, N. F., Stump, G., Martin, J., Miller, S., Balon, C., Yukish, M., McComb, C., & Cagan, J.
- Affiliations
- Pennsylvania State University (US)
- Carnegie Mellon University (US)
- Corresponding author
- McComb, Christopher
- Cagan, Jonathan
- ORCID
- Venue
- Design Studies
- Publisher
- Elsevier BV
- Published
- 2022-03
- Volume
- 79
- Pages
- 101094
- ISSN
- 0142-694X (ISSN-L)
- OpenAlex
- W4213195259
- Cited by
- 59
- References
- 59
- Citations by year
- 2026: 15 · 2025: 14 · 2024: 16 · 2023: 10 · 2022: 4
- FWCI
- 10.39
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Design Studies (publishedVersion, cc-by-nc-nd)
- Funders
- Advanced Research Projects Agency
- Topics
- Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- team agility, human-AI collaboration, team coordination, team communication, team performance, engineering design, solution space exploration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
