Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
Open accessAbstract
Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the benefits and deficiencies of existing tools, as well as the respective roles of the human and automation in ML workflows. Finally, we discuss design implications for the future of Auto-ML tool development. We argue that instead of full automation being the ultimate goal of Auto-ML, designers of these tools should focus on supporting a partnership between the user and the Auto-ML tool. This means that a range of Auto-ML tools will need to be developed to support varying user goals such as simplicity, reproducibility, and reliability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Commercial AutoML offerings can be categorised broadly into two groups: those offered by major cloud providers and those offered by dedicated AutoML organisations.
“The commercial Au-toML oferings can be categorised broadly into two groups: those ofered by major cloud providers (e.g., Google, Microsoft), and those ofered by dedicated AutoML organisations (often startups, e.g., DataRobot, H2O.ai, 4Paradigm) [157].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [157]
In one of the few studies exploring practitioner experiences using AutoML tools and platforms, Xin et al. demonstrated a dissonance between those building AutoML tooling seeking to achieve 'full automation' and that of actual users for whom 'a complete automation is neither a requirement nor a desired outcome'.
“Similarly, in one of the few studies exploring practitioner experiences using AutoML tools and platforms, Xin et al. demonstrated a dissonance between those building AutoML tooling seeking to achieve”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [157]
Users expressed interest in a more 'human-in-the-loop' AutoML that aims to empower users in undertaking ML development, rather than replace them completely.
“Instead, they found that users expressed interest in a more”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [157]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Data Structures for Statistical Computing in Python
McKinney, W. (2010) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-92bf1922-00a · Cited by 11,511 · see citing works
The NumPy Array: A Structure for Efficient Numerical Computation
van der Walt, S., Colbert, S. C., & Varoquaux, G. P. (2011) · Computing in Science & Engineering · doi:10.1109/mcse.2011.37 · Cited by 10,445 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1× · see citing works
“Conversely, lack of agency and tinkering can result in a lack of interpretability and the type of non-transparency caused by illiteracy [10].”
Background(2015) · neural information processing systems · Cited by 1,305 · see citing works
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206 · see citing works
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R. A., Gall, H. C., Kamar, E. S., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,181 · see citing works
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · see citing works
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective
Roh, Y., Heo, G., & Whang, S. E. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2946162 · Cited by 980 · see citing works
Auto-Keras: An Efficient Neural Architecture Search System
Jin, H., Song, Q., & Hu, X. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330648 · Cited by 963 · see citing works
The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility
Ackerman, M. S. (2000) · Human–Computer Interaction · doi:10.1207/s15327051hci1523_5 · Cited by 892 · see citing works
Snorkel: rapid training data creation with weak supervision
Ratner, A., Bach, S. H., Ehrenberg, H. R., Fries, J., Wu, S., & Ré, C. (2019) · The VLDB Journal · doi:10.1007/s00778-019-00552-1 · Cited by 808 · see citing works
Ratner, A., Bach, S. H., Ehrenberg, H. R., Fries, J., Wu, S., & Ré, C. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3157794.3157797 · Cited by 790 · see citing works
Serverless Computing: Current Trends and Open Problems
Baldini, I., Castro, P., Chang, K. S.-P., Cheng, P., Fink, S. J., Ishakian, V., Mitchell, N., Muthusamy, V., Rabbah, R., Slominski, A., et al. (2017) · Springer eBooks · doi:10.1007/978-981-10-5026-8_1 · Cited by 645 · see citing works
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633 · see citing works
Proceedings of the 27th International Conference on Neural Information Processing Systems
Zoubin Ghahramani, A., Welling, M., Cortes, C., Lawrence, N. D., & Weinberger, K. Q. (2014) · Cited by 481 · see citing works
Scaling tree-based automated machine learning to biomedical big data with a feature set selector
Le, T. T., Fu, W., & Moore, J. H. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz470 · Cited by 456 · see citing works
Enterprise Data Analysis and Visualization: An Interview Study
Kandel, S., Paepcke, A., Hellerstein, J. M., & Heer, J. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.219 · Cited by 443 · see citing works
Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359313 · Cited by 342 · see citing works
Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow
Wongsuphasawat, K., Smilkov, D., Wexler, J., Wilson, J., Mané, D., Fritz, D., Krishnan, D., Viégas, F. B., & WATTENBERG, M. P. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744878 · Cited by 332 · see citing works
Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702509 · Cited by 263 · see citing works
Talbot, J., Lee, B., Kapoor, A., & Tan, D. S. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518895 · Cited by 226 · see citing works
Human factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219 · see citing works
Human model evaluation in interactive supervised learning
Fiebrink, R., Cook, P. R., & Trueman, D. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978965 · Cited by 207 · see citing works
AdaNet: Adaptive Structural Learning of Artificial Neural NetworksPreprint
Cortes, C., Gonzalvo, X., Kuznetsov, V., Mohri, M., & Yang, S. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.01097 · Cited by 182 · see citing works
Grounding Interactive Machine Learning Tool Design in How Non-Experts Actually Build Models
Yang, Q., Suh, J., Chen, N.-C., & Ramos, G. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196729 · Cited by 156 · see citing works
A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration PerspectivePreprint
Roh, Y., Heo, G., & Whang, S. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03402 · Cited by 150 · see citing works
Interactive machine teaching: a human-centered approach to building machine-learned models
Ramos, G., Meek, C., Simard, P., Suh, J., & Ghorashi, S. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1734931 · Cited by 120 · see citing works
Drozdal, J., Weisz, J., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M., Ju, L., & Su, H. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377501 · Cited by 119 · see citing works
Wang, Q., Ming, Y., Jin, Z., Shen, Q., Liu, D., Smith, M. J., Veeramachaneni, K. K., & Qu, H. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300911 · Cited by 113 · see citing works
Gestalt: integrated support for implementation and analysis in machine learning
Patel, K., Bancroft, N., Drucker, S. M., Fogarty, J., Ko, A. J., & Landay, J. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866038 · Cited by 112 · see citing works
Understanding and Visualizing Data Iteration in Machine Learning
Hohman, F., Wongsuphasawat, K., Kery, M. B., & Patel, K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376177 · Cited by 110 · see citing works
Towards human-guided machine learning
Gil, Y., Honaker, J., Gupta, S., Ma, Y., D'Orazio, V. J., Garijo, D., Gadewar, S. P., Yang, Q., & Jahanshad, N. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302324 · Cited by 80 · see citing works
Research Advances in Cloud Computing
Buyya, R., Somani, G., & Chaudhary, S. (2017) · Springer eBooks · doi:10.1007/978-981-10-5026-8 · Cited by 72 · see citing works
Serverless Computing: Current Trends and Open ProblemsPreprint
Baldini, I., Castro, P., Chang, K. S.-P., Cheng, P., Fink, S. J., Ishakian, V., Mitchell, N., Muthusamy, V., Rabbah, R., Slominski, A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.03178 · Cited by 68 · see citing works
Weidele, D. K. I., Weisz, J. D., Oduor, E., Muller, M. J., Andrés, J., Gray, A., & Wang, D. (2020) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3377325.3377538 · Cited by 58 · see citing works
A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead.
Lee, D. J. L., Macke, S., Xin, D., Lee, A. H.-C., Huang, S., & Parameswaran, A. (2019) · IEEE Data(base) Engineering Bulletin · Cited by 53 · see citing works
Self-citationThe effects of mixing machine learning and human judgment
Vaccaro, M., & Waldo, J. (2019) · Communications of the ACM · doi:10.1145/3359338 · Cited by 49 · see citing works
Xin, D., Macke, S., Ma, L., Liu, J., Song, S., & Parameswaran, A. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3297753.3297763 · Cited by 48 · see citing works
Self-citationLudwig: a type-based declarative deep learning toolboxPreprint
Molino, P., Dudin, Y., & Miryala, S. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.07930 · Cited by 28 · see citing works
Bringer, E., Israeli, A., Shoham, Y., Ratner, A., & Ré, C. (2019) · doi:10.1145/3329486.3329492 · Cited by 19 · see citing works
Demystifying a Dark Art: Understanding Real-World Machine Learning Model DevelopmentPreprint
Lee, A. H.-C., Xin, D., Lee, D., & Parameswaran, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.01520 · Cited by 8 · see citing works
Self-citationAn Application of GPU Acceleration in CFD Simulation for Insect Flight
Yao, Y., & Yeo, K. S. (2017) · Supercomputing Frontiers and Innovations · doi:10.14529/jsfi170202 · Cited by 7 · see citing works
PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines
Ono, J. P., Castelo, S., López, R., Bertini, E., Freire, J., & Silva, C. T. (2020) · PubMed · doi:10.48550/arxiv.2005.00160 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Shoumik Palkar, James J Thomas, Anil Shanbhag, Deepak Narayanan, Holger Pirk, Malte Schwarzkopf, Saman Amarasinghe, Matei Zaharia, and Stanford InfoLab. 2017. Weld: A common runtime for high performance data analytics. In Conference on Innovative Data Systems Research (CIDR). CIDR, Chaminade, California, 45.
- Daniel Karl I. Weidele, Justin D. Weisz, Eno Oduor, Michael Muller, Josh An- dres, Alexander Gray, and Dakuo Wang. 2019. AutoAIViz: Opening the Black- box of Automated Artificial Intelligence with Conditional Parallel Coordinates. arXiv:cs.LG/1912.06723
- Doris Xin, Stephen Macke, Litian Ma, Jialin Liu, Shuchen Song, and Aditya Parameswaran. 2018. Helix: holistic optimization for accelerating iterative ma- chine learning. Proceedings of the VLDB Endowment 12, 4 (2018), 446-460.
Cited by
Sorted by citations
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228 · see citing works
AutoML: A systematic review on automated machine learning with neural architecture search
Salehin, I., Islam, M. S., Saha, P., Noman, S. M., Tuni, A., Hasan, M. M., & Baten, M. A. (2023) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2023.10.002 · Cited by 194 · see citing works
IoT data analytics in dynamic environments: From an automated machine learning perspective
Yang, L., & Shami, A. A. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105366 · Cited by 100 · see citing works
Automated evolutionary approach for the design of composite machine learning pipelines
Nikitin, N. O., Vychuzhanin, P., Sarafanov, M., Polonskaia, I. S., Revin, I., Barabanova, I. V., Maximov, G., Kalyuzhnaya, A. V., & Boukhanovsky, A. V. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.08.022 · Cited by 89 · see citing works
Automated Machine Learning (AutoML): The Future of Computational Intelligence
Mengi, G., Singh, S., Kumar, S., Mahto, D., & Sharma, A. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-22018-0_28 · Cited by 58 · see citing works
Zheng, C., Wang, D., Wang, A. Y., & Ma, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517615 · Cited by 54 · see citing works
Benchmarking AutoML for regression tasks on small tabular data in materials design
Conrad, F., Mälzer, M., Schwarzenberger, M., Wiemer, H., & Ihlenfeldt, S. (2022) · Scientific Reports · doi:10.1038/s41598-022-23327-1 · Cited by 49 · see citing works
Feng, K. J. K., Coppock, M. J., & McDonald, D. W. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596101 · Cited by 41 · see citing works
Lethbridge, T. C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89159-6_14 · Cited by 40 · see citing works
Balayn, A., Yurrita, M., Yang, J., & Gadiraju, U. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604674 · Cited by 40 · see citing works
AutoML in The Wild: Obstacles, Workarounds, and Expectations
Sun, Y., Song, Q., Gui, X., Ma, F., & Wang, T. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581082 · Cited by 31 · see citing works
Lewicki, K., Lee, M. S. A., Cobbe, J., & Singh, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581463 · Cited by 30 · see citing works
“demonstrated a dissonance between those building AutoML tooling seeking to achieve “full automation” and that of actual users for whom “a complete automation is neither a requirement nor a desired outcome” [153].”
BackgroundMethodInfluentialShankar, S., Garcia, R., Hellerstein, J. M., & Parameswaran, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653697 · Cited by 106 · see citing works
Self-citationWang, K., Hou, W., Hong, L., & Guo, J. (2025) · Electronics · doi:10.3390/electronics14030420 · Cited by 27 · see citing works
Enhancing cryptocurrency market volatility forecasting with daily dynamic tuning strategy
Feng, L., Qi, J., & Lucey, B. M. (2024) · International Review of Financial Analysis · doi:10.1016/j.irfa.2024.103239 · Cited by 25 · see citing works
A multivocal literature review on the benefits and limitations of industry-leading AutoML tools
Quaranta, L., Azevedo, K., Calefato, F., & Kalinowski, M. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107608 · Cited by 24 · see citing works
How Domain Experts Work with Data: Situating Data Science in the Practices and Settings of Craftwork
Jung, J. Y., Steinberger, T., King, J. L., & Ackerman, M. S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512905 · Cited by 24 · see citing works
An Empirical Study on the Usage of Automated Machine Learning Tools
Majidi, F., Openja, M., Khomh, F., & Li, H. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00014 · Cited by 24 · see citing works
Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
Weerts, H., Pfisterer, F., Feurer, M., Eggensperger, K., Bergman, E., Awad, N. H., Vanschoren, J., Pechenizkiy, M., Bischl, B., & Hutter, F. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14747 · Cited by 27 · see citing works
Feng, K. J. K., & McDonald, D. W. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584064 · Cited by 22 · see citing works
Rahman, S., & Kandogan, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502068 · Cited by 20 · see citing works
A comprehensive survey on machine learning applications for drilling and blasting in surface mining
Munagala, V. S. K. R., Thudumu, S., Logothetis, I., Bhandari, S., Vasa, R., & Mouzakis, K. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100517 · Cited by 18 · see citing works
“It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation
Showkat, D., & Baumer, E. P. S. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533483 · Cited by 18 · see citing works
AutoML from Software Engineering Perspective: Landscapes and Challenges
Wang, C., Chen, Z., & Zhou, M. (2023) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr59073.2023.00019 · Cited by 17 · see citing works
HAMLET: A framework for Human-centered AutoML via Structured Argumentation
Francia, M., Giovanelli, J., & Pisano, G. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.12.035 · Cited by 14 · see citing works
How far are we with automated machine learning? characterization and challenges of AutoML toolkits
Alamin, M. A. A., & Uddin, G. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10450-y · Cited by 14 · see citing works
Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey
Gu, Y., You, H., Cao, J., Yu, M., Fan, H., & Qian, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3773084 · Cited by 23 · see citing works
Prinzel, L. J., Krois, P. A., Ellis, K. K. E., Vincent, M. J., Stephens, C. L., Oza, N. C., Chancey, E. T., Davies, M., Mah, R. W., Ackerson, J., et al. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-1603 · Cited by 11 · see citing works
Goodman, S. M., McDonnell, E. J., Froehlich, J. E., & Findlater, L. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713294 · Cited by 11 · see citing works
Soft-Sensor for Class Prediction of the Percentage of Pentanes in Butane at a Debutanizer Column
Niño-Adan, I., Landa-Torres, I., Manjarrés, D., Portillo, E., & Orbe, L. (2021) · Sensors · doi:10.3390/s21123991 · Cited by 11 · see citing works
The Benefits of Automated Machine Learning in Hospitality: A Step-By-Step Guide and AutoML Tool
Castelli, M., Pinto, D. C., Shuqair, S., Montali, D., & Vanneschi, L. (2022) · Emerging Science Journal · doi:10.28991/esj-2022-06-06-02 · Cited by 11 · see citing works
Tracing and Visualizing Human-ML/AI Collaborative Processes through Artifacts of Data Work
Rogers, J., & Crisan, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580819 · Cited by 11 · see citing works
Diving Into AutoML in Medical Imaging: Solution for Non-ML Practitioners
Rodrigues, A. M. M., de Almeida, T. D., Silva, L. A. B., & Costa, C. M. A. (2024) · IEEE Access · doi:10.1109/access.2024.3441469 · Cited by 10 · see citing works
Assessing the Use of AutoML for Data-Driven Software Engineering
Calefato, F., Quaranta, L., Lanubile, F., & Kalinowski, M. (2023) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem56168.2023.10304796 · Cited by 10 · see citing works
Enabling Collaborative Data Science Development with the Ballet Framework
Smith, M. J., Cito, J., Lu, K., & Veeramachaneni, K. K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479575 · Cited by 9 · see citing works
Schofield, A., Wu, S., de Volo, T. B., Kuze, T., Gomez, A., & Sultana, S. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701201 · Cited by 9 · see citing works
A comparative analysis of ML techniques for bug report classification
Laiq, M., bin Ali, N., Börstler, J., & Engström, E. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112457 · Cited by 9 · see citing works
Bär, A., Uhrig, J., Umesh, J. P., Cordts, M., & Fingscheidt, T. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00399 · Cited by 8 · see citing works
Practitioner Motives to Use Different Hyperparameter Optimization Methods
Kannengießer, N., Hasebrook, N., Morsbach, F., Zöller, M.-A., Franke, J. K. H., Lindauer, M., Hutter, F., & Sunyaev, A. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3745771 · Cited by 8 · see citing works
A Model for Types and Levels of Automation in Visual Analytics: A Survey, a Taxonomy, and Examples
Domova, V., & Vrotsou, K. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3163765 · Cited by 8 · see citing works
Towards a Non-Ideal Methodological Framework for Responsible ML
Mothilal, R. K., Guha, S., & Ahmed, S. I. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642501 · Cited by 9 · see citing works
EDCA – An Evolutionary Data-Centric AutoML Framework for Efficient Pipelines
Simões, J., & Correia, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-90065-5_5 · Cited by 6 · see citing works
Evaluating Models with Dynamic Sampling Holdout in Auto-ML
Larcher, C. H. N., & Barbosa, H. J. C. (2022) · SN Computer Science · doi:10.1007/s42979-022-01406-4 · Cited by 6 · see citing works
Unlocking the Tacit Knowledge of Data Work in Machine Learning
Cha, I., Oh, J., Park, C. Y., Han, J., & Lee, H. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585616 · Cited by 6 · see citing works
How Do Categorical Duplicates Affect ML? A New Benchmark and Empirical Analyses
Shah, V., Parashos, T. J., & Kumar, A. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3648160.3648178 · Cited by 6 · see citing works
Tooling for Developing Data-Driven Applications: Overview and Outlook
Weber, T., & Hußmann, H. (2022) · Mensch und Computer (MuC) · doi:10.1145/3543758.3543779 · Cited by 6 · see citing works
Demystifying Artificial Intelligence for Data Preparation
Chai, C., Tang, N., Fan, J., & Luo, Y. (2023) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3555041.3589406 · Cited by 6 · see citing works
AssistML: an approach to manage, recommend and reuse ML solutions
Zacarias, A. G. V., Reimann, P., Weber, C., & Mitschang, B. (2023) · International Journal of Data Science and Analytics · doi:10.1007/s41060-023-00417-5 · Cited by 6 · see citing works
Momin, M., & Ali, O. (2023) · HighTech and Innovation Journal · doi:10.28991/hij-2023-04-03-014 · Cited by 6 · see citing works
Human-Centered AI for Data Science: A Systematic ApproachPreprint
Wang, D., Ma, X., & Wang, A. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01108 · Cited by 6 · see citing works
Bayero, S. A., Manoharan, G., & Kumar, S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0129-7.ch002 · Cited by 5 · see citing works
Hong, M.-H., & Crisan, A. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3744750 · Cited by 6 · see citing works
AutoML as a Catalyst for Predictive Maintenance Innovation: Strategies and Outcomes
Jayasurya, B., Suguna, M., Saravanan, P., & Revathi, M. (2024) · International Conference on Artificial Intelligence for Internet of Things (AIIoT) · doi:10.1109/aiiot58432.2024.10574753 · Cited by 5 · see citing works
Ko, E., & Kang, P. (2025) · IEEE Access · doi:10.1109/access.2025.3553870 · Cited by 5 · see citing works
Fofanah, A. J., Koroma, S., & Bangura, H. I. (2023) · British journal of healthcare and medical research · doi:10.14738/bjhmr.102.14427 · Cited by 5 · see citing works
Toward Artificial General Intelligence: Deep Reinforcement Learning Method to AI in Medicine
Nguyen, D. S. W., & Odigie, R. (2023) · Journal of Computer and Communications · doi:10.4236/jcc.2023.119006 · Cited by 5 · see citing works
Predicting Delays in Indian Lower Courts Using AutoML and Decision Forests
Bhatnagar, M., & Huchhanavar, S. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-53717-2_16 · Cited by 4 · see citing works
Rong, Y., Xu, M., Li, R., Xin, L., & Bao, W. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00075-6 · Cited by 4 · see citing works
Visualization and Automation in Data Science: Exploring the Paradox of Humans-in-the-Loop
Rogers, J., Anastacio, M., Bernard, J., Chakhchoukh, M., Faust, R., Kerren, A., Koch, S., Kotthoff, L., Turkay, Ç., & Wall, E. (2024) · IEEE Visualization in Data Science (VDS) · doi:10.1109/vds63897.2024.00005 · Cited by 4 · see citing works
AssistML: A Concept to Recommend ML Solutions for Predictive Use Cases
Zacarias, A. G. V., Weber, C., Reimann, P., & Mitschang, B. (2021) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa53316.2021.9564168 · Cited by 4 · see citing works
AutoML Insights: Gaining Confidence to Operationalize Predictive Models
Stoica, F., & Stoica, L. F. (2024) · Business, management and economics · doi:10.5772/intechopen.1004861 · Cited by 4 · see citing works
Non-expert AI Users Building Machine Learning Models: A Short Survey of AutoML SystemsPreprint
Palazuelos, C., Duque, R., Tîrnȃucȃ, C., Pérez, A., & Casas, A. (2024) · doi:10.36227/techrxiv.170619255.52099254/v1 · Cited by 4 · see citing works
Data Science collaboration in Human AI: Decision Optimization using Human-centered Automation
Choudari, S., Sanwal, R., Sharma, N., Shāstri, S., Singh, D. A. P., & Deepa, G. (2024) · International Conference on Computational and Characterization Techniques in Engineering & Sciences (IC3TES) · doi:10.1109/ic3tes62412.2024.10877543 · Cited by 3 · see citing works
Case-Study-Based Requirements Analysis of Manufacturing Companies for Auto-ML Solutions
Schuh, G., Stroh, M.-F., & Benning, J. (2022) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-16407-1_6 · Cited by 3 · see citing works
Zhu, Q., Wang, D., Ma, S., Wang, A. Y., Chen, Z., Khurana, U., & Ma, X. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661517 · Cited by 3 · see citing works
Towards Continuous Experiment-Driven MLOps
Rajenthiram, K., Abdullah, M., Gerostathopoulos, I., Hnětynka, P., Bureš, T., Pons, G., Bilalli, B., & Queralt, A. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00018 · Cited by 4 · see citing works
Manousis, A., Shah, H., Milner, H., Li, Y., Zhang, H., & Sekar, V. (2021) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3487552.3487819 · Cited by 3 · see citing works
Faragó, K. B., Skaf, J., Forgács, S., Hevesi, B., & Lörincz, A. (2022) · Applied Sciences · doi:10.3390/app12146927 · Cited by 2 · see citing works
Data Makes Better Data Scientists
Zhao, J., Gal, A., & Krishnan, S. (2023) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/3597465.3605228 · Cited by 2 · see citing works
Eliciting Model Steering Interactions From Users via Data and Visual Design Probes
Crisan, A., Shang, M., & Brochu, E. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3322898 · Cited by 2 · see citing works
Unlocking AutoML: Enhancing Data with Deep Learning Algorithms for Medical Imaging
Jesus, R., Rodrigues, A. M. M., & Costa, C. M. A. (2024) · Journal of Data and Information Quality · doi:10.1145/3705896 · Cited by 2 · see citing works
ExpEngine: A Tool for Data Analytics Workflow Optimization Through User-Driven Experimentation
Rajenthiram, K., Abdullah, M., Gerostathopoulos, I., Hnětynka, P., & Bureš, T. (2024) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) · doi:10.1109/acsos-c63493.2024.00058 · Cited by 2 · see citing works
Automatic Machine Learning Applied to Electrical Biosignals: A Selective Review
Branco, J. A. C., Marques, B. T., Cruz, L. A., Magalhães, R. P., & de Macêdo, J. A. F. (2023) · IEEE International Conference on eScience · doi:10.1109/e-science58273.2023.10254793 · Cited by 2 · see citing works
AutoML as Facilitator of AI Adoption in SMEs: An Analysis of AutoML Use Cases
Polzer, A. K., Zeiringer, J. P., & Thalmann, S. (2023) · Bled eConference · doi:10.18690/um.fov.6.2023.45 · Cited by 2 · see citing works
DroidAutoML: uma Ferramenta de AutoML para o Domínio de Detecção de Malwares Android
Assolin, J., Kreutz, D., Siqueira, G., Rocha, V. D. S., Miers, C. C., Mansilha, R. B., & Feitosa, E. L. (2022) · Simpósio Brasileiro de Cibersegurança (SBSeg) · doi:10.5753/sbseg_estendido.2022.227037 · Cited by 1 · see citing works
AutoML based Stacking Integration for Vehicle Network Anomaly Detection
Du, Y., Wang, P., & Li, Y.-F. (2023) · International Conference on Artificial Intelligence and Automation Control (AIAC) · doi:10.1109/aiac61660.2023.00014 · Cited by 1 · see citing works
Comparison of AutoML frameworks for effective analysis in MAPE-K self adaptive systems
Lecheheb, S., Boulehouache, S., & Brahimi, S. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01083-9 · Cited by 1 · see citing works
From Where to What: The Geo-Intervention Modelling Framework
Wen, R., & Li, S. (2026) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi15070292
Luchs, I. (2026) · Information Communication & Society · doi:10.1080/1369118x.2026.2740039 · Cited by 1 · see citing works
Using Pipeline Performance Prediction to Accelerate AutoML Systems
Zhang, H., López, R., Santos, A. S. R., Ono, J. P., Bessa, A., & Freire, J. (2023) · Workshop on Data Management for End-to-End Machine Learning (DEEM) · doi:10.1145/3595360.3595856 · Cited by 1 · see citing works
Can (and Should) Automated Surrogate Modelling Be Used for Simulation Assistance?
Kurchyna, V., Berndt, J. O., & Timm, I. J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61034-9_1 · Cited by 1 · see citing works
Streamlining the generation of AI tools on a cloud medical imaging platform
Jesus, R., Rodrigues, A., Silva, L. A. B., & Costa, C. M. A. (2024) · IEEE Mediterranean Electrotechnical Conference (MELECON) · doi:10.1109/melecon56669.2024.10608464 · Cited by 1 · see citing works
Kutsepau, A., Kren, А. P., Nikiforov, A. V., & Tursunov, N. K. (2024) · Devices and Methods of Measurements · doi:10.21122/2220-9506-2024-15-3-231-239 · Cited by 1 · see citing works
An autoML algorithm to select suitable decision tree algorithm for classification
Shivappa, N. (2024) · Global Conference on Communications and Information Technologies (GCCIT) · doi:10.1109/gccit63234.2024.10862540 · Cited by 1 · see citing works
Machine learning-based optimal value calculation for welding variables in AR training
Song, C. S., & Nam, J.-H. (2025) · International Journal of Naval Architecture and Ocean Engineering · doi:10.1016/j.ijnaoe.2025.100652 · Cited by 1 · see citing works
LoCoML: A Framework for Real-World ML Inference Pipelines
Maddireddy, K., Methukula, S. K., Sridhar, C., & Vaidhyanathan, K. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00017 · Cited by 1 · see citing works
Ayyıldız, E. (2023) · International Conference on Pioneer and Innovative Studies · doi:10.59287/icpis.812
COMPARACIÓN EMPÍRICA DE DOS MODELOS DE APRENDIZAJE AUTOMÁTICO GENERADOS MEDIANTE PROCESOS DIFERENTES
Gómez, A. R., Calderón-Benavides, L., & Valencia, J. A. P. (2023) · REVISTA COLOMBIANA DE TECNOLOGIAS DE AVANZADA · doi:10.24054/rcta.v1i39.1369
MH-AutoML: Transparência, Interpretabilidade e Desempenho na Detecção de Malware Android
Assolin, J., Canto, G., Kreutz, D., & Feitosa, E. L. (2024) · Simpósio Brasileiro de Cibersegurança (SBSeg) · doi:10.5753/sbseg_estendido.2024.243362
Collecting Performance Prediction for the Rubber Collector in Horizontal Wellbore Based on AutoML
Li, S., Li, Y., Chen, L., Wang, X., & Kong, W. (2025) · Sensors · doi:10.3390/s25061836
Exploring Predictors of Adolescents’ Creative Thinking Ability: using AutoML and SHAP Analysis
Choi, J., & Chung, H. (2025) · Journal of Curriculum and Evaluation · doi:10.29221/jce.2025.28.2.249
POE-ML: An Automated Pipeline for Optimization and Evaluation of Machine Learning
Nielsen, M., & Ayvaz, S. (2025) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c65153.2025.00076
Bhatnagar, M., & Huchhanavar, S. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01340-1 · Cited by 1 · see citing works
Starkova, V., & Howley, I. K. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789153
Evaluating AutoML Frameworks in the Context of Autism Diagnosis: A Comparative Analysis
Karpagam, C., M, P. M. S., & Manivasagam, G. (2025) · IEEE International Conference on Smart Innovations in Systems, Infrastructure, Mechanical, Power, AI and Computing Technologies (SISIMPACT) · doi:10.1109/sisimpact67725.2025.11439368
Cultivating a Culture of Innovation With AI
Eragamreddy, N. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-3510-9.ch007
- Authors
- Xin, D., Wu, E. Y., Lee, D. J.-L., Salehi, N., & Parameswaran, A.
- Affiliations
- University of California, Berkeley (US)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-16
- OpenAlex
- W3160033795
- Cited by
- 110
- References
- 43
- Citations by year
- 2026: 15 · 2025: 19 · 2024: 26 · 2023: 27 · 2022: 16 · 2021: 7
- FWCI
- 8.70
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (IIS-1940759, IIS-1940757)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Keywords
- automated machine learning, machine learning automation, machine learning workflow, reproducibility, reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
