Improving Designer Learning in Design Space Exploration by Adapting to the Designer’s Learning Goals

Abstract

Design space exploration is a design method by which the designer tries to learn important information about a design problem (e.g., main design trade-offs, sensitivities, common features among good designs) to help them make better design decisions. This paper presents preliminary results of a study characterizing the effects on a designer’s learning of an AI assistant that adapts to the designer’s goals during design space exploration. Specifically, we compare the designer’s learning when the AI assistant adapts to explicit learning goals shared by the designer versus when it does not adapt. The AI assistant used for the study is Daphne, which helps engineers design Earth observation satellite systems. The designer’s learning process is modeled as an iterative hypothesis generation and testing process. First, the designer shares with Daphne a certain learning goal in the form of a hypothesis (e.g., designs with feature F are more likely to be on the Pareto front). Then, Daphne adapts to this goal by searching for more designs that have the feature being tested and showing the user the extent to which the data supports their hypothesis. The participants in the preliminary study are N = 10 students from Texas A&M University. We ask each participant to design earth observation satellite constellations to meet a set of requirements while trying to learn about the design problem. The results show that participants with the adaptive AI assistant consistently score higher on their learning about the design task compared to the baseline design assistant as measured by a post-task test. A negative effect is observed on task performance with the adaptive AI assistant condition due to a smaller number of design creation actions, which is consistent with findings from previous studies. Recommendations are provided for the design of similar future AI assistants based on the results of this study. Finally, a power study is done to set a goal for statistical validity of the study.