A Large‐Scale Evaluation of an Intelligent Discovery World: Smithtown

Abstract

Smithtown is an intelligent tutoring system designed to enhance an individual's scientific inquiry skills as well as to provide an environment for learning principles of basic microeconomics. It was hypothesized that computer instruction on applying effective interrogative skills (e.g., changing one variable at a time while holding all else constant) would ultimately lead to the acquisition of the specific subject matter. This paper presents an evaluation of Smithtown in two studies of individual differences in learning. Experiment 1, an exploratory study, demonstrated that Smithtown fared very well when compared to traditional instruction on economics and delineated the performance indicators which separated better from worse learners in this discovery environment. Experiment 2 extended the findings from the exploratory study using a large sample of subjects (N = 530) from a different population interacting with Smithtown and showed that the performance indicators relating to hypothesis generation and testing were the most predictive of successful learning in Smithtown, accounting for considerably more of the variance in our learning criterion than a measure of general intelligence. Overall, the system performed as expected. Tutoring on scientific inquiry skills resulted in increased knowledge of microeconomics. The differentiating behaviors between more and less successful subjects were in agreement with specific behaviors relating to individual differences found in general studies on problem solving and concept formation. From an instructional perspective, the behaviors we have denoted can serve as a focal point for relevant intervention studies. From a design perspective, findings from these studies suggest modifications to intelligent tutoring systems so they may be more like the individualized teaching systems they have the potential to be.