Sins Committed in the Name of Ecological Validity

Abstract

Do you want conclusions and findings from education research to generalize to real-world education, where students are taught, learn, and are tested? In critiquing education research, are you concerned when it is not ecologically valid? Most of us would answer “yes” to both of these questions. Even if we believe that broad generalizability may not be feasible in any one experiment, conducting “ecologically valid” research is no doubt an ideal to which we still strive. In the present chapter, we agree that generalizability is essential, but we deconstruct ecological validity and argue that the term is often misused and misrepresented in a manner that may ultimately stifle the development of methods for achieving generalizability. For those of you who answered both questions with a resounding “yes,” and the present authors certainly have done so, we must emphasize that our goal is not to undermine the ideal of validity—every social scientist’s desire is to conduct research that is generalizable to a target ecology. Indeed, we mean to hold this ideal with the highest regard, and by doing so, we will reveal the sins committed in the name of ecological validity and the promise of research programs that are informed by representative design.