The Modified Delphi Technique - A Rotational Modification

Abstract

This study describes and illustrates a modification of the Delphi technique that was designed to extend its applicability to large competency data sets. The approach consisted of rotating sub-sets of a larger set of competencies through three sub-panels and then examining the viability of the approach. The context of the study was the identification of competencies needed to update training requirements for 21st century plastering contractors. Based on the outcomes of this study, the rotational approach represents a promising mechanism for extending the usefulness of the Delphi tool to larger data sets. The findings also indicate that the factors that erode the validity and usefulness of traditional modified Delphi studies also negatively affect the rotational approach. An obstacle facing many educators has been identifying an appropriate research methodology for developing curriculum when teaching vocational subjects. This is especially true when a research topic is very broad and when new curricular areas are being developed. In such situations there is often a large number of competencies to be evaluated by recognized experts in the content area. A number of techniques have been used that are generally associated with the competency-based approach to curriculum development. These include such tools as occupational analysis, DACUM (Developing a Curriculum), and the Delphi technique ( Finch & Crunkilton, 1989 ; Frykland, 1970 ; Miller, 1990 ; Rothwell & Kazanas, 1992 ). These techniques span a range of approaches from observing workers on the job to conducting meetings with workers to conducting analyses based on work previously conducted on related occupations ( Blank, 1982 ). Each technique has advantages and disadvantages, especially in dealing with diverse groups of experts involving relatively large data sets. This manuscript describes a modification developed for use with the Delphi technique, which was selected for use in a curriculum competency study. The Delphi technique was selected for use in this study due to its ability to obtain expert input from individuals who were widely dispersed geographically. This technique has been used in a number of fields for long-range planning - including education, international affairs, transportation, leisure activities and the like ( McCampbell & Stewart, 1992 ). The Delphi survey technique was developed in the 1950s by two research scientists working at The Rand Corporation, Olaf Helmer and Norman Dalkey. They developed the procedure as a tool for forecasting future events using a series of intensive questionnaires interspersed with controlled-opinion feedback ( Dalkey & Helmer,1993 ; McCampbell & Stewart, 1992 ; Weaver, 1971 ). Participants were solicited experts in the issues related to national defense such as forecasting probable bombing targets the Russian government might choose in the event of an attack on the United States ( Dalkey & Helmer, 1963 ). The Delphi begins with an open-ended questionnaire that is given to a panel of selected experts to solicit specific information about a subject or content area. In subsequent rounds of the procedure, participants rate the relative importance of individual items and also make changes to the phrasing or substance of the items. Through a series of rounds (typically three) the process is designed to yield consensus. The modified Delphi technique is similar to the full Delphi in terms of procedure (i.e., a series of rounds with selected experts) and intent (i.e., to predict future events and to arrive at consensus). The major modification consists of beginning the process with a set of carefully selected items. These pre-selected items may be drawn from various sources including related competency profiles, synthesized reviews of the literature, and interviews with selected content experts. The primary advantages of this modification to the Delphi is that it (a) typically improves the initial round response rate, and (b) provides a solid grounding in previously developed work. Additional advantages related to the use of the modified Delphi technique include reducing the effects of bias due to group interaction, assuring anonymity, and providing controlled feedback to participants ( Dalkey 1972a, 1972b, & Judd 1972 ). Brooks (1979) noted that three mailings are usually sufficient in order to arrive at consensus.