When to Use Web-based Surveys

Abstract

Even in this randomized trial era, surveys of patients and professionals remain an important epidemiologic technique for capturing cross-sectional or longitudinal data, providing fundamental insights about health and disease.1 There are several alternative methods for collecting such data, which include conducting face-to-face or telephone interviews; circulating a questionnaire by mail, fax, or e-mail; and eliciting responses to a survey posted on an open Web site. Other methods that are currently less widely exploited include digital interactive television; use of a software package to capture survey data, either by mailed floppy disks or as a Java applet over the Web; and automatic telephone menu systems. The preceding article by Schleyer and Forrest2 explores some of the issues arising from a Web survey, but it may be useful first to consider the range of survey methods that are available. Two major features define a survey method: how participants are identified and the data capture technique. Let us consider these features and their implications in turn. How participants are identified determines how much control the investigators exert over their selection. For complete control, eligible participants are selected according to specific characteristics.3 We can check characteristics from an existing database or mailing list, including an e-mail discussion list, or carry out a preliminary mailing or interview to capture the necessary eligibility data. If, instead, we wish only a modest degree of control, we can distribute copies of the survey to a group known to us and ask them in turn to send it out to their contacts—the “snowball” technique (e.g., Forsstrom et al.4). With this method we have much less control over who receives a copy of the survey, and it makes sending reminders or follow-up surveys more difficult, too. We often do not even know how many copies of the survey have been sent out, making it impossible to calculate a response rate. Thus, generalizing results from a snowball survey is difficult. Finally, there are occasions when we want to encourage anyone and everyone to complete our survey, with no control over the type of participants. We can distribute the survey on the street, in a newspaper, or by random mail shots or we can e-mail it to random members of a general mailing list or place it on a public-access Web site. Such techniques mean that follow-up and reminders are usually impossible and response rates hard to calculate. Schleyer and Forrest used the Web but still retained control over who could respond by issuing passwords by e-mail, allowing selected participants to gain one-off access to a closed Web site. Passwords can also be issued by mail or telephone. The second key feature is how the data are captured. Many conventional paper questionnaires are designed to capture free-form text, which is usually transcribed. Face-to-face or telephone interviews can be tape-recorded and transcribed, but this is a lengthy process. Such qualitative data can then be analyzed using appropriate software tools such as nud*ist or martin to extract recurrent themes.5 Data analysis is easier, faster, and more accurate if data can be captured electronically at source, for example, using specially designed forms and optical character or optical mark recognition, which are increasingly used in clinical trials.6 Often the special OCR/OMR forms are faxed to a data center, speeding return of data.7 Other techniques for capturing electronic data include telephone voice menu systems, e-mailed text, Web forms, and mailed interactive software packages. As well as speeding data analysis and reducing transcription errors8 the choice of data capture method also determines how much control the researchers have over the order of question completion and the time to be allowed. For example, with a paper or e-mailed questionnaire, there is little, if any, control—the participant can leaf back and forth before answering the first question. Even standard HTML forms allow little control over question order or the time allotted, as devious participants can work through the survey answering obligatory questions randomly, then go back to alter their responses. This makes it hard to conduct experiments such as eliciting a diagnosis on a simulated case before and after receiving the output of a decision support system.9 On the other hand, face-to-face or telephone interviews, interactive software, and Java applets delivered via the Web allow complete control over the time participants are allowed and the ordering of questions. The combination of these two features determines three further significant issues: The type of data that can be collected: responses only (postal, e-mail, Web—because of variable delays), responses and time taken to respond (automatic phone system, software, recorded interview), responses and observer comments (face-to-face encounters like a conventional job interview) The overall survey cost: hig