DOI 10.1007/s11135-005-8081-8 Quality & Quantity (2006) 40:435–456 © Springer 2006 Conducting Online Surveys MARTINE VAN SELM 1, and NICHOLAS W. JANKOWSKI 2 1 Department of Social Science Research Methodology, Faculty of Social Sciences, Radboud University Nijmegen, 6500 HE Nijmegen, The Netherlands; 2 Department of Communication, Radboud University Nijmegen, The Netherlands Abstract. The World Wide Web (WWW) is increasingly being used as a tool a
of 22
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  DOI 10.1007/s11135-005-8081-8Quality & Quantity (2006) 40:435–456 © Springer 2006 Conducting Online Surveys MARTINE VAN SELM 1 , and NICHOLAS W. JANKOWSKI 2 1 Department of Social Science Research Methodology, Faculty of Social Sciences, Radboud University Nijmegen, 6500 HE Nijmegen, The Netherlands;  2 Department of Communication,Radboud University Nijmegen, The Netherlands Abstract.  The World Wide Web (WWW) is increasingly being used as a tool and platformfor survey research. Two types of electronic or online surveys available for data collection arethe email and Web based survey, and they constitute the focus of this paper. We address amultitude of issues researchers should consider before and during the use of this method of data collection: advantages and liabilities with this form of survey research, sampling prob-lems, questionnaire design considerations, suggestions in approaching potential respondents,response rates and aspects of data processing. Where relevant, the methodological issuesinvolved are illustrated with examples from our own research practice. This methods reviewshows that most challenges are resolved when taking into account the principles that guidethe conduct of conventional surveys. Key words:  online survey, Web based survey, email survey, research methods review,sampling procedures, response rate 1. Introduction The Internet is increasingly used as a tool for and object of social scientificstudy. This may confront social scientists with the question whether or notnew social science research methods need to be developed in order to enterthe ‘Internet arena’. With respect to this issue at least three positions canbe distinguished, ranging from the idea that Internet research is ‘businessas usual’ to ‘the urgent need for entirely new methods’. We employ a third,middle-position in which, on the one hand, the usefulness of old ways of conduct are emphasized, and on the other hand, the development of prin-ciples and tools in order to deal with new challenges posed to researchersconducting Internet research. This research methods review is about oneparticular method increasingly employed in Internet research: online sur-veys. We discuss the advantages and liabilities with this form of surveyresearch.  Author for correspondence: Martine van Selm, Department of Social Science ResearchMethodology, Faculty of Social Sciences, Radboud University Nijmegen, P.O. Box 9104,6500 HE Nijmegen, The Netherlands. Tel.: ++31-243611709; Fax: ++31-243612351;  436  MARTINE VAN SELM AND NICHOLAS W. JANKOWSKI Frequent Internet surfers may encounter online surveys almost daily andbe invited to visit specific sites where the questionnaires are located. Arecently conducted search for Web based surveys 1 with a popular searchengine resulted in a large amount of hits. The search produced many pageswith information on software for the construction of ‘do it yourself’ Webquestionnaires. In addition, a rough categorization of the first 100 pagessuggests that online surveys are used for a wide range of topics, rang-ing from studies conducted among (medical) professionals, consumers, stu-dents, Internet users, job seekers, employees and more, and that (academic)discussion is prevalent about the ‘ins and outs’ of online surveys.Schaefer and Dillman (1998) emphasize that the proportion of house-holds with access to the Internet is too small to conduct general public sur-veys by email or via the WWW. Although the number of individuals andhouseholds with Internet access is increasing, the present penetration of thetechnology does not permit its use for such surveys (see, Dillman et al.,1998a; Van Dijk, 2000; Crawford, et al., 2001; Miller et al., 2002). Never-theless, for some groups, such as company employees, members of profes-sional associations, or college students, email access has reached nearly alland these groups can relatively easily be surveyed by email (Schaefer andDillman, 1998: 378).In this paper we address a multitude of issues researchers have to con-sider before and during the use of this method of data collection. First, weconsider sampling issues related to these types of surveys. Here, the spe-cific characteristics of sampling from an electronic sampling frame are pre-sented. Further, attention is paid to the construction of an instrument foran online survey. Various ways of launching a questionnaire using email orthe WWW are discussed as well as issues related to response rate, speed of return, and the quality of responses. The reminder of the paper is devotedto considerations of data processing and data cleaning, and to discussionof supportive software useful for performing tasks associated with surveyresearch. Where relevant, the methodological issues involved are illustratedwith examples from our own research practice. 2. Reasons for Conducting Online Surveys When do researchers conduct online surveys and what are their experienceswith this research method? This question is frequently addressed in jour-nals and academic forums. 2 One of the reasons for conducting online sur-veys has to do with the object of study. Online surveys are often employedin studies of Internet use in order to reach a population with Internet expe-rience. An example of the latter is the well-known Web survey of GeorgiaTech’s Graphics Visualization and Usability Center (GVU), the purpose of which is to monitor changes in Internet user demographics and attitudes  CONDUCTING ONLINE SURVEYS  437towards technology and online commercial activity (Pitkow and Recker,1995; Kehoe and Pitkow, 1996). The Nielson/NetRatings is another exam-ple of this kind of use. Inasmuch as the proportion of Internet users amongmost general populations is relatively small, securing a random sample byconsulting postal addresses or telephone numbers would generate a largenumber of respondents who do not have access to the Internet (Kay andJohnson, 1999).In addition to the object of study, particular characteristics of the pop-ulation may also be a reason for conducting an online survey. For therecruitment of potential respondents with special interests, consultation of specific Internet environments can be valuable such as Usenet discussionforums, newsgroups, bulletin boards, chat rooms, and MUDs. 3 VariousInternet environments serve as a meeting place for persons with similarhobbies, experiences, interests, and lifestyles. Also, support groups are fre-quently established in which personal experiences are shared (Mickelson,1997). These self-organized groups can be valuable for researchers inter-ested in designing a study of persons with particular characteristics. Theseselected groups enable, in other words, the conduct of “expert interroga-tions” (Swoboda et al., 1997: 243).A third reason for conducting an online survey is to facilitate recruit-ment of respondents with deviant or covert behaviors. The anonymity pos-sible on the Internet is believed to help in gaining access to respondentsnormally difficult to reach, and it may facilitate the sharing of their expe-riences and opinions. Therefore, online surveys are considered useful whenthe issues being researched are particularly sensitive (see Coomber, 1997).A fourth point has to do with the attractiveness of computers to par-ticular (age) groups. Using the Internet for survey purposes among youngpeople may lead to higher response rates as compared to paper-and-pencilsurveys. Beebe et al. (1997), in a discussion of design and developmentissues of computerized school surveys, suggest that the computer mayhave special relevance for younger groups. Use of the Internet in, forinstance, school-based surveys is expected to improve response rate fromsuch respondents.There are other reasons for employing the Internet as a tool in sur-vey research, such as economic advantage (Internet surveys are inexpensiveas related to conventional paper-and-pencil surveys) and efficiency (datacollection via the Internet is fast), reach, and simply gaining experiencein using the Internet for survey purposes. With respect to  cost  reduction,Mann and Stewart (2000) warn that although computer-mediated commu-nication (CMC) can cut production costs, the start-up expenses involved inWeb based surveys, particularly expenses incurred to secure the necessaryexpertise for designing instruments, can be quite substantial.  438  MARTINE VAN SELM AND NICHOLAS W. JANKOWSKI A similar point is made regarding the  time benefits  offered by CMC usefor survey purposes. Although Web based surveys can accelerate the rate of response, researchers often end up spending considerable time solving tech-nical problems before and during implementation of an online survey. Time-consuming activities related to implementation of email surveys include:searching for email addresses, checking and replacing invalid email addresses,and explaining the form and procedures to less technically savvy respondents. Reach  refers to the ease by which potential respondents can beapproached. Online surveys are particularly attractive in this respect whenthe population under study is distributed across a large geographic region.Mann and Stewart (2000), however, remind us about the risk of loosingsight of who is responding to the questionnaire. The persons to whoman email survey is forwarded may not have the characteristics expected of respondents because of geographical (and cultural) distance. Also, accessto the Internet in some workplaces and countries is limited or restricted.And, there is a strong bias against (survey) researchers in some computer-oriented subcultures. In the words of Mann and Stewart (2000: 73), “Astranger wanting to do academic research is sometimes seen as an unwel-come arbitrary intrusion.”This observation should temper the enthusiasm of researchers who feel thatreaching Internet users is merely a matter of applying the technology. Withrespect to possible invasion of privacy, it seems that the identity of respondentsinemailsurveysislessprotectedthaninWebbasedsurveys.Whenarespondentusesthe‘reply’functionofanemailprogramtoreturnthecompletedsurvey,themessage carries at least the email address of the sender in the message header.Respondents can takes measures in order to guarantee anonymity by sendingmessages through, for instance, a low profile remailer service. A remailer stripsmessagesoftheiroriginalheadersandreplacesthemwithotherinformation.Inthis way the sender of a message becomes more difficult to trace. This increasein anonymity on behalf of respondents means a loss of control over the qualityof data, however, as the remailer camouflages multiple responses from a singleperson. Web based surveys, in contrast, do allow for anonymity in as much asrespondents are free to withhold their names. However, multiple responses arealsopossibleinthisformofonlinesurveyfromasingleperson.Laterweelabo-rate on possible interventions from a researcher concerned with controlling formultiple responses.Other advantages to online surveys mentioned in the literature (Metha andSivadas, 1995; Smith, 1997; Medlin et al., 1999; Brennan et al., 1999) include: ã  absence of interviewer bias; ã  removal of the need for data entry in as much as respondents directlyenter data into an electronic file; ã  convenience for respondents.
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!