Imagine That You Have Been Hired as a Consultant for a Firm That Plans to Develop Surveys That Are Presented on the Internet, on Websites. They Want to Obtain Accurate Information About People’s Beliefs and Attitudes from These Surveys. Based on Psychological Research on Surveys, What Advice Would You Give to This Firm? Support your Recommendations with Psychological Evidence.
Creating a survey is a much more complicated procedure than many people believe, and there is much to take into account when aiming to accurately collect information on issues such as beliefs and attitudes. Moreover, there has recently been a dramatic move by much of the psychological community towards using the internet as an effective means of such data collection (as well as an ‘online laboratory’ for conducting experiments). What many of these psychologists have found is that, although this new means of conducting research has many advantages, it also has a number of potential problems that must be taken into consideration in order for the work to be valid and effective. A company looking to take advantage of this new medium must take all of these things into consideration, maximising the benefits, avoiding the problems and ensuring that the survey remains correctly constructed and administered.
The first aspect to consider is what sample to use. Depending on the firm, and the reasons for the information gathering (as well as the particular attitudes being targeted), there are a number of very different options available for enlisting participants. The Internet offers the opportunity to enlist subjects that are demographically and culturally diverse on a scale previously unthinkable. Traditionally, samples used in such surveys have suffered the problem of being unrepresentative, and therefore it has been difficult to suggest that the results are generalisable. In particular, it has often been the case that researchers have used university students (particularly Psychology students) who take part for course credit or due to some incentive such as economic compensation (Epstein et al, 2001).
The result is that the participants show no great diversity on education, social status, age, and other variables. However, despite the potential for great diversity in these and other dimensions, a similar criticism has been levelled by some at the Internet. One recent estimation in the US suggested that up to 65% of Internet users are white or Asian, male, are well educated, have an average income of around $70,000 (US) and are between 45-55 years old (Best et al, 2001). Despite this claim, many have pointed out that internet use is growing daily and across all demographic groups. Moreover, this growth is so fast that it is impossible to guess with any degree of accuracy the current demographic constituency of users (Buchanan & Smith, 1999).
Therefore what is vitally important is that the firm prefix the survey with a demographic information form that is compulsory. By doing so, the company will be able to consider this information in their statistical result analysis, and therefore avoid the potential problem of confounding variables. Moreover, they will be able to separate and compare the results on factors such as age groups, ethnicity or gender, and even discount any data that they deem irrelevant to their research. Such an identification sheet would also provide the ability to disallow multiple submissions.
If the firm is looking for particular attitudes and beliefs which are connected to a service or product they offer, it is probably more advantageous for them to search out a specific group that will want to use that service or product. For example, it would be of little use for a company looking into services for women suffering menopause to have the attitudes of teenage males (or in fact anyone other than women suffering or worrying about suffering the condition) (Conboy et al, 2001). Such responses would almost certainly serve to confound the results and subsequent analysis. Simply putting the survey on their web site should serve this purpose to some extent, as it will only be accessed by people who have a vested interest in the topic matter. This may, however, prove to be too limiting.
There are then two options for soliciting responses, active and passive advertisement (Bailey et al, 2000). Active advertisement would involve posting requests on ‘newsgroups’ on the Internet which serve as discussion forums for specific issues. By targeting specific newsgroups that are associated with the firm’s interests (such as those on menopause, or women’s groups) they can ensure that all those involved would be informed and interested in the survey subject. Passive advertisement would involve indexing the questionnaire under search engines so that when specific words are typed into the search engine (such as menopause, or women’s problems) the survey will be presented for the individual to access. Moreover this technique can be adapted by using more or less specific words in the search string (menopause vs. woman) to alter specificity of interests by the participant. What should be noted is that such a technique may be more open to self-selection biases (Birnbaum, 2000).
The next thing to consider are the questions used within the study. Even before all the considerations regarding the presentation of the survey on the Internet, the structure of a questionnaire can be vitally important. It is necessary to decide what attitudes and beliefs are being measured, and how the existence and subsequently the strength and nature of these factors can be best explored. For this last point in particular, it is necessary to assess when and how often to use open and closed questions (Gillham, 2000).
The closed style employs statements or questions to which the response style is simply to circle one of a number of possibilities (eg Yes/No, or All The time/ Most of The Time/ Sometimes / Never). These types of question have a number of obvious advantages both in general and on the Web. Firstly they are very easy to statistically analyse, as each response can be given a value, and then each grouping (such as for a particular attitude being measured) can be easily amalgamated to give a total score for that section.
This is of particular benefit when the survey is presented on the Internet, as the data collection and analysis can be fully automated as part of the server software package. It even allows for instant feedback to the respondent, which may be an appeal for them to take part in the first place. These questions are also very effective at measuring attitude and belief strengths, as long as the presentation is counterbalanced to control for acquiescence effect (Gillham, 2000). A drawback with the closed question is that it can be very difficult, if not impossible to tell if a question has been misinterpreted, and so answered inaccurately. This is even more of a potential problem when it is considered that there may be a great number of Internet respondents who do not have English as their native language (therefore another important question for the demographic form).