Research Tips

Collecting Data by Open Invitation

Situations in which the public at large is offered the opportunity to share their opinions or preferences are now very common. For example, guests at hotels or restaurants can fill out comment cards or rate establishments online, Web site viewers can click buttons to participate in polls or surveys related to content on the site, or ads can be placed in traditional or electronic media inviting people to participate in research projects.

Given the prevalence of these open invitations to provide data, it’s important to understand the strengths and limitations of this approach. On the positive side, it provides an easy way for people to share their opinions and may be a relatively quick, easy, and inexpensive way for organizations to access data. On the negative side, it is more prone to self-selection bias than nearly any other form of data collection.

Most market research is subject to some degree of self-selection bias in that there are very few situations in which people can be compelled to participate. However, the problem is worse in the case of open invitations because there are also no controls on who is invited to participate. This makes it even more likely that those who have the most extreme views on the subject or are most motivated by any incentive being offered are disproportionately likely to participate. The views of those people may not be representative of the population of interest, so relying only on data collected through open invitation methods may lead to a distorted view of the true attitudes, opinions, and behaviors of that population.