Research Tips
Repeated Samples
Many people use or hear terms such as margin of error or statistical significance without realizing that they are based on the notion of repeated samples. While many surveys and other forms of research involve a single round of data collection, the statistics derived from them are based on the likelihood of getting the results that were found using repeated samples given some true, known value for the parameters being estimated. Of course, in real situations, the research is generally being done because the true values are not known, but it’s still worth considering the benefits of repeated samples.
Most people know that increasing sample size helps to reduce the chances of accepting an erroneous result, though many don’t understand that the extent of improvement diminishes as the sample size increases. In many cases, rather than increasing the size of a single sample, it may pay to use a series of repeated samples. First, use of such samples makes acceptance of erroneous results even less likely (all other things being equal) because the same erroneous result would need to occur in each sample, and the more samples you have the lower the probability of that. Second, if samples are repeated at regular intervals (monthly, quarterly, yearly, etc.), then it makes it possible to identify trends over time.