Research Tips
Diminishing Returns to Increasing Sample Sizes
As discussed in previous tips, determining the sample size that’s appropriate for a given project is more complex than many people imagine. People often over-estimate the benefits of increasing sample sizes above a minimum threshold. This is because the gain in precision associated with increasing the sample size diminishes as the sample size grows. For example, the following graph shows how the maximum margin of error associated with the estimate of a proportion (based on a population of 5,000,000 and a 95% confidence level) changes based on the sample size1:
The exact numbers change, but a similar pattern holds for estimating different types of statistics, and with different population sizes, confidence levels, and sampling methods. Increasing sample sizes almost always increases the cost of a project, so it only makes sense to use very large sample sizes when high precision is extremely important (e.g., ensuring the dose of a powerful drug is correct).
1 Calculations made using an online application developed by: Lenth, R. V. (2006), “Java Applets for Power and Sample Size” [computer software], retrieved April 28, 2007, from http://www.stat.uiowa.edu/~rlenth/Power.