Understanding survey sample size

When you’re asking “how many survey respondents do I need?”, what you’re really asking is, “how big does my sample need to be in order to accurately estimate my population?” These concepts are complex, so we’ve broken the process into 5 steps, allowing you to easily calculate your ideal sample size and ensure accuracy in your survey’s results.

5 steps to make sure your sample accurately estimates your population:

1. What is Your Population?

By population, we mean the entire set of people who you want to understand (your sample is going to be the people from this population who end up actually taking your survey).

So, for example, if you want to understand how many people in your organisation would be interested in a new cycle to work scheme at your London office, your population would be all employees at the London office. If you’re trying to understand how many people in the Marketing department at your Berlin offices would be interested in using the new electric scooter hub outside their office then your population would be all employees in the marketing department at the Berlin offices.

Regardless of whether it’s a single department, the entire office or all employees globally, figuring out what population you’re trying to understand is a vital first step. Once you know what your population is, figure out how many people (roughly) make up that population. For example, roughly 67 million people live in the UK and we’re guessing that your company has fewer employees than that.

Got your number? Okay then let’s keep going…

2. How Accurate Do You Need To Be?

Think of this step as an assessment of how much of a risk you’re willing to take that the answers you get to your survey will be off by a little bit due to the fact that you’re not surveying your entire population. So here are your two questions to answer:

1. How sure do you need to be that the answers reflect the views of your population?
   This is your margin of error. So if, for example, 90% of your sample cycle works every day. A 5% margin of error would add 5% on either side of that number, meaning that actually, 85-95% of your sample cycle work every day. 5% is the most commonly used margin of error, but you may want anywhere from 1-10% for a margin of error depending on your survey. Increasing your margin of error above 10% is not recommended.

2. How sure do you need to be that the sample accurately samples your population?
   This is your confidence level. A confidence level is a likelihood that the sample you picked mattered in the results you got. The calculation is usually done in the following way. If you picked 30 more samples randomly from your population, how often would the results you got in your one sample significantly differ from those other 30 samples? A 95% confidence level means that you would get the same results 95% of the time. 95% is the most commonly used confidence level but you may want a 90% or 99% confidence level depending on your survey. Decreasing your confidence level below 90% is not recommended.

3. How Big a Sample Do I Need?

Using the chart below, select your approximate target population, then choose your margin of error to estimate the number of completed surveys you’ll require, Alternatively check out our calculator here for a more precise result.

Now that you have your numbers from Step #1 and Step #2 check out this handy chart below to figure out how big a sample you’ll need…

Note: These are intended as rough guidelines only. Also, for populations of more than 1 million, you might want to round up slightly to the nearest hundred.

4. How Responsive Will People Be?

We’re sorry to say that not everybody you send your survey to will fill it out.

The percentage of people who do actually fill out a survey that they receive is known as the “response rate”. Estimating your response rate will help you determine the total number of surveys you’ll need to send out to obtain the required number of completed surveys.

Response rates vary widely depending on a number of factors such as the relationship with your target audience, survey length and complexity, incentives, and the topic of your survey. For online surveys in which there is no prior relationship with recipients, a response rate of between 20-and 30% is considered to be highly successful. A response rate of 10-15% is a more conservative and safer guess if you haven’t surveyed your population before.

5. How many invites should I sent out

Just divide the number you got from step #3 by the number you got from Step #4. That’s your magic number.

So, for example, if you need 278 employees to decide which option they like between 2 options and you think about 10% of your employees will actually fill it out, then you need to send it to 278/10% of employees = 2,780!

6. What if the number of invites is more than the number of employees?

If after calculating the number of invites you need to send out you discover that it is more than the number of employees you have, you may then need to reconsider your margin or error, however, this may mean that your data will be less reflective of your population's views. If compromising on the Margin of error is not an option then you could try to motivate your employees to fill in your survey using an incentive.