Does anyone have any data showing the maximum number of questions you should ask in a survey i.e. the number beyond which survey dropout rate goes up drastically? Any idea of dropout rate per question? Are there any other parameters to be aware of when setting up surveys that would majorly affect dropout rate? Advice appreciated!
All of those are bigger factors than the number of questions.
To take an extreme example: Your post here is basically a 3 question survey. If we guess 100 people have looked at it, you now have a 1% response rate. On the other hand, a multiple choice exam is the same format as a survey. It might have a couple of hundred questions & a 99.9% response rate.
The only times I would focus on number of questions would be: 1) if I was optimising an existing survey, 2) if my revenue depended directly on the number of questions I was asking.
How is the survey being delivered and who are the audience? The optimum length really depends on type of survey, eg email to customer database, web exit survey, pre-recruited panellists, etc
We run 100s of online surveys each year through each of these methods and optimal survey length really depends on the nature of the audience and delivery method.
For customer email address surveys we typically ask clients for sample at a ratio of 25:1 and usually achieve target quotas if the survey is sub-10 mins and we have a good, unbiased prize draw. For web exit surveys I would say less than 5 mins is ideal. Panel surveys can be longer but I would say anyone who is willing to spend more than 15 mins completing a survey is typically not that representative.
Hope this helps,
Chris
http://www.spa-mr.com
Suzanne Locke
Head of Digital Development at Emap Middle East
26 April 2009 10:21am
We need to get a certain amount of responses therefore we will use different routes including email to database, adverts on site etc to recruit respondents. We are working with a third party who are keen to ask lots and lots of questions but I do not want to put respondents off with an over-long survey hence initial question. The survey will be incentivised with a relevant and not enormous incentive to, again, attract the right people to respond.
@ Chris Handford - how many questions roughly is sub-10 mins and sub-5 mins, would you say?
We typically work on a conservative 2 questions per minute basis presuming that there will be a range of question types (single-code, multi-code, grid, open text, etc). In practice we can usually achieve circa 25 questions in 10 minutes and 12-15 questions in 5 minutes but this includes demographic and screening questions.The best way to measure this is to get some people not involved in the project to pilot the survey and if its too long then cut some questions.
One thing to note on the incentive is that what some people think is relevant can often limit the survey response, eg A £500 cash prize draw incentive will be a lot more appealing and achieve a higher response than a weekend holiday prize or a new TV. There is also the chance for bias if your incentive will only appeal to certain people or involve giving away your company's own products.
Hope this helps,
Chris
Suzanne Locke
Head of Digital Development at Emap Middle East
27 April 2009 13:02pm
Chris thanks so much, it really does help. Regards, Suzanne
Survey flow is incredibly important. Start with easy, quick questions which help engage your respondent and gets them into the right frame of mind. However, bearing in mind that survey drop offs tend to happen towards the end of the survey, you need to balance this with getting your most important questions in early.
Too many surveys are poorly thought out and structured and are far too long - this leads to high drop offs. Before creating your survey, set out your research objectives and then compare your survey against these. Be ruthless about eliminating unnecessary questions. Don't ask questions you can research using other means (e.g. webstats)
Usability is also important. Test your surveys with several people and watch them fill them out. Ask them for commentary. It's so easy to ignore a potential response (e.g. how many surveys have you completed (or dropped off!) where you want to select n/a but the surveyor has not included this as a possible response)
I am Salman in my final semester of masters and I assigned a project by the college. I need around 50 data samples of this survey after which I’ll analyze the data and prepare my reports.
I am Salman in my final semester of masters and I assigned a project by the college. I need around 50 data samples of this survey after which I’ll analyze the data and prepare my reports.
Head of Digital Development at Emap Middle East
22 April 2009 14:06pm
Does anyone have any data showing the maximum number of questions you should ask in a survey i.e. the number beyond which survey dropout rate goes up drastically? Any idea of dropout rate per question? Are there any other parameters to be aware of when setting up surveys that would majorly affect dropout rate? Advice appreciated!
E-Business Consultant at Dan Barker
23 April 2009 09:47am
hi, Suzanne,
here are some more factors:
All of those are bigger factors than the number of questions.
To take an extreme example: Your post here is basically a 3 question survey. If we guess 100 people have looked at it, you now have a 1% response rate. On the other hand, a multiple choice exam is the same format as a survey. It might have a couple of hundred questions & a 99.9% response rate.
The only times I would focus on number of questions would be: 1) if I was optimising an existing survey, 2) if my revenue depended directly on the number of questions I was asking.
Hope that's some use!
dan
--
http://www.barker.dj
Research Director/Head of Web research at SPA
24 April 2009 09:57am
Hi Suzanne,
How is the survey being delivered and who are the audience? The optimum length really depends on type of survey, eg email to customer database, web exit survey, pre-recruited panellists, etc
We run 100s of online surveys each year through each of these methods and optimal survey length really depends on the nature of the audience and delivery method.
For customer email address surveys we typically ask clients for sample at a ratio of 25:1 and usually achieve target quotas if the survey is sub-10 mins and we have a good, unbiased prize draw. For web exit surveys I would say less than 5 mins is ideal. Panel surveys can be longer but I would say anyone who is willing to spend more than 15 mins completing a survey is typically not that representative.
Hope this helps,
Chris
http://www.spa-mr.com
Head of Digital Development at Emap Middle East
26 April 2009 10:21am
We need to get a certain amount of responses therefore we will use different routes including email to database, adverts on site etc to recruit respondents. We are working with a third party who are keen to ask lots and lots of questions but I do not want to put respondents off with an over-long survey hence initial question. The survey will be incentivised with a relevant and not enormous incentive to, again, attract the right people to respond.
@ Chris Handford - how many questions roughly is sub-10 mins and sub-5 mins, would you say?
Many thanks to you both for responding!
Research Director/Head of Web research at SPA
27 April 2009 10:02am
Hi Suzanne,
We typically work on a conservative 2 questions per minute basis presuming that there will be a range of question types (single-code, multi-code, grid, open text, etc). In practice we can usually achieve circa 25 questions in 10 minutes and 12-15 questions in 5 minutes but this includes demographic and screening questions.The best way to measure this is to get some people not involved in the project to pilot the survey and if its too long then cut some questions.
One thing to note on the incentive is that what some people think is relevant can often limit the survey response, eg A £500 cash prize draw incentive will be a lot more appealing and achieve a higher response than a weekend holiday prize or a new TV. There is also the chance for bias if your incentive will only appeal to certain people or involve giving away your company's own products.
Hope this helps,
Chris
Head of Digital Development at Emap Middle East
27 April 2009 13:02pm
Chris thanks so much, it really does help. Regards, Suzanne
Managing Director at Content Formula
28 April 2009 16:29pm
Survey flow is incredibly important. Start with easy, quick questions which help engage your respondent and gets them into the right frame of mind. However, bearing in mind that survey drop offs tend to happen towards the end of the survey, you need to balance this with getting your most important questions in early.
Too many surveys are poorly thought out and structured and are far too long - this leads to high drop offs. Before creating your survey, set out your research objectives and then compare your survey against these. Be ruthless about eliminating unnecessary questions. Don't ask questions you can research using other means (e.g. webstats)
Usability is also important. Test your surveys with several people and watch them fill them out. Ask them for commentary. It's so easy to ignore a potential response (e.g. how many surveys have you completed (or dropped off!) where you want to select n/a but the surveyor has not included this as a possible response)
Head of Digital Development at Emap Middle East
29 April 2009 13:17pm
Great stuff, cheers Dan - Suzanne
student at university
30 April 2009 17:52pm
I am Salman in my final semester of masters and I assigned a project by the college. I need around 50 data samples of this survey after which I’ll analyze the data and prepare my reports.
If you are working in Software Development, Web Development, Testing or similar field, please I would appreciate, if you would take 5 mins of your time to give feedback.
http://www.kwiksurveys.com/online-survey.php?surveyID=HNIJK_41b94243
PLEASE FORWARD IT TO YOUR FRIENDS/COLLEAGUES WHO ARE ELIGIBLE TO TAKE THIS UP.
Thanks a lot for your help
Regards,
Salman
student at university
30 April 2009 17:52pm
I am Salman in my final semester of masters and I assigned a project by the college. I need around 50 data samples of this survey after which I’ll analyze the data and prepare my reports.
If you are working in Software Development, Web Development, Testing or similar field, please I would appreciate, if you would take 5 mins of your time to give feedback.
http://www.kwiksurveys.com/online-survey.php?surveyID=HNIJK_41b94243
PLEASE FORWARD IT TO YOUR FRIENDS/COLLEAGUES WHO ARE ELIGIBLE TO TAKE THIS UP.
Thanks a lot for your help
Regards,
Salman