In theory, online surveys are actually pretty simple, or at least they are when you know what you're doing! This article aims to provide a guide to help you speed up your learning curve and start doing good online surveys from the very beginning.
The biggest problem with all online surveys is often the sample. There is no such thing as an online directory of email names (or at least one that is comprehensive and you can use for commercial purposes). Moreover there probably never will be such a directories and so random sampling is out. The solutions to this problem is as follows:
1. If you doing a website survey, just intercept people as they leave your website using a "pop-up" survey.
2. If you want to do more general research (or conventional research online) use one of the panels of Internet users. In the US there are plenty of these and they are a number with literally millions of people on them. However in Europe, these panels are still being developed, so you may need to think about creating your own!
3. If you want to interview customers or employees, you are generally OK, as you should have their email addresses. However in the case of customers, take care that you have their permission to survey them and that you have a mechanism to ensure email addresses are up to date.
Having sorted out your sample, you need to work out how you are going to create your online survey. You can just create online surveys as simple web pages in "html", but you will have great problems extracting the data and ever running anything that has CATI-like functionality. Therefore you need to get some software.
All traditional MR software vendors now have web versions of their products (eg SPSS, Pulse Train, Voxco etc.). As a general rule though, many of these lack something and are merely the CATi interface re-written for the web. However currently, the best products for online surveys are often those create specifically for the web (eg Confirm, GMI, etc). They have a number of key benefits over conventional MR software options (although some conventional software now offer some of these too.). Firstly, all come with online reporting allowing you (and your clients, if you wish) to view results as they happen. Many of them will even generate an automatic PowerPoint presentation of them for you too. Most also allow you to run the survey in many languages simultaneously. The other key advantage is that the surveys themselves look so much better. You usually have full control over the web page design template and also the way the questions appear (GMI is particularly good at this.) The other benefit of some of them is that they can be cheap, if not free (visit www.zoomerang.com for probably the best of this type.)
Online surveys can get good response rates, but they can also get very low ones! A lot of this is down to executional factors and you must take great care over these. Here are what we feel are the 10 most important factors to bear in mind:
1. Take care with Pop-up surveys designs. When doing this type of interview, we regularly get response rates of 30%. However we have had ones at 9% and also at 59%! We have found that, apart from relationship with the brand/site, the most important factor affecting response rate is survey style. Do not start surveys with a box giving people the option to either "take the survey or not". What you should do is to launch directly into questions. However ensure that the first page of your survey looks short and does not have a scroll bar.
2. Make navigation in online surveys simple. Most online survey software comes with options to have buttons to progress forward, back, stop or event delete your data (often called "reset"). As a general rule, turn off these options and ensure that you only ever give people the arrows to go forward (and back, if appropriate).
3. Use a progress bar. Again most software will allow you to insert a progress bar. It is very important that you use it so that people know here they are in your survey as there is no interviewer to tell them. Note, if you don't have a progress bar option, you can always just write ("Page 2 of 6" etc). Indeed, I actually think this can sometimes be better than an automatic progress bar, as the algorithms for these are sometimes lead to pretty misleading bars!
4. Include branding. Always provide reassurance of who the client is or your branding on surveys. It helps completion rates. Indeed, if you are doing website surveys, make your survey look like the site generally.
5. Keep online surveys short. If you're doing a pop-up visitor survey, you have a maximum of just 4-5 minutes of someone's time. If doing a survey amongst your customers, you probably have 5-10 minutes and panel members 10-15 minutes. However note there are no hard and fast rules here except the longer the survey, the bigger the dropout!
6. Avoid grids. Work has been done to see when people abandon online surveys and grids are the #1 location. This is partly because in most MR software type online interviews, the interface looks terrible and is very difficult to complete. By using some of the new online software, you can generally make them look better or even change them to dropdowns. However as a rule, make sure you cut down your image attributes to a minimum.
7. Check your survey on different browsers/screens. The web browser was supposed to be a platform independent. It isn't! Surveys can look quite different in Netscape and Internet Explorer. More importantly the users individual screen settings can have a drastic effect on your survey. The basic rule of thumb here is to design for a 640x480 screen with large fonts and then look at it in both Netscape 4 and Internet Explorer 4 & 5.
8. Take care with incentives. These are necessary for online panel surveys. People have been using a variety of incentives from conventional loyalty points (eg Air Miles), electronic money (like Beenz), real money and prize draws. Make sure you use a system that you can easily fulfill! Also take care with incentives on website surveys. We have shown that they have comparatively little effect on response rate and can seriously affect data quality by: encouraging multiple submissions, rubbish data or even people to flock to the website to "fill out the competition", ie your nice genuine visitor survey.
9. Have norms to interpret online data. Like all data, online data has its own baises. Some would argue that it can result in more accurate data, as there is no interviewer effect. Also you tend to get better quality opens. However most agree that there is an effect and so if you are converting a conventional survey online (eg a tracking study), take care to validate it and know what the differences are. For website surveys, we have developed our own norms. To illustrate how important they are, it may come of some surprise that even at the worst website we have ever researched, we still found over the half users saying they'd use it again!
10. Ensure you abide by Code of Conduct. ESOMAR has a clear Internet code of conduct - visit www.esomar.nl. In essence it makes sure that you apply the current code to online surveys too. Some key factors are therefore transparency, ie you must tell people it is a survey and provide them ways to contact/verify the researcher is bone fide. You must alert people to the fact that survey will cost them money to complete and tell them if you're using cookies. You must provide a link to your privacy policy. Finally take great care not to interview minors (under 14/15) without prior parental consent. Therefore pop-up surveys with kids are against code of conduct!
The biggest problem with all online surveys is often the sample. There is no such thing as an online directory of email names (or at least one that is comprehensive and you can use for commercial purposes). Moreover there probably never will be such a directories and so random sampling is out. The solutions to this problem is as follows:
1. If you doing a website survey, just intercept people as they leave your website using a "pop-up" survey.
2. If you want to do more general research (or conventional research online) use one of the panels of Internet users. In the US there are plenty of these and they are a number with literally millions of people on them. However in Europe, these panels are still being developed, so you may need to think about creating your own!
3. If you want to interview customers or employees, you are generally OK, as you should have their email addresses. However in the case of customers, take care that you have their permission to survey them and that you have a mechanism to ensure email addresses are up to date.
Having sorted out your sample, you need to work out how you are going to create your online survey. You can just create online surveys as simple web pages in "html", but you will have great problems extracting the data and ever running anything that has CATI-like functionality. Therefore you need to get some software.
All traditional MR software vendors now have web versions of their products (eg SPSS, Pulse Train, Voxco etc.). As a general rule though, many of these lack something and are merely the CATi interface re-written for the web. However currently, the best products for online surveys are often those create specifically for the web (eg Confirm, GMI, etc). They have a number of key benefits over conventional MR software options (although some conventional software now offer some of these too.). Firstly, all come with online reporting allowing you (and your clients, if you wish) to view results as they happen. Many of them will even generate an automatic PowerPoint presentation of them for you too. Most also allow you to run the survey in many languages simultaneously. The other key advantage is that the surveys themselves look so much better. You usually have full control over the web page design template and also the way the questions appear (GMI is particularly good at this.) The other benefit of some of them is that they can be cheap, if not free (visit www.zoomerang.com for probably the best of this type.)
Online surveys can get good response rates, but they can also get very low ones! A lot of this is down to executional factors and you must take great care over these. Here are what we feel are the 10 most important factors to bear in mind:
1. Take care with Pop-up surveys designs. When doing this type of interview, we regularly get response rates of 30%. However we have had ones at 9% and also at 59%! We have found that, apart from relationship with the brand/site, the most important factor affecting response rate is survey style. Do not start surveys with a box giving people the option to either "take the survey or not". What you should do is to launch directly into questions. However ensure that the first page of your survey looks short and does not have a scroll bar.
2. Make navigation in online surveys simple. Most online survey software comes with options to have buttons to progress forward, back, stop or event delete your data (often called "reset"). As a general rule, turn off these options and ensure that you only ever give people the arrows to go forward (and back, if appropriate).
3. Use a progress bar. Again most software will allow you to insert a progress bar. It is very important that you use it so that people know here they are in your survey as there is no interviewer to tell them. Note, if you don't have a progress bar option, you can always just write ("Page 2 of 6" etc). Indeed, I actually think this can sometimes be better than an automatic progress bar, as the algorithms for these are sometimes lead to pretty misleading bars!
4. Include branding. Always provide reassurance of who the client is or your branding on surveys. It helps completion rates. Indeed, if you are doing website surveys, make your survey look like the site generally.
5. Keep online surveys short. If you're doing a pop-up visitor survey, you have a maximum of just 4-5 minutes of someone's time. If doing a survey amongst your customers, you probably have 5-10 minutes and panel members 10-15 minutes. However note there are no hard and fast rules here except the longer the survey, the bigger the dropout!
6. Avoid grids. Work has been done to see when people abandon online surveys and grids are the #1 location. This is partly because in most MR software type online interviews, the interface looks terrible and is very difficult to complete. By using some of the new online software, you can generally make them look better or even change them to dropdowns. However as a rule, make sure you cut down your image attributes to a minimum.
7. Check your survey on different browsers/screens. The web browser was supposed to be a platform independent. It isn't! Surveys can look quite different in Netscape and Internet Explorer. More importantly the users individual screen settings can have a drastic effect on your survey. The basic rule of thumb here is to design for a 640x480 screen with large fonts and then look at it in both Netscape 4 and Internet Explorer 4 & 5.
8. Take care with incentives. These are necessary for online panel surveys. People have been using a variety of incentives from conventional loyalty points (eg Air Miles), electronic money (like Beenz), real money and prize draws. Make sure you use a system that you can easily fulfill! Also take care with incentives on website surveys. We have shown that they have comparatively little effect on response rate and can seriously affect data quality by: encouraging multiple submissions, rubbish data or even people to flock to the website to "fill out the competition", ie your nice genuine visitor survey.
9. Have norms to interpret online data. Like all data, online data has its own baises. Some would argue that it can result in more accurate data, as there is no interviewer effect. Also you tend to get better quality opens. However most agree that there is an effect and so if you are converting a conventional survey online (eg a tracking study), take care to validate it and know what the differences are. For website surveys, we have developed our own norms. To illustrate how important they are, it may come of some surprise that even at the worst website we have ever researched, we still found over the half users saying they'd use it again!
10. Ensure you abide by Code of Conduct. ESOMAR has a clear Internet code of conduct - visit www.esomar.nl. In essence it makes sure that you apply the current code to online surveys too. Some key factors are therefore transparency, ie you must tell people it is a survey and provide them ways to contact/verify the researcher is bone fide. You must alert people to the fact that survey will cost them money to complete and tell them if you're using cookies. You must provide a link to your privacy policy. Finally take great care not to interview minors (under 14/15) without prior parental consent. Therefore pop-up surveys with kids are against code of conduct!