8 Tips for Creating the Perfect Online Survey

You put a survey online, 60 people took part and you still cannot do much with the answers? That is rarely down to the participants. It usually comes down to three or four decisions made while you were writing the questionnaire.

A successful online survey comes from deciding before your first question which decision you want to make at the end, keeping the questionnaire to 10 to 15 minutes, asking about only one thing per question, keeping answer options free of overlap and having three people who do not know the topic fill in the draft before you send it.

At empirio.ai, an online survey tool from Germany, the AI reviews the whole questionnaire and that is part of the free plan. The figures in this article come from the survey methodology literature on questionnaire length, response rates and data quality.

📌 The key points at a glance

• Ideal length of an online survey: 10 to 15 minutes.
• Every question asks about one thing, never two.
• In the US, web surveys sit about 9 points below other modes.
• AAPOR defines the response rate in six different versions.
• A pretest with three outsiders finds most of the errors.

What makes an online survey work

A survey has worked when you can answer a question at the end that you could not answer before. Not when lots of people joined in. A survey with 40 clean answers to one clear question is worth more than 400 answers to a questionnaire that asks about everything a little bit.

That is why it pays to work through the tips below by effect rather than in order. The biggest losses happen early: in the aim, in the length and in the wording. A survey tool will assemble the questionnaire for you, but no tool checks whether your questions are the right ones.

The eight tips cover the whole route, from the goal question to the analysis. If you are planning the outline at the same time, the detailed guide to survey structure helps with the individual sections. And if you would rather follow along on your own draft, you can create an online survey with empirio.ai in a few minutes and try every tip right away.

Tip 1: The goal question comes before the questionnaire

The goal question is the one sentence you want to answer at the end of your analysis. It comes before the first survey question, not after it. Without it you almost automatically end up with a questionnaire that touches on twenty interesting things and supports no decision at all.

An example from a local club: “Should we move the summer party to a Saturday?” is a goal question. “What do members think about the summer party?” is not. The first version tells you which questions you need, namely availability on Saturdays, travel distance and caregiving duties. The second version invites you to ask about everything.

The test for a goal question takes two minutes. Take every question in your draft and say out loud what you would do if 70 percent answered one way and 30 percent the other. If nothing comes to mind, the question can go. In a first draft this often removes a third of the questions and the survey gets noticeably shorter. The goal question also settles the shape of the study: do you need frequencies at the end, or reasons?

Any question whose answer changes nothing costs you participants and gives you nothing.

Tip 2: How long should an online survey be?

An online survey should take about 10 to 15 minutes, and the point at which respondents lose patience sits at 20 to 25 minutes on average. Those figures come from Melanie Revilla and Jan Karem Höhne (2020), who asked around 7,800 people in two online panels directly about the ideal and the maximum length. The data were collected in July and August 2019 and published in the International Journal of Market Research.

You will often read that a survey must not run longer than five minutes. That only holds for quick polls among friends. For a survey with a real research question you are giving away content that participants would happily grant you. What matters is not the number of minutes on its own, but whether the length fits the occasion and whether you announce it honestly beforehand.

Guide values, calculated at roughly 20 to 30 seconds per question. As of November 2026.

Why the announced length decides who takes part

The announced length affects participation before anyone sees the first question. Mirta Galesic and Michael Bosnjak (2009) told participants their questionnaire would take 10, 20 or 30 minutes. The longer the announcement, the fewer people started at all and the fewer reached the end. Questions placed further back were also answered faster, more briefly and more uniformly than the same questions at the start (Public Opinion Quarterly).

How to announce the length honestly

The honest time estimate is the one you measured in your own test run, rounded up. A flattering figure only pushes the drop-offs further back instead of preventing them: someone who notices at minute twelve that the promised five minutes were wrong will quit, and half-finished questionnaires are usually lost for the analysis. So put the questions your analysis depends on in the first half as well.

Tip 3: Write questions everyone reads the same way

A good survey question is understood in the same way by every participant and can be answered without thinking about the wording. Timo Lenzner and Natalja Menold set out the rules for this in the GESIS Survey Guidelines (2015). Most mistakes in everyday surveys break exactly four of them.

Almost every guide misses a fifth mistake: the unbalanced alternative question. “Do you think all members should take a shift?” names only one of the two options. According to Lenzner and Menold the option that is spelled out gains more agreement than the one left implicit. The fix is unspectacular: write out both sides, so “… or should that be left to each member?”.

Which question type fits which piece of information is a separate decision. The question types for questionnaires and surveys give an overview with examples, and the trade-off between free text and fixed options is covered in open and closed questions.

Tip 4: Keep answer options complete and free of overlap

Answer options have to meet two conditions: they cover the whole range, and no two options overlap. The classic version of the second mistake is age bands such as “20 to 30” and “30 to 40”. Anyone who is 30 can check both, and the analysis of that question is ruined.

Completeness is the less obvious condition. If the option a person would actually pick is missing, one of two things happens: they skip the question, or they check something at random. Both distort your result, and in the second case you will not even notice. An “Other” field with a text line costs you nothing and catches exactly these cases.

When a “don't know” option makes sense

A “don't know” option belongs everywhere the person being asked genuinely cannot know the answer. For knowledge questions and for topics that do not affect everyone it is essential. For attitude questions, however, it is a comfortable way out: anyone who would rather not commit will take it, and you lose the information. Rule of thumb: offer it where not knowing is realistic, otherwise put a filter question in front instead.

Odd or even scales for rating questions

An odd scale with five or seven steps has a real middle, while an even scale with four or six steps forces a direction. Both forms are legitimate, they simply do not measure the same thing. If a neutral position is a valid answer, use the odd scale. If you need a direction, as in a satisfaction measurement, the even scale is the more honest choice. Above all, stick to one form within a single questionnaire. How the most common of these scales looks and is labeled is covered in the Likert scale.

Why the order of answer options matters

In lists you click through, the options at the top are chosen more often than those at the bottom. Lenzner and Menold describe this primacy effect for visual presentation; when options are read out it reverses and the last items win. So never sort options by your own preference. Where there is no natural order, a random arrangement helps, and many survey tools offer that as a setting.

How to label scales and how many steps suit which question is covered in detail in rating questions in questionnaires and surveys.

Tip 5: Plan the order of your questions deliberately

The order of your questions has a say in how good the answers turn out. Galesic and Bosnjak (2009) showed that questions at the end of a web questionnaire are answered faster, more briefly and more uniformly than the same questions at the start. The last questions therefore get the weaker data every time.

A plain running order follows from that. What you genuinely need goes at the front. What would be nice to know goes at the back. Demographic details such as age, gender or major belong at the end, because they persuade nobody to take part and are still filled in reliably down there.

1. Opening question. Easy, concrete, answerable without thinking.
2. Core questions. The ones your analysis depends on.
3. Detail. Follow-up questions, free text, special cases.
4. Demographics. Age, gender, role, only if you will analyze them.

There is one exception: filter questions. If part of your questionnaire applies only to certain people, the filter question has to come before that part. So ask “Were you at the summer party?” first and show the detail questions only to those who said yes. That saves you from jumping backwards through the questionnaire.

Which demographic details you actually need is also a privacy question. Every extra detail shrinks the group an individual disappears into. In a class of 25, major, year and age together often make individuals identifiable, and for research at a university the institutional review board decides what you may collect. Demographic questions in surveys goes into this in detail and how participation stays genuinely anonymous is covered in the anonymous survey.

Tip 6: Have the questionnaire tested before you send it

A pretest is the single most effective step before sending and at the same time the one most often skipped. Three to five people who do not know your topic fill in the questionnaire while you sit next to them or stay on the phone. They tell you for every question how they understood it.

The pretest is nearly always skipped to save time, and that is the most expensive time saving in the whole process. An ambiguous question can be rewritten in two minutes before you send it. Once the link is out, your only options are to drop the question from the analysis or to report it with a caveat.

Which question to ask in a pretest

The value of a pretest hangs on the question you put to your testers. “Is that clear?” achieves nothing, because everyone politely says yes. “What exactly do you mean by your answer?” and “What did you understand this question to be asking?” bring the misunderstandings out, because the tester then describes in their own words what they read. In survey methodology this is called a cognitive pretest, and the GESIS Survey Guidelines list it as standard practice.

Checklist before you send

The points below can be checked in ten minutes and catch exactly the errors that cannot be fixed later. Go through them once in full before you send the link. If your questionnaire branches, click through every path separately, as described in logic and branching in surveys.

• Every question asks about exactly one thing.
• No answer category overlaps with another.
• The questionnaire works on a phone without sideways scrolling.
• The filter routing never jumps backwards.
• The announced length matches the test run.
• The privacy notice sits before the first question.

Many people underestimate the privacy notice. As soon as you collect personal information you need a short, understandable note on what you store and why. Which details count and what such a note can look like is covered in data protection in surveys.

If you cannot gather three testers, a second pair of eyes from outside still helps. At empirio.ai the AI reviews the whole questionnaire, and that is part of the free plan: a free account is enough for it. It does not replace a cognitive pretest with real people, because an AI cannot tell you how a club treasurer reads your question. It does catch the errors you can no longer see in your own draft.

Tip 7: Win participants and avoid drop-offs

The invitation text decides a large share of your responses. It answers three questions in three sentences: what this is about, how long it takes and what happens to the data. Hardly anyone reads more than that, and anything less costs you participants.

The second lever is timing. A link sent on a Friday afternoon disappears into the weekend, and one sent in the middle of finals week disappears into the rush. Send it when your group is reachable, and plan exactly one reminder after four to five days. Two reminders annoy more than they bring in.

What belongs in the invitation

✓ Topic in one sentence, length in minutes
✓ What happens to the answers and who sees them
✕ A long preamble about you, your major and your adviser
→ Only write “No names or email addresses are stored” if it is true

How to calculate the response rate of a survey

The response rate is the number of completed interviews divided by the number of eligible reporting units in the sample. That is how the American Association for Public Opinion Research defines it in the Standard Definitions (10th edition, 2023). Six versions of the rate sit side by side in that document, differing only in how they count partial interviews and cases of unknown eligibility. AAPOR also separates the response rate from the cooperation rate, which counts only the eligible people who were actually reached.

The figure your survey tool shows is usually a different one again. A completion rate divides finished questionnaires by the number of people who opened the questionnaire, and it is not an AAPOR rate at all. If you share a survey through an open link you cannot calculate a response rate, because nobody knows how many people saw the link. For a thesis that is no disaster, but it belongs in the methods section in exactly those words.

Why online surveys are answered less often than other modes

Online surveys get about 12 percentage points less response than the same survey by mail, by telephone or face to face. Jessica Daikeler, Michael Bošnjak and Katja Lozar Manfreda added up 114 experimental comparisons from 75 publications for this (Journal of Survey Statistics and Methodology, 2020). In the United States the gap is smaller than the average, at about 9 points.

What makes the gap smaller is more interesting than the average itself. For people who are already on a panel or a list the gap is about 9 points, and for people contacted once it is about 21. If the invitation arrives by email it shrinks to about 6 points. For your survey that means a mailing list you already have and an email invitation are the two levers with the greatest effect.

What incentives really do

Incentives raise response rates across all modes. Klaus Pforr collected the research on this for the GESIS Survey Guidelines (2016). Two findings are practical: cash works better than gifts, and an amount everybody receives up front works better than one only participants receive afterwards. For web studies the same review puts the odds of a response about 20 percent higher with an incentive.

The effect on answer quality is a different matter. According to the same review there is only weak evidence that incentives reduce skipped questions. Returns also shrink as the amount grows, so a larger payment does not buy a proportionally better response rate. And if your survey runs through a university, the institutional review board usually has rules on what you may offer and how you may describe it.

When a low response rate becomes a problem

A low response rate does not automatically mean a biased survey. Robert M. Groves compared 30 methodological studies and found no strong relationship between the size of the response rate and the size of the bias (Public Opinion Quarterly, 2006). What matters is not how many people answered, but whether those who answered differ from those who did not in exactly the characteristic you want to measure.

In practice that means two things. A rate of 20 percent is not a reason to discard a study on its own, but it does call for an explanation. And a high rate offers no protection when a whole group is systematically missing: 80 percent response from an Instagram following remains a statement about your followers. Write into your methods section who you reached and who you probably did not. Your institution's own guidance decides, not this article.

How many answers you need therefore depends less on the sheer number of participants than on how they were selected. A survey with 500 answers from an Instagram story describes your followers, not the population. Whether a survey is genuinely representative or not comes down to whether every person in your target group had the same chance of being asked. Which channels are worth it and which survey exchanges come at a price is covered in the ways to find participants for your online survey.

Tip 8: Spot useless answers without discarding the good ones

Useless answers are identified most reliably by completion time, not by attention traps. Dominik Leiner (2019) tested experimentally which markers give away carelessly completed questionnaires, and published the raw data alongside. The result is awkward for a lot of survey advice, because it recommends the opposite.

Leiner’s study is freely available in Survey Research Methods. Two points from it matter for your own analysis.

Why attention traps discard too many good cases

The classic trap question (“Please check ‘strongly agree’ here”) looks excellent at first glance. In Leiner’s third study it correctly identified 365 of 368 deliberately careless questionnaires, or 99 percent. But it also flagged 347 of 852 carefully completed ones as useless, or 41 percent. Screening strictly on that basis throws away roughly four out of every ten good answers.

Completion time as the better measure

Leiner proposes a relative speed index instead. You divide the typical completion time of the sample by the time an individual respondent spent per page on average. As a practical threshold he suggests a value of 2.0: anyone who was twice as fast as the rest very probably did not read the questions.

Leiner names one limitation himself, and it matters. For knowledge and factual questions, time is not a valid measure, because people who know the subject simply answer faster than those who do not. Then you are left looking at response patterns: someone who checks the same point twenty times in a rating grid either has very firm views or none at all.

For your analysis that means: do not screen on a single marker, look at the striking cases one by one. And record what you removed and why. In a thesis that documentation is a mark of quality, not an admission.

Common mistakes in online surveys

The four mistakes below turn up in almost every first survey. All four can be fixed in a few minutes before you send, and not at all afterwards.

All four show up as a data problem only in the analysis, even though they were created while the questionnaire was being written. Once you know the patterns, you will usually spot them in your own draft right away.

The survey measures something other than planned

A question such as “How satisfied are you with our offer?” measures the mood on the day of the survey, not the quality of the offer. If you want to know what specifically bothers people, you have to ask specifically, so separately about opening hours, price and range. Otherwise all you are left with is a pleasant average that supports no action at all.

The questionnaire is unusable on a phone

Wide scale batteries with seven columns look fine on a laptop and are barely usable on a phone. A large share of responses comes from phones, though, especially when you share the link through messengers or social networks. Open your draft on your own phone once before you send it.

The analysis was not thought through in advance

Free-text boxes take seconds to add and are expensive to analyze. With 200 responses, five open questions will cost you several evenings. So plan open questions sparingly, and one at the end is usually enough. For every closed question, decide in advance which analysis you intend to run on it.

Anonymity is promised but not delivered

“The survey is of course anonymous” sits in the invitation while email collection is still switched on in the tool. If that comes out, the trust is gone, and not only for this one survey. Check the setting in the tool before you use the word anonymous.

Conclusion

A successful online survey is not a question of the tool but of the groundwork. Anyone who settles the goal question, plans the length honestly, cuts every question down to one thing and has the questionnaire filled in once by outsiders has dealt with the four biggest sources of error. Everything else can be improved as you go. And if the response rate turns out thinner than you hoped: what matters is not the rate, but whether a whole group is missing.

What to read next

• Want to work through the questionnaire step by step? Writing a questionnaire properly
• Writing a thesis right now? Surveys for a bachelor's or master's thesis
• Still looking for the right tool? Survey tools compared

Ready for a survey of your own?

At empirio.ai, an online survey tool from Germany, you set up the questionnaire for free and try out logic and design before you send the link.

Create a free account