Questionnaire Design: How to Create a Questionnaire

You want to know what your customers, members or colleagues actually think, and you have an empty document in front of you that now needs questions in it? The hard part is not thinking of questions. The hard part is asking questions whose answers actually get you somewhere.

You create a questionnaire by translating your aim into measurable individual questions, arranging those into a fixed order of introduction, main section, questions about the person and closing, and giving the draft to a few test readers before you send it. By the end of this article you will have a questionnaire you can actually analyse, rather than one that falls apart when you try.

📌 The key points at a glance

• Every question must trace back to what you want to know.
• The structure follows four parts: introduction, main section, personal details, closing.
• Questions about the person belong at the end, and that is evidenced.
• Two topics in one question make the answer unusable.
• The average reading age in the UK is nine years.

What is a questionnaire, and where does it end?

A questionnaire is a fixed sequence of questions and response options presented to every participant in identical form. That sameness is exactly what makes the answers comparable, and therefore countable: only if everyone is asked the same thing may you add the answers up at the end.

Survey and questionnaire get used interchangeably in everyday speech. The survey is the whole undertaking, from the aim through the question of who you actually reach to the analysis. The questionnaire is the instrument inside it. The distinction sounds pedantic, but it saves you from the most common mistake: a good questionnaire cannot rescue a survey that the wrong people answered.

The questionnaire decides what you can measure. Who answers decides who the result applies to.

Only both together give you a result you can rely on. Where that line runs is set out in our article on the representative survey. You will often read that a survey with numbers needs a representative sample. It does not: representativeness depends on how participants were selected, not on the method and not on the sheer quantity. A survey among your own customers is still worth analysing, it simply applies to that group.

How to create a questionnaire: the seven steps

A questionnaire comes together in seven steps, and the order is not a matter of taste. Anyone who starts by writing rather than by working out the aim ends up with questions that sound interesting and answer nothing.

The steps below apply whether you are running a customer survey, asking your members or collecting data for a dissertation. The effort is distributed differently from what most people expect: steps 1 and 2 take the most time, typing everything into a tool takes the least.

1. Set the aim. Write down in one sentence what you want to know at the end.
2. Break it into parts. How would you recognise it if it were the case?
3. Write the questions. One or more concrete questions per part.
4. Choose the response format. Open, tick-box or scale.
5. Fix the order. Introduction, main section, personal details, closing.
6. Run a pilot. At least five people from your target group.
7. Revise and launch. Send only after the final correction.

The tool only enters at step 7. If you survey online, you enter the questions directly into a survey tool, set routing and required fields there, and share the questionnaire as a link or QR code. On paper you carry the same structure into a document, but you need your own solution for entering the answers.

From what you want to know to the questions you ask

Between your aim and the first question in the questionnaire lie two intermediate steps that almost every guide skips: you have to break your topic into parts and find, for each part, something that can actually be asked. The technical term for this route is operationalisation. In practice it is one question: how would I notice?

Here is an example that recurs throughout this article. Priya runs a small online shop for cycling accessories and wants to know why so many customers never come back after their first order. She cannot ask anyone about “satisfaction” directly, at least not usefully. The answer would be a shrug and four stars out of five, which tells her nothing. The topic has to be taken apart first.

The practical value of these four levels shows up when you cut. Any question you cannot assign to a part goes. That is the most effective measure against an overlong questionnaire, and it works better than any cap on the number of questions. Going through hers, Priya notices that the question about favourite bike is charming but feeds none of her four parts.

💡 Tip

Add a second column beside your questionnaire and note, for every question, which part it covers. Delete that column before you send. It doubles as your analysis plan, and if you are collecting data for a dissertation, half of your methodology chapter.

The structure: four parts, and why personal details go at the end

A questionnaire has four parts in a fixed order: introduction, main section, questions about the person and closing. The main section opens with easy, general questions and becomes gradually more specific, with sensitive questions placed late but not right at the end.

The order is not a cosmetic detail, it changes the answers. An experiment by Mirta Galesic and Michael Bosnjak shows what happens to questions the further back they sit: they are answered faster, more briefly and more uniformly (Galesic & Bosnjak 2009, Public Opinion Quarterly). The same study shows that a longer announced completion time produces fewer starts and fewer completions.

The rule of thumb follows directly: what matters most to you belongs at the front. Details such as age, gender or where someone lives belong at the end, because they are still answered reliably once attention drops. They cost almost no thought, and they are the first thing participants find intrusive when they appear at the start. Which characteristics you actually need is covered in our article on demographic questions.

You will often read that difficult questions belong in the middle, because attention supposedly peaks there. That rule appears in several older handouts without any evidence, and it does not fit the findings of Galesic and Bosnjak. Plan the other way round: first the questions you really care about, then everything else.

Wording questions so they measure what they should

A good question is one that every respondent understands in the same way and can answer without much effort. Answering a survey question means working through four mental steps: understanding the question, recalling relevant information, forming a judgement, and fitting that judgement into the response format offered.

The questionnaire design guidance published by the Government Analysis Function and owned by the ONS Data Quality Hub turns this into practical rules, and it adds a number worth remembering: the average reading age in the UK is nine years. That is the level your wording has to clear, not the level of your reading list.

Understandable: no jargon, no nested clauses

Replace technical terms and abbreviations with everyday words, write in the active voice and avoid turning verbs into nouns. “Sufficient working time is available to me for the acquisition of new skills” becomes “I have enough working time to learn new skills”. Both sentences mean the same thing, the second costs less mental effort.

Unambiguous: one issue per question

Double-barrelled questions are unanswerable for anyone who rates the parts differently. “How satisfied are you with your pay and your working hours?” leaves no valid answer for someone happy with one and unhappy with the other. Split the question. The same applies to negatives: a negatively worded statement combined with an agreement scale creates a double negative that nobody wants to untangle.

Neutral: no leading, no half alternative

Phrases such as “leading scientists believe” invite agreement. The less obvious case: in a yes-or-no question, spelling out only one of the two alternatives gets that alternative more agreement. Instead of “Should everyone on the team be able to work from home?” ask “Should everyone on the team be able to work from home, or should each person decide for themselves?”

Answerable: clear timeframe, clean categories

Vague terms such as “recently” or “often” mean something different to everyone. Name a concrete period, and match it to the behaviour: short periods for frequent things, long ones for rare events. Response categories must be exhaustive and must not overlap. Someone who gets up at 6:30 should not have to choose between “6:00 to 6:30” and “6:30 to 7:00”.

Question types and scales: only as much as you need

The response format decides what you can calculate later: closed questions and scales produce countable data, open questions produce wording that you have to code first. As a rule of thumb, anything destined for a table in your dissertation is asked as a closed question.

Which type suits which job is set out in detail in our overview of question types and response options. The choice between free and prescribed answers is covered in our comparison of open and closed questions, and the number of points, the middle category and the labelling have their own article on the rating scale.

One point still belongs here, because it concerns the structure. Lists with several response options produce order effects: on screen the first options are chosen more often, when read aloud the last ones. The ONS guidance recommends placing socially desirable options at the end of a list for exactly that reason, and having each option rated individually rather than offering a tick-all list.

Plan the analysis while you are still writing

What you can do with the answers later is decided while you build the questionnaire, not afterwards. The response format determines what can end up in a table or a chart and what you will have to sort by hand.

For Priya this makes a concrete difference. If she asks how many orders someone placed last year, she gets a number and can work with it, calculate an average or compare two groups of customers. If she instead asks for categories such as “rarely, sometimes, often”, she has a rank order without fixed intervals: she knows that “often” is more than “sometimes”, but not by how much. Both questions look equivalent while writing. In the analysis they are not. Which format permits which calculation is explained in our article on the level of measurement.

Three things to settle before you send:

• Response format per question, matched to what you plan to do with it.
• Coding of the answers, meaning which number stands for which category.
• Routing, so nobody sees questions that do not apply to them.

If you need benchmark figures for the UK population, take them from the Office for National Statistics rather than from your own data. Open questions are the item most often underestimated: sorting two hundred free-text answers takes longer than the entire rest of the analysis.

The introduction: what has to come before the first question

The introduction answers four questions before anyone asks them: who is asking, what for, how long it takes and what happens to the data. Four to six sentences are enough, nobody reads more.

It is the part of the questionnaire with the worst effort-to-effect ratio, and that is in favour of the effect. People who do not know what they are answering for drop out. For Priya’s survey the text might read:

I am Priya and I run the cycling accessories shop. I would like to know what went well with your first order and what did not. The survey takes about four minutes. Your answers are analysed anonymously, I do not link them to your order. Thank you for taking the time.

The duration belongs in there, but it has to be true. Measure it in the pilot rather than guessing. And as soon as you collect personal data, a friendly sentence is no longer enough: the UK GDPR and the Data Protection Act 2018 bring information duties with them. That is exactly why Priya’s last clause is more than politeness, it describes what she actually does. What has to go into the text in that case is set out in our article on data protection in surveys. If you are collecting for a university or a larger organisation, their own rules apply on top.

Pilot testing: the step almost everyone skips

A pilot test is a trial run of your questionnaire with a few people from the target group before you really send it out. It is not an optional extra for unusually thorough projects, it is the only way to find out beforehand whether your questions are understood the way you meant them.

The ONS guidance distinguishes three forms of testing: cognitive testing, which examines how people understand and process a question, usability testing, which examines whether they can operate the form, and cogability testing, which combines the two and is the form the guidance recommends. For most questionnaires the small version of this is enough.

1. Find five people from the target group. Nobody who already knows what you are getting at.
2. Ask them to think aloud. Tell them to say whatever goes through their mind as they complete it.
3. Time it. The measured duration goes into your introduction later.
4. Collect the queries. Every question they ask you marks a question that is unclear.
5. Revise and repeat. Until no more queries come back.

The most common finding in a pilot test, incidentally, is not the ambiguous question but the missing response option. If testers hesitate between two categories or look for a third, the list is not complete.

Common mistakes when building a questionnaire

The most expensive mistakes are made while planning, not while writing. They only surface during analysis, when nothing can be changed any more and the answers are already in.

The four below turn up again and again, in customer surveys as much as in club surveys and dissertations. All four can be fixed in half an hour before you send.

Collecting questions instead of deriving them

Gathering questions because they sound interesting produces data that cannot answer the question you actually had. Work backwards through the questionnaire before sending and assign each question to a part. Whatever is left over gets cut, however much it hurts.

Switching scales halfway through

If half your statements sit on a five-point scale and half on a seven-point scale, you cannot analyse them together. Decide on one format and keep it across the whole questionnaire.

Making everything a required field

A questionnaire that forces an answer to every question does not produce complete data, it produces drop-outs and guessed answers. Use required fields only where a missing entry makes the case unusable, and offer an opt-out on sensitive questions.

Testing the questionnaire on yourself

Your own questionnaire is always clear, because you know what you meant. That is precisely why reading it through yourself does not replace a pilot test. Ask people who do not know your topic, and listen without explaining.

Conclusion

A good questionnaire is not a piece of writing, it is a carefully thought-through list. If you take the time to break your aim into parts and have the draft tested once, the wording gets easier almost by itself, because you know what every question is for. The most common reason for disappointing results is not a badly worded question, it is a question that should never have been asked.

Where to go next

• Would you rather not start from scratch? Ready-made questionnaire templates to adapt
• Would you rather not sign up at all? Creating a survey without signing up: what works and what doesn't
• Surveying your customers? Customer surveys: timing, occasion and analysis
• Collecting data for a dissertation? Surveys for your dissertation: the extra requirements
• Do you need more participants? Find survey participants: the five channels compared
• Is the response rate stalling? Tips for solid answers and a good response rate

Questions ready, questionnaire still missing?

With empirio.ai, an online survey tool from Germany, you set up your questions with scales and routing and share the questionnaire as a link or a QR code.

Build your questionnaire online