How to Evaluate a Questionnaire: 5 Steps to Results

The survey is done, 80 completed questionnaires are sitting in your inbox, and now what? This is exactly where most surveys stall. Collecting the answers was half the work, the other half is the evaluation, and from a distance it looks more complicated than it is.

Evaluating a questionnaire means turning individual answers into a result: you check the response rate, sort and code the data, count the closed questions by frequency, group the open questions into categories and finally read off what the numbers say about your research question. This guide walks you through the evaluation in five steps, from the first table to the finished statement, whether you work in Excel or use a survey tool.

📌 Key takeaways

• Evaluating means: check the response rate, sort the data, count closed questions, categorize open questions, interpret.
• How you evaluate a question depends on the level of measurement, not on gut feeling.
• Closed questions give you frequencies and, depending on the scale, averages; open questions a category system.
• In Excel one row stands for one person and one column for one question.
• A survey tool does the counting automatically and saves you the typing.

What does evaluating a questionnaire mean?

Evaluating a questionnaire means summarizing the collected answers so that they answer your original question. Many individual opinions turn into a pattern that can be expressed in numbers, tables and a few sentences. Evaluating an online survey works the same way, the answers are simply already digital.

The path the evaluation takes is decided by the type of question. Closed questions with preset answers are evaluated quantitatively, so you count how often each answer occurs. Open questions with free text are evaluated qualitatively, so you group similar statements into categories. How the two types of question differ is covered in the article on open and closed questions.

This distinction is not theory, it saves time: it sets which calculations are allowed in the first place. Taking an average across genders makes no sense, no matter how cleanly Excel works it out. Which question types exist and how to evaluate them is laid out in the overview of question types and answers.

Evaluating a questionnaire in five steps

You evaluate a questionnaire in five steps: check the response rate, sort the data, count the closed questions, categorize the open questions, interpret the results. The order keeps the evaluation clean, because each step builds on the one before.

1. Check the response rate. Count how many complete questionnaires came back and sort out empty or obviously unserious ones. Only this number tells you how reliable your later evaluation is. Whether it is enough to draw a conclusion depends on the sample.
2. Sort the data. Transfer all answers into a table, one row per person, one column per question. Closed answers get a number so that you can count and calculate later.
3. Count the closed questions. For each question, determine how often each answer occurs and, depending on the level of measurement, the right measure, so mode, median or mean.
4. Categorize the open questions. Read the free-text answers, form categories for recurring statements and assign each answer to a category.
5. Interpret the results. Read off from the frequencies and categories what they say about your original question, and capture the key findings in a few sentences.

💡 Tip

Set up the evaluation while you are still building the questionnaire. Anyone who knows in advance how a question is meant to be evaluated phrases it differently, often more closed. How that works is in the guide on creating a questionnaire.

Evaluating closed questions: frequencies and averages

You evaluate closed questions by counting the answers and determining the right measure for each level of measurement. Which measure fits depends on the level of measurement of the question, that is, whether the answers can only be told apart, ranked or actually calculated with.

That sounds academic, but it decides whether your result holds up. The level of measurement sets, for each question, which evaluation is allowed.

The most common answer format in questionnaires is the Likert scale, that is, agreement from "strongly disagree" to "strongly agree". It is ordinal, because the steps are not equal distances apart. The clean choice is therefore the median, yet in practice the mean is often reported. Both are defensible, as long as you note what you calculated. More on these questions is in the piece on rating questions.

⚠️ Careful

An average over a nominal scale is always wrong. "Gender 1.7" or "department 2.3" says nothing, even if Excel works the number out without complaint. For nominal questions you only count, you do not calculate.

How the counted values become reliable measures is explored further in the article on descriptive statistics.

Evaluating open questions: categories instead of quotes

You evaluate open questions by grouping similar answers into categories and then counting how often each category occurs. That turns free text into a structure you can evaluate just like a closed question.

The appeal of open questions is that respondents answer in their own words. That is exactly what makes the evaluation more work, because 80 answers first of all mean 80 different wordings. What open questions deliver and where their limits lie is set out in the article on them.

In practice you go through three steps:

1. Read the answers across. Get an overview without sorting yet. The first recurring themes will start to show.
2. Form categories. Group similar statements into a few, clearly named categories. Five to eight almost always suffice, more categories get confusing.
3. Assign and count the answers. Assign each answer to exactly one category and, at the end, count how often each category was named.

After that there is a number next to each category, and the open question can be reported like any other: "Of 80 respondents, 31 named the processing time as the biggest annoyance." You can add individual verbatim quotes, but they do not replace the count.

Evaluating a questionnaire in Excel

In Excel you evaluate a questionnaire by transferring the answers into a data table and counting them with a PivotTable. The table is the basis for everything else, so its structure has to be right from the start.

The rule is simple: one row per person, one column per question. Closed answers you enter as a number, for example 1 for "strongly disagree" through 5 for "strongly agree". You record this assignment in a small code plan so that it stays clear later what the 3 stood for.

Once the table is in place, three tools lead to the result:

• A PivotTable for frequencies and crosstabs, so "how many answers per level".
• Functions like COUNT, AVERAGE and IF for individual measures.
• Charts for the presentation, usually a bar or column chart per question.

For small surveys that is entirely enough. It only gets laborious when you first have to type in the answers from paper forms, because that is where transposed digits creep in that nobody spots later in the evaluation.

💡 Tip

The typing disappears completely when the survey runs digitally from the start. A survey tool counts closed questions automatically and draws the charts from the first answer onwards.

Which mistakes ruin the evaluation?

Most evaluation mistakes do not happen during the calculation, but before and after it: in the choice of average, in the amount of data and in the interpretation. Four mistakes come up again and again.

Wrong average for the level of measurement

The classic is the average over a question that does not allow one. As soon as you take a mean over nominal or ordinal answers, you get a number without meaning. The table above shows which value belongs where.

Percentages on tiny sample sizes

"67 percent prefer option A" sounds strong, but with three respondents it means two people. Percentages need a base size before they carry weight. With small groups it is better to name the absolute number.

Inferring from your result to everyone

What your respondents say holds, at first, only for your respondents. Whether the result can be generalized depends on how representative the sample is. Without that basis, every statement stays limited to the participants.

Ignoring non-responses

Unanswered questions are not nothing, they are information. If a question is skipped strikingly often, it may have been unclear or too sensitive. Anyone who simply calculates missing answers away shifts the result unnoticed.

Checklist before the evaluation

These five points are best caught before you report the first number.

• Every question is evaluated according to its level of measurement.
• Percentages appear only with a sufficient sample size.
• Absolute numbers are named for small groups.
• Missing answers are shown, not hidden.
• The reach of the statement matches the sample.

Conclusion

Evaluating a questionnaire is less about calculating than about ordering. Whoever gets the data cleanly into a table, treats each question according to its level of measurement and groups the open answers into categories has done the real work. The rest is counting, and the software can take that over if need be. The biggest source of error is not the statistics, but the average in the wrong place.

Where to go from here

• Want to build your questionnaire from the ground up? Creating a questionnaire the right way
• Looking for the right tool? Survey tools compared
• Need the statistics behind it? Descriptive statistics and inferential statistics
• Evaluating a customer survey? Calculating the Net Promoter Score

Evaluate a survey without typing it up?

With empirio.ai, an online survey tool from Germany, the answers flow straight into the evaluation and every question is counted automatically. Create a survey for free