Rating Questions in Questionnaires & Surveys

Picture two feedback forms about the same university canteen. One asks “Do you like the food?” with yes and no, the other asks “How would you rate the food?” with five steps from very good to very poor. The first splits people into two camps, the second shows how far apart they are.

A rating question is the question type that measures attitudes, opinions and self-reported behaviour in graded steps. You put a statement or a question to your respondents and they answer it on a graded scale, using words, numbers or symbols. After this article you will know which rating options you have and how to word the question so that the answers mean something.

📌 The key points at a glance

• A rating question measures degrees instead of yes or no.
• Verbal, numeric and visual are the three rating options.
• Rating question means the wording, rating scale means the answer format.
• Porst advises against double stimuli and negatives.
• Without an open follow-up question you never learn the reason.

What is a rating question?

A rating question is a question type that measures attitudes, opinions and behaviour. Respondents are shown a statement or a question and judge it on a scale with several steps, for example from “very satisfied” to “very dissatisfied”.

Among the question types in a questionnaire, the rating question belongs to the closed questions. Lenzner and Menold (2015) separate three large groups of answer formats in the GESIS Survey Guidelines: open questions, closed questions with rating scales and closed questions without rating scales. A rating question works with the second format.

One note on the term itself, because it saves time while you are hunting for sources. “Rating question” is mainly a practitioners’ word. Methods literature usually names the answer format instead and talks about rating scales or rating procedures. Search for rating scale rather than rating question and far more solid material turns up.

Rating question and rating scale: where the difference lies

The rating question is the wording, the rating scale is the answer format underneath it. The two belong together and are still decided separately: the wording fixes what gets rated, the scale fixes which steps are allowed. How many steps your scale needs, whether it has a middle and how you label the steps are therefore decisions of their own, and our article on the rating scale works through them.

Plenty of overviews online throw both together and list nothing but scale variants under “rating questions”. What counts is the combination. A carefully built scale does not rescue an unclear question, and a precise question does not help if the steps do not fit it. This article deals with the question side.

The scale decides how finely you measure. The question decides what you measure.

What you may calculate with answers from a rating question

Answers to a rating question sit on an ordinal scale, strictly speaking. The steps come in a clear order, but there is no evidence that the gap between “good” and “average” is the same size as the gap between “average” and “poor”. You often read that rating questions sit at interval level and that the mean is therefore fine. That is not a property of the data, it is an assumption.

The mean is defensible only when you state that assumption and justify it. As a rule of thumb, a single rating question is better described by its frequency distribution and its median. Only when several statements on the same topic are combined into one overall score, as is usual with the Likert scale, does the mean become the standard, and even then equal spacing stays an assumption that belongs in your methods section.

Which rating options are there in a questionnaire?

Three rating options cover almost everything a rating question can do: the steps carry words, they carry numbers or they carry pictures. What respondents see on screen shapes how carefully they answer and how well you can read the answers afterwards.

The choice is not a matter of taste. Words are the least ambiguous, numbers only work with systems your target group already knows, and pictures collect answers where words get in the way. Let us go through the three one by one.

Verbal rating: one word per step

With verbal rating every step carries its own wording, for example “does not apply at all”, “applies slightly”, “applies in part”, “applies quite a lot” and “applies completely”. Fully labelled steps are widespread in academic questionnaires, because nobody has to guess what a step means. The price is space: beyond seven steps you will struggle to find words whose gaps feel even.

Numeric rating: numbers instead of words

With numeric rating, numbers carry the steps, usually 1 to 5, 1 to 7 or 0 to 10. Numbers save space and they travel well across languages. What they do not do on their own is tell anyone which end is the good one.

School grades are where that bites, because they are the number scale most people carry around in their heads, and in the UK that scale is not the one everyone expects. In England, GCSEs are graded on a scale that runs from 9 down to 1, with 9 as the highest grade, which replaced the older letters from A* to G. Wales kept the A* to G scale, so even a UK-wide sample brings two habits into one questionnaire.

Across borders it gets sharper. Someone schooled in Germany reads 1 as the best mark, someone who sat GCSEs in England reads 1 as the bottom of the scale. Put a bare scale from 1 to 6 in front of both groups and one half answers in the opposite direction from the other, without anyone noticing. So label at least the two ends with words, every time.

One special form of numeric rating has grown into a standard of its own. The Net Promoter Score uses a range from 0 to 10, where usually only the two ends carry a word, and turns the answers into a single figure by a fixed rule.

Visual rating: symbols, stars, smileys

With visual rating, pictures take over the steps: stars, smileys, thumbs or icons. Their strength shows wherever reading slows people down, so with children, with quick feedback after an appointment and in surveys that cross language borders. Their weakness is interpretation, because a smiling face means “good” to one person and “very good” to the next.

For a dissertation, visual rating is therefore rarely the first choice. For a village fête or a quick word at the counter it is often the only one that gets answered at all.

Where rating questions are used

Rating questions come in whenever respondents are meant to give a subjective judgement and a yes or no would be too blunt. In sociology and psychology they are basic equipment, and in market and opinion research they shape a large part of most questionnaires.

The common core of these fields is a translation job. Feelings, attitudes and satisfaction do not arrive as numbers, and a rating question turns them into graded categories you can work with afterwards.

Four fields turn up especially often in practice:

• Satisfaction: with a product, a course, a club
• Self-assessment: your own nerves, your own skill, your own workload
• Judging others: a teacher rates a class, a team rates its management
• Importance: how important each criterion is to respondents

At universities the module evaluation questionnaire is the best known case, and it shows how tightly wording and purpose hang together. Statements such as “The module was well structured” get rated in steps there, because one step carries more information than a yes or no, and a form that stays the same keeps results comparable across several terms. In companies the same pattern runs under measuring employee satisfaction.

Wording a rating question: four rules from survey research

A rating question works when it raises exactly one issue, carries no negative and no assumption, names a clear time frame and offers answer steps that match the dimension you asked about. Four rules cover nearly all of it.

Most useless ratings are not caused by the scale, they are caused by the line above it. When a question asks about two things at once, or leaves open what it refers to, respondents still tick something. Afterwards, though, nobody can say what they meant.

There is well documented guidance for this part of the job. Rolf Porst gathered his in the “10 commandments” of question wording (Porst 2000, ZUMA How-to series no. 2) and points out himself that they are not unshakeable rules. The GESIS Survey Guidelines on question wording by Lenzner and Menold (2015) explain, independently of him, why such errors overload respondents. For the UK, the questionnaire design guidance of the Government Analysis Function (2023) arrives at strikingly similar advice for government surveys. Three of the four rules below go back to Porst, the fourth is about how question and scale work together.

Rule 1: one issue per question, not two

Porst puts this in his fourth commandment: avoid double stimuli and negatives. Lenzner and Menold describe the same problem. Paraphrased from their German text: questions that raise two or more different issues get hard to answer as soon as respondents judge those issues differently.

The Government Analysis Function turns that into an instruction: “If you have a question that asks for two pieces of information, you should split it into two separate questions.” Its own example is a course where the facilitator and the materials get rated in one line. Anyone who liked one and not the other has no honest answer left.

Rule 2: no negative and no assumption in the wording

Negatives are especially expensive in rating questions, because the scale underneath has a direction. Lenzner and Menold, citing Dillman and colleagues, recommend avoiding negations such as “not” or “no” in questions altogether, since they are easily overlooked; that recommendation is a paraphrase of their German text, not a quotation. Under a negated statement with an agreement scale, the meaning of an answer flips for part of your sample.

Porst’s fifth commandment names assumptions as well. “How much did you like the new layout?” already takes for granted that you liked it, and every step of the scale then measures degrees of a liking that may not exist.

Rule 3: one clear frame of reference, in subject and in time

A rating needs an object and a period, otherwise every person rates something else. Porst therefore asks for questions with an unambiguous time reference and treats concrete dates as anchors. The Government Analysis Function says it plainly: “You should be specific about when a time period begins and ends.” Instead of “last week”, the guidance suggests naming the actual dates.

For rating questions that means “How would you rate the supervision you got last term?” rather than “How would you rate the supervision?” Two words of difference, and suddenly everybody is rating the same stretch of time.

Rule 4: wording and answer steps have to match

The steps under the question have to cover exactly the dimension you asked about. A question about satisfaction gets satisfaction steps, a question about frequency gets frequency steps. Sounds obvious, happens constantly: under “How important is quick feedback to you?” you will surprisingly often find steps from “strongly agree” to “strongly disagree”. What else answer options have to deliver is in the article on closed questions.

One check remains that no set of rules can replace. Read your rating question out to somebody who does not know the topic and ask what they are supposed to be rating. If the answer differs from your intention, the question is at fault, not the reader. Where such a test run belongs in the process is in the guide to creating a questionnaire.

Rating or criticism? Why a follow-up question belongs with it

A rating question tells you how things stand and never why. It hands you a step, not a reason: whether somebody rates the canteen as “poor” because the food was cold, because the queue is too long or because prices went up, the ticked step does not say.

So plan an open follow-up question for every rating question whose result is meant to lead to a decision. Who actually gets to see it is a second step. That is one more field in the questionnaire and it usually brings you more than a second scale would. How to turn the text answers into countable categories afterwards is shown in the article on open questions.

Asking for criticism on purpose is a different job from collecting a poor rating. If you want to know what somebody would change, ask about the change rather than the mark: “What should we change first about this module?” produces concrete suggestions, while a scale from “very good” to “very poor” only records how satisfied somebody is.

Conclusion

A rating question takes a minute to write and is ruined just as quickly. One issue per question, a clear time frame and answer steps that match what you asked will do more for you than any extra step on the scale. And if the why interests you, plan the open field in from the start, because somebody who has already voted can hardly be asked for the reason later.

What to do next

• Still deciding on steps and a middle? Rating scale: steps, labels and the middle
• Working with several statements on the same topic? Likert scale
• Unsure about the level of measurement? Determining the level of measurement
• Answers already in? Evaluating a questionnaire

Want to see your rating question in a real form before it goes out?

With empirio.ai, an online survey tool from Germany, you set up a graded rating question in a few minutes and check after a handful of test answers whether the distribution gives you what you want to report.

Get started with empirio.ai for free