Open & Closed Questions: Comparison, Difference

Picture two questionnaires about the same campus dining hall. One asks “How satisfied are you with the food?” and offers five graded steps, the other asks “What comes to mind when you think about the food here?” and leaves an empty field underneath. Both questionnaires cover the same lunch, and yet they measure something entirely different.

Open-ended questions let respondents answer freely, closed-ended questions supply the choices in advance. The difference sits in the response format alone: both forms demand the same mental work of understanding and remembering, and they part company only at the final step, when the answer takes its shape. By the end of this article you will know which form fits your research question and how to combine the two without doubling your workload.

📌 The key points at a glance

• Closed-ended questions supply the answers, open-ended questions do not.
• Pew Research Center measured a 23 point gap between the two formats.
• GESIS distinguishes three response formats rather than two.
• Semi-open questions pair a set list of choices with a field for anything else.
• Question wording decides more about your data than the question form does.

Closed vs. open questions

Closed-ended and open-ended questions differ in exactly one respect: with a closed-ended question the answer choices already stand in the questionnaire, with an open-ended question respondents word the answer themselves. Everything else, from the analysis to the conclusions you may draw, follows from that single decision.

How much that decision changes the result was measured by the Pew Research Center after the 2008 presidential election. The same question about the most important issue for voters was asked once openly and once with five given options. In the closed-ended version 58 percent named the economy, in the open-ended version only 35 percent. And 43 percent of the respondents who saw the open version named something that did not appear on the list at all, against 8 percent who volunteered an off-list answer in the closed version.

Closed-ended questions produce quantitative data, meaning numbers you can count and compare. Open-ended questions produce qualitative data, meaning text that has to be sorted into categories before it can be counted. Rather than a question of effort, that is a question of what you want to learn: whoever needs to know how many needs categories, and whoever needs to know why needs the words of the respondents themselves.

Both forms appear in surveys of every kind, in person as much as in an online survey. The difference in the response format stays the same throughout, only the analysis usually costs less time online.

What is a closed question?

A closed-ended question supplies the answer choices, and respondents select one or several of them. The questionnaire settles in advance which answers are possible at all, and exactly there sit both the strength and the weakness of this form.

Underneath the familiar two-way split, however, hides a distinction that matters a great deal in practice. Lenzner and Menold (2015) separate not two but three response formats in the German-language GESIS Survey Guidelines on question wording: open-ended questions, closed-ended questions with a rating scale and closed-ended questions without one. Separate rules apply to the two closed forms.

Closed questions with a rating scale

A rating scale grades the answer choices along a single dimension, running for instance from very important to not at all important. Respondents place themselves somewhere on that scale. Because the sequence itself carries meaning, the order of the steps stays fixed, while the number of points, the labeling and the middle category each follow rules of their own.

Closed questions without a rating scale

A list without any internal grading covers cases such as college majors or political parties. Respondents pick either exactly one answer (single-choice questions) or several (multiple-choice questions). Since no natural order exists, the position in the list influences the choice, which is why Pew Research Center randomizes most of its answer lists and leaves only ordinal scales in sequence.

Examples of closed questions

• What is your favorite city in the United States? Chicago, Boston, Seattle, Austin
• Have you ever been to Chicago? Yes, no
• How would you rate your last vacation? From very good to unsatisfactory

Answer categories have to be exhaustive and mutually exclusive, a pair of conditions that Pew Research Center names in its own guidance and that takes minutes to check before a survey goes out. How you build such categories and how many steps a scale can carry is set out at length in the article on closed questions.

What is an open question?

An open-ended question offers no answer choices at all, only the question text and an empty field. Respondents answer in their own words, and those words are the result that no list of options could ever produce.

Open-ended questions usually start with one of the classic question words, so with why, how, what or which. Yet the question word alone does not make a question open. What counts is whether the question can be answered with a plain yes or no: “Do you know the reasons why Chicago is so popular?” invites a curt yes, while “Which reasons speak for Chicago?” does not.

Examples of open questions

• Why is Chicago your favorite city?
• What do you dislike about Boston?
• Which other cities have you traveled to?
• What would you like to tell us before you finish?

Placement matters here almost as much as wording. Pew Research Center reports that respondents are far more likely to repeat concepts from earlier closed-ended questions in their free-text answers, and it therefore places its open-ended questions about national problems near the beginning of the questionnaire. Generally, open-ended questions pay off wherever you do not yet know the possible answers, meaning in exploratory work. How you code those answers afterward and still arrive at numbers is shown in the article on open questions.

Semi-open questions: the hybrid of both

A semi-open question offers a list of answer choices and adds a free text field, usually labeled “other, please specify”. You get countable categories and still keep whatever escaped you while you were building the list.

In the literature the same form travels under several names, since semi-open, partly open and semi-closed all mean the same thing. Lenzner and Menold (2015) recommend the other category expressly for every case in which the possible answers cannot be listed in full, and with substantive questions that is the rule rather than the exception. Pew Research Center reaches the same requirement from the other direction, because an answer list has to be exhaustive, and an other category is what makes exhaustiveness achievable.

Without that category two things can happen, and neither of them is good. When an obvious answer is missing, the question either stays unanswered, or something gets checked at random. Annoying in the first case, dangerous in the second, because a random answer looks like a result in the analysis rather than like an error.

When the hybrid form pays off

Semi-open questions make sense whenever you know the subject area well but not completely. A question about the mode of transportation someone uses covers almost everything with six categories, but not the cargo bike and not the RV. Equally, the form pays off at the end of a questionnaire, where a single open question about anything remaining often yields the most usable pointers.

What you have to watch

The free text field belongs directly beside its category and not in a question of its own, otherwise nobody sees it. In the analysis you handle these answers separately: first you check whether several of them can be merged into a new category, then you count them alongside the given ones. When entries pile up under other, that is no accident but the finding that your list was too short.

💡 Tip

Read the other answers once about 30 people have taken part. When the same thing appears three times, you can still add the category to the list while the survey is running.

The wording of the questions matters

Question wording decides more about the quality of your data than the question form does, and a solid reason stands behind that: of the four mental steps a respondent goes through, the question form touches only the last one.

Lenzner and Menold (2015) describe this response process after Tourangeau as four tasks: understanding the question, retrieving the relevant information, forming a judgment from it and expressing that judgment in the given format. Whether you ask openly or with fixed choices takes effect only at the fourth stage. The first three hang entirely on the question text, and there arise the errors that no amount of later work can repair.

What every question has to satisfy, whatever its form

• One issue per question, never two stimuli at once.
• No jargon, technical terms or abbreviations in the question text.
• No vague words such as often or recently.
• No double negatives, and ideally no negation at all.
• No leading questions, both alternatives spelled out.
• Short active sentences instead of passives and nominalizations.

An example from the same source shows how quickly this goes wrong. The statement that the state must ensure a decent living in illness, hardship, unemployment and old age carries four issues at once. Anyone who supports three of them and rejects the fourth cannot answer the question, and anyone who answers regardless produces a number that nobody can interpret afterward.

⚠️ Watch out

Agree-disagree scales fit any statement and therefore get overused. Pew Research Center warns that they invite acquiescence bias, because less educated and less informed respondents agree more readily regardless of content. Offering a choice between two alternative statements avoids the effect.

No answer list in the world can rescue a badly worded question.

Final decision

Which question form you take is decided by the kind of statement you want to make at the end. Do you need shares, comparisons and charts, then the road runs through closed-ended questions. Do you need reasons, phrasings and new aspects, then you need open ones.

For the typical standardized survey one weighting has proved itself: mostly closed-ended questions for the measurable content, plus a few open ones at the points where the why interests you. Such a mixture keeps the analysis workload manageable without giving up the explanations your respondents can offer.

Which further question forms you will meet in a questionnaire, from the matrix to the ranking, is sorted out in the overview of question types in surveys.

Conclusion

Choosing between open-ended and closed-ended questions is not a matter of taste but an anticipation of the analysis: whatever you fix as a category up front you can count later, and whatever you leave open you have to sort out later. Yet neither of the two forms decides on its own how good your data turns out. Question wording does, and that is the same work in both cases.

Where to go next

• Want to build answer categories properly? Closed-ended questions: definition, types, pros and cons
• Need to analyze free-text answers? Open-ended questions and how to code them
• Looking for further question forms? Question types and answers in questionnaires
• Building your questionnaire right now? How to create a questionnaire

Pew Research Center sets out its own approach in Writing Survey Questions, and the German-language rules on question wording are published in full by Lenzner and Menold (2015).

Want to try both question forms side by side?

With empirio.ai, an online survey tool from Germany, you can put free text fields, answer lists and rating scales into the same questionnaire and watch the results come in live. Create a survey for free