why open ended questions belong at the start of a survey (2026)

Open-ended questions belong at the start of a survey because they collect unprompted answers while the respondent’s attention and willingness are at their peak, and because they cost the analysis nothing you did not ask for. Question order effects run one direction: an early closed-ended item frames everything after it, while an early open item frames nothing. Put the open question first and the closed questions that follow get more accurate, because the response options were built from real language rather than from the researcher’s assumptions.

That is the short answer. The longer version involves a choice most questionnaire designers never make consciously: whether the first thing a respondent reads should shape the rest of their answers, or whether the survey should hand them a blank box and get out of the way. For anyone who has watched a stakeholder call free-text answers unusable, or who has run a pilot where a third of the “other (please specify)” bucket was doing the real work, the second option usually produces the better data.

The rest of this piece works through the argument, shows the same survey sequenced two ways, and gives you the cases where the rule should be broken.

Why Open-Ended Questions Belong at the Start of a Survey

Because an opening open-ended question captures the respondent’s own mental model before your questionnaire supplies one, and because respondent attention is a depreciating asset you spend fastest at the front of a survey.

Three mechanisms do the work, and it is worth keeping them apart because they fail in different ways.

Peak attention and willingness. Someone who has just clicked into a survey is briefly curious. By question fifteen they are calculating whether it is worth finishing. An open question is the most expensive question type to answer in effort terms, so asking it when willingness is highest is a matter of arithmetic, not elegance. This is the fatigue-protection argument, and Pew Research’s own survey practice reflects it: general open-ended items about broad topics tend to sit near the beginning, before the specific closed items that need a working mental model.

Uncontaminated recall. The order effects in a questionnaire are asymmetric, and that asymmetry is the whole case. An early closed item does not sit passively on the page. It tells the reader which dimensions matter, supplies vocabulary they will then reuse, and makes a later open answer narrower than it would have been. An early open item constrains nothing. It is the cheapest insurance in survey design.

Response-option discovery. This is the use case nobody in my experience with these discussions puts on the slide, and it is probably the most practical of the three. A hundred unprompted answers tell you, in the respondents’ own words, which reasons actually exist. You then build your closed-ended options from that list instead of from a brainstorm. When the “other (please specify)” bucket later comes back at three percent instead of thirty, you have a closed item that was built on data.

Note what “start” means here in practice. It means early enough to be uncontaminated, not necessarily first. Eligibility screeners, consent language, and a warm-up item that establishes the topic all sit ahead of the first real open question, and that is fine. The rule is that nothing should frame the topic in a way that narrows the first free-text answer.

What Open-Ended Questions Reveal That Closed Questions Miss

A closed-ended question tells you how often something happened inside a list you already wrote down. An open-ended question tells you what the respondent thinks happened, in words you did not supply, and the gap between those two is where most of the useful information hides.

Five things show up in free text that a well-built closed item usually cannot reach:

• Unexpected language. Respondents name things differently than researchers do. A closed item built on the team’s vocabulary can measure a real behaviour and still report zero, because the option list used words nobody in the sample would use.
• Unprompted benefits. Ask “what did you like most about your last purchase” and you get the benefit the respondent actually thought about, not the one on the feature list. If you ask “rate the importance of durability and portability” first, you have already told them durability and portability are the candidates.
• Emotional reaction. Frustration, embarrassment, relief, the small daily irritations that never make it into a satisfaction scale. Emotion is the most common reason a free-text answer gets quoted in a readout and the least common reason a closed item is remembered.
• The respondent’s own framing. Whether someone describes a product as expensive, as not worth it, or as fine for what it is tells you three different things about how they read your pricing. The label is the finding.
• Conditional behaviour. “I would use it if it worked offline” is not a low score. A closed scale has nowhere to put that answer, so it gets averaged away.

The classic version of this is a purchase follow-up survey. Ask “why did you choose this option” as a closed item with five reasons, and the results tell you the distribution across your five reasons. Ask it open first, read three hundred answers, and you will frequently find two reasons you never listed taking half the volume, plus a segment that describes the choice in a way your whole segmentation ignores.

Dimension	Open-ended question	Closed-ended question
Data type	Unstructured qualitative text, codeable into themes	Counts, means, shares, distributions
Respondent effort	High, and unpredictable	Low, usually a tap or a slider
Analysis cost	Manual coding, hours to days	Mostly automatic once cleaned
Main bias risk	Vagueness, one-word answers, social desirability in open text	Order effects, acquiescence, forced categories
Comparability over time	Weak without a coding frame	Strong
Best placement	Early, after screeners	Middle block, once options are evidence-based

Neither form is the good one. The open question is where you find out what you did not know, and the closed question is how you find out how much of it there is. A survey with only closed items reports on a world the researcher built; a survey with only open items leaves you with a folder of quotes and no denominator.

How Question Order Changes Survey Responses

Yes, order matters, and the mechanism is worth naming precisely: earlier questions create context, memory cues, and an answer frame that later respondents reuse whether they mean to or not.

Ask someone to rate delivery speed, then ask why they bought, and the second answer arrives pre-shaped by the first. Ratings set a register. When the survey has just established that this respondent found the experience excellent, an open “what could be better” answer tends to soften. When the same respondent rated the experience poor ten questions earlier, the free text gets longer and blunter.

Researchers see the same drift in closed items without noticing it. Ask four satisfaction items in sequence and the responses correlate heavily, not because the underlying construct is one-dimensional but because respondents learn what the study wants and start answering consistently. That consistency gets reported as a finding. It is partly an artefact of the sequence.

The two named effects are the primacy effect, where early items are remembered more strongly, and the recency effect, where late items are. Both are real, and they are why the standard advice sends the hardest content into the middle and the safest content to the two ends. A broad open warm-up at the top primes nothing beyond “tell me about this in your own words”, and a closing comment box at the bottom gives late-recency respondents a chance to add what the closed items missed.

The last third of any survey is the weakest data you will collect. Respondent fatigue sets in, satisficing kicks in, and satisficing with an open question produces four words where you asked for a paragraph. That is why “the questionnaire is just too long” is the single most common comment in pilot feedback, and why the same design advice that puts open items first in a five-minute survey fails in a fifteen-minute one.

Why Open Ended Questions Belong at the Start of a Survey

The practical version of the rule: give the respondent a chance to define the issue in their own words, so the survey does not quietly supply the categories you will later analyse it through.

That logic is not equally strong in every kind of study, and practitioners argue about it constantly. In exploratory research, where the whole point is to find out what nobody knew to ask about, unprompted answers at the top are close to non-negotiable. The instrument is supposed to be under-specified. In confirmatory research, where you are testing a defined hypothesis against a comparison group and the scale has been validated, the priority flips: the risk of a respondent improvising categories is higher than the value of a fresh one, and the instrument may be fixed by a prior study or a comparison requirement.

So the honest statement is not “open questions always go first”. It is: when you do not yet know the categories, asking for them first costs you nothing and buying the categories yourself costs you a wave. A pilot of twenty open answers is cheaper than a wave of six hundred responses analysed against option lists that were wrong.

One more asymmetry is worth holding onto. An early open question contaminates a later closed item only mildly, because the closed item has its own options and its own answer frame. An early closed item contaminates a later open item severely, because the words are now sitting in the respondent’s head. The damage is not symmetric, which is why the sequencing recommendation exists in the direction it does.

When Open-Ended Questions Should Not Come First

Break the rule when panel logistics, comparability, or channel constraints make a contaminated first answer cheaper than a missing one. These are the cases I would defend in a design review.

Screener-heavy panels and paid qualification. When the panel profile defines your quotas and a respondent’s eligibility depends on it, qualification items go first regardless of what is methodologically elegant. A researcher in r/UXResearch described exactly this tension: the rules that come with a recruited panel and the rules that come with good sequencing pull in opposite directions, and the panel contract wins. An open question after a screener is still early enough to be useful.

Standardized tracking studies. If a metric has to stay comparable across waves, years, or countries, the order is part of the instrument. Changing where the open item sits breaks the series, and a break in a trend line is usually more expensive than the analytic value of the open text. Keep the sequence and accept the contamination, or run the open item as a separate pulse.

Priming-sensitive topics. Anything involving health, money, body image, or political behaviour, where reading the first few items can change how a respondent answers the rest, and where the first question should not be a free-text prompt about the sensitive subject. Lead with a short neutral scale, then open up later.

Mobile-only funnels. Typing on a phone is a different act from typing on a laptop, and in-product surveys usually collect three or four words per open field. Put a large open box in an in-app mobile survey and you will get a small answer from a tired user, at the exact moment you were counting on their willingness. On mobile, one closed item early, then the open item, usually wins.

Fixed validated instruments. If the questionnaire is a validated scale, the sequence is part of the validation. Adding an open item at the front is a change to the instrument, not a design improvement, and it should be described that way in your write-up.

Here is the decision rule I would use: if you are still deciding what to ask, open first. If you already know exactly what you are asking and how the answers will be compared, put the open item late, ask one, and protect the run.

A Strong Survey Opening: Four Example Question Sets

Four openers for four different research goals, each with the follow-up prompts that pull the useful detail out of a short answer.

1. Recent purchase motivation. Opener: “Think about the last time you bought a [category]. What made you choose it over the other options you looked at?” Follow-ups: “Was there anything that almost stopped you?” and “What, if anything, did you give up to get it?” The insight: the trade-off people made and the objection that nearly killed the sale, neither of which shows up on a features list.

2. Brand association. Opener: “When you think of [brand], what comes to mind first?” Follow-ups: “Is that a good thing or a bad thing?” and “Where do you think that impression came from?” The insight: the respondent’s own framing of the brand, which is the material for repositioning work and almost never matches the internal description.

3. Unmet customer need. Opener: “What is the thing about [job or service] that annoys you most, even if it is small?” Follow-ups: “How often does that happen?” and “What have you tried so far to fix it?” The insight: unprompted pain points in the customer’s own language, with a severity estimate attached. The small-annoyance framing is what keeps the answers concrete.

4. Advertising response. Opener: “What was going through your head when you first saw this ad?” Follow-ups: “Did it make you want to know more, or less?” and “Did anything in it remind you of something else?” The insight: whether the ad was even noticed as advertising, which is a different and more useful finding than a relevance score.

Each opener asks about a specific, recent, completed event. That is doing most of the work. “What do you think of our service?” is answered with a compliment or a complaint; “What was going through your head when you first saw this ad?” is answered with a story.

How to Write an Open-Ended Survey Question

Name the exact thing you want to hear about, anchor it to a recent event, and keep the prompt short enough that reading it feels like less work than skipping it.

Starter words that reliably produce content: what (most common, asks for the thing itself), how (process and experience), why (motivation, and the one most likely to produce a guess), describe (invites detail without a judgement), explain (asks for reasoning, works well as a second prompt), and tell me about (warm, best with existing customers rather than prospects).

The four upgrades that change the response rate of the field itself:

• Anchor to a specific event. “What factors influenced your decision?” becomes “What was the main reason you chose this option over the others?” Event-anchored prompts get sentences. Abstract ones get adjectives.
• Ask one thing. “What was good and bad about the delivery experience?” is a double-barrelled question, and half the answer is lost. Ask about the good part, then the bad part, separately.
• Drop the leading noun. “How satisfied are you with our fast and reliable delivery?” plants two words the respondent will hand back to you. Neutral prompts like “How was the delivery?” do not.
• Show the target. Prompt-specific instructions under the box, and a length cue only if you genuinely need length. “One or two sentences” produces more usable text than “please be detailed”.

For numeric topics, decide whether you want a vague or a precise open answer, because they are different instruments. A vague quantifier, “how many people work in your household?”, produces the modal or average number. A numeric open item, “how many people work in your household? __”, produces a distribution you can analyse. Research comparing the two, including a 2014 study by Al Baghal in the Social Research Methodology journal on vague versus numeric open-ended questions, found the forms behave differently enough that the choice has to be deliberate rather than accidental.

One honest warning about open text: it is not automatically more honest. Respondents write longer and more careful in a free-text box than they click in a scale, which means social desirability bias can show up as polished text rather than as a middle rating. Researchers on r/AskAcademia and ResearchGate have argued for years about whether a mixed open and closed instrument still counts as quantitative, and the honest position is that the labels describe the analysis, not the instrument.

How to Analyze Spontaneous Answers Without Losing Meaning

Read a sample first, build a code frame from what is actually there, then code against the frame. The order matters: coding first and reading afterwards is how you end up with a framework that describes the questionnaire instead of the respondents.

A workable sequence for a single open item:

1. Read 30 to 50 answers with no categories in mind. Jot down every distinct reason people give, in their wording.
2. Group them into themes. Aim for six to twelve codes for a typical item. If you have twenty-four, the codes are too fine and two will always be empty.
3. Test for saturation. Code the next fifty. If new themes are still appearing regularly, keep going; if the last fifty produced almost nothing new, the frame is holding.
4. Have two people code a shared subset. Agreement below roughly 80 percent means the code definitions are vague, not that the respondents were unclear.
5. Report counts and quotes together. Counts tell you how common a theme is. A verbatim quotation tells a stakeholder what the theme feels like, and it is often the quote that changes the decision.

Check three things as you go. Ambiguity: a code that covers several different meanings will inflate its own count, so read ten raw responses inside every large code. Repetition: a handful of very long answers can dominate a theme, so report the share of respondents, not the share of words. Interviewer influence: in moderated work, a probe like “so it was the price, then?” rewrites the next answer, and the probe needs to be logged with the verbatim so the reader knows which parts were elicited.

How many open items to include is a budget question before it is a methods question. One well-asked open item usually outruns three weak ones, because the value comes from saturation rather than volume. For most mixed surveys I would budget one open item in the first third and one targeted follow-up later, then check the coding hours against the sample size before committing to more. If the analysis budget is two days, one item is a good survey; four will be abandoned mid-coding.

There is a fast way to test whether the open item is earning its place: read thirty answers and count the ones that could not have come from your own response options. If that number is close to zero, the question is decoration.

Frequently Asked Questions

Start with one open question, run it on twenty to thirty people, and read the answers before you write a single response option. That is the whole method in a sentence: let the sample tell you the categories, then use the closed items to count them. If you do that, the reason open-ended questions belong at the start of a survey stops being a rule you follow and becomes a step you take.