When is my online survey representative?

‘Representative poll reveals…’ Headlines like that sound like established fact, and that is exactly how they are meant to sound. Behind the word, though, sits a concrete methodological promise, and plenty of surveys simply cannot keep it.

A survey is representative when its sample realistically mirrors the make-up of a target group defined in advance, when every person in that group had a chance of being selected, and when the selection followed a recognised random or quota method. The number of responses does not decide it. By the end, you will know whether your own project can make that claim and how to describe your results honestly.

📌 Key points at a glance

• Representative means the sample mirrors the target group’s make-up.
• A large number of responses never makes a survey representative.
• The British Polling Council makes its members disclose their sampling.
• Online surveys shared by link rarely meet the random selection test.
• 385 responses give a 5-point margin at 95 per cent confidence.

What does ‘representative’ mean in a survey?

A representative survey is a survey whose sample reflects a clearly defined target group in the characteristics that matter for the research question, closely enough for the results to be carried over to that whole group. The term is not a verdict on quality. It describes how far a finding reaches.

Every question about representativeness involves two groups: the group you want to learn about (the population) and the group you actually ask (the sample). If a poll of adults in Great Britain finds that 75 per cent support a policy, the claim is that roughly the same majority exists among all adults in Great Britain. That jump from the small group to the large one is the whole point.

In the UK, the British Polling Council, the association of polling organisations that publish their results, does not certify anyone’s methods as representative. Its rules state that membership implies no approval of a pollster’s methods. What it demands instead is disclosure. Germany has gone a step further: in May 2026 its four research associations ADM, ASI, BVM and DGOF agreed a joint definition, under which a sample counts as representative if recognised selection, weighting and analysis methods allow it to reflect the defined population adequately.

Is ‘representative’ a scientific term?

‘Representative’ has no single agreed definition in survey research, and many methodologists avoid the word altogether. In the first of a series of papers in the International Statistical Review, published in 1979, the statisticians William Kruskal and Frederick Mosteller showed that outside science the word was used in five main senses: as general praise for the data, as the absence of selective forces, as a miniature of the population, as a typical or ideal case and as coverage of the whole population. Their advice was to use the expression with great caution, because a more specific description of the sampling usually says more.

Survey statisticians prefer to talk about bias. An estimate is unbiased if, were you to repeat the same survey many times, the average of all the results would land on the true value for the target group. For your own work the distinction is more than hair-splitting, because it explains why a properly drawn sample of 500 people is worth more than 50,000 self-selected clicks.

In practice, the four criteria in the next section give you a workable everyday definition, while a methodologically careful dissertation is better served by asking about systematic bias. Both lead to the same test: who had any chance of ending up in the sample, and who did not?

When is a survey representative? The four criteria

A survey is representative once four conditions are met at the same time: the sample matches the structure of the target group, every person in that group had a chance of selection, participants were chosen by a scientific random or quota method, and the target group the result applies to is stated explicitly. German market and social research associations set out these four conditions in a joint paper in May 2026. The British Polling Council approaches the same questions from the other side: instead of setting criteria, it requires its members to publish the population represented, the method of interviewing and a full description of the sampling procedure, so that anyone can check the claim.

1. Matching structure. The sample realistically reflects the make-up of the target group in the characteristics that matter for your question.
2. A chance for everyone. Every person in the target group could, in principle, be selected. No subgroup is ruled out from the start.
3. A scientific selection method. Participants are chosen through a systematic random procedure or through quota sampling on characteristics fixed in advance, such as age, sex or education.
4. A named target group. Anyone who claims representativeness says which group it applies to. ‘Representative’ without a reference group tells you nothing.

In practice, a chance for everyone is the hardest of the four. A survey that runs only on a single website reaches only people who visit that website. Everyone else has no way into the sample, and no calculation afterwards can repair that gap.

What the gold standard looks like in Britain became clear after the 2015 general election. The official inquiry into the polls compared them with the British Election Study and British Social Attitudes, which at the time both drew a multi-stage, stratified random sample of addresses from the Postcode Address File (PAF). At each address, one eligible adult was selected at random. Interviewers called back several times, at different times of day and on different days of the week, and nobody who could not be reached was replaced by a more convenient respondent. That rigour costs a great deal of time and money, which is why such studies are the exception.

How many participants does a representative survey need?

A representative survey of a very large target group needs about 385 usable responses if you accept a margin of error of 5 percentage points at 95 per cent confidence. For small groups the required share rises steeply: a group of 100 people needs about 80 responses. Neither figure is a magic number. Both come out of a calculation with three dials.

The first dial is the margin of error, the range by which your result may differ from the true value. The second is the confidence level, usually 95 per cent. The third is the size of the population, and it only makes a big difference for small groups. The standard formula is n = z² · p · (1 − p) / e², with a finite population correction for smaller populations. If you would rather just enter the three values, you can calculate your sample size with our calculator.

Our own calculation using the standard formula: 95 per cent confidence, 5-point margin of error, expected proportion of 50 per cent, rounded up. As of August 2026.

At first glance the relationship looks the wrong way round: the smaller your target group, the larger the share of it you need to hear from. For the 100 members of a local running club you need 80 responses; for a million people, just 385. It works the other way too. Tighten the margin of error from 5 points to 1 and you need 9,604 responses instead of 385.

Why online surveys are almost never representative

Online surveys almost always fail the ‘chance for everyone’ test, because participants decide for themselves whether to take part. A survey shared as a link reaches only the people who see the link, and it is filled in mainly by those who already care about the topic.

Whether you run your survey in empirio.ai, an online survey tool from Germany, in Google Forms, Google’s free form builder, or in SurveyMonkey makes no difference here. The bias does not come from the software. It comes from how you reach the target group. Professional online access panels do not fully solve the problem either, because people sign up to them voluntarily and are then recruited from that pool. People who volunteer for surveys differ from people who do not, often in ways nobody has measured.

One group is missing from internet-only surveys by design: people who do not use the internet. According to the Office for National Statistics, 6.3 per cent of adults in the UK had never used the internet in 2020, and only 54 per cent of those aged 75 and over were recent internet users, against 99 per cent of 16 to 44-year-olds. The group is small, but it differs systematically from everyone else. Weighting the data afterwards only helps up to a point, because it can only balance characteristics whose distribution in the target group is known.

There is one exception, and it covers most projects in clubs, courses and teams: if you have a complete contact list for your target group, you can invite everyone. Every member of a society, every employee of a company, every student on a module. You are then close to a full census, and the question of representativeness no longer arises in the same form. What matters for a survey like that is covered in our tips for creating a better online survey.

Not representative but still useful: what belongs in your dissertation

A non-representative survey is perfectly acceptable in an undergraduate or master’s dissertation, as long as you state its limits openly. Markers assess your methodological approach and how carefully you interpret your results, not whether they can be generalised to the whole population. What counts in the end are your university’s regulations and your supervisor’s guidance.

In research practice, non-representative samples are the norm. Sleep research needs people willing to spend several nights in a lab, and that group is never a mirror image of the population. The findings are still medically valuable. What matters is not whether a study is representative, but whether it says honestly who its conclusions apply to.

Four details therefore belong in your methodology chapter, before you report a single result: how you recruited participants, how many usable responses were left, which groups are missing as a result and which group your conclusions apply to. A workable wording for the limitations section runs along these lines: The sample was recruited through calls for participants in subject-specific groups and was not randomly drawn. The results apply to the students surveyed and cannot be generalised to all students.

How to set up a non-representative study in a methodologically sound way is explained in detail in our guide to the empirical dissertation. Which quality criteria matter besides generalisability is covered in the article on objectivity, reliability and validity.

Not being representative is no flaw. Claiming to be representative when you are not is.

How to spot a representative survey: the transparency check

You recognise a survey that deserves the label ‘representative’ by the information it gives about itself, not by the word. In the UK, the rules of disclosure of the British Polling Council set out what member pollsters must publish alongside every social or political poll that enters the public domain. You need no specialist knowledge to check these details yourself.

Seven things a published poll should tell you about itself

• Who commissioned the survey
• When the interviews took place
• Method: in person, by telephone or online
• Population represented, such as all adults or voters
• Sample size and geographic coverage
• Complete wording of the questions asked
• Web address of the full data tables

The first five are mandatory for BPC members, the last two are expected whenever practical, and within two working days the pollster must also post a full description of its sampling and weighting on its own website. If several of these details are missing, be sceptical. How much uncertainty remains even in a well-run poll is spelled out in the same rules: with voting intention polls for an election, members have to publish a statement that, on the historical record of the polls at recent general elections, there is a 9 in 10 chance that a party’s true support lies within 4 points of the estimate, and a 2 in 3 chance that it lies within 2 points. That range is noticeably wider than pure sampling error, because real polls face more sources of error than chance alone.

Our reading tip: The British Polling Council’s quick guide for journalists to the use and reporting of opinion polls is free, written for non-specialists and built around five questions to ask of any poll, from how and when it was conducted to which questions were asked. According to the BPC, it takes five minutes to read.

Common mistakes with representative surveys

Most mistakes around representativeness happen not in the maths but while the survey is being planned. Once you know them, you will spot them in other people’s studies too, and that is the difference between reading numbers and judging them.

Four patterns turn up in dissertations and press releases alike: confusing a large number of responses with representativeness, never naming the population, trusting weighting to fix the sample and overlooking where and when people were asked. You can avoid the first three before you collect any data. The fourth can only be handled by describing it honestly afterwards.

Mistake 1: Confusing a large number of responses with representativeness

The number of respondents never replaces the selection method. A striking example is the European Commission’s public consultation on clock changes in summer 2018: 4.6 million responses from all 28 member states, with 84 per cent in favour of ending the twice-yearly change. Participation, however, was extremely uneven. According to the European Commission (2018), 3.79 per cent of the population took part in Germany, 0.04 per cent in Italy and just 0.02 per cent in the United Kingdom. Despite millions of responses, the result could not be carried over to the EU population, and certainly not to the British public.

Mistake 2: Never naming the population

Without a named target group, representativeness has no content. ‘Representative of Great Britain’ means something entirely different from ‘representative of first-year undergraduates’, and each claim needs a different sample. So write your population down before you draft the first question, and name it again in every results section.

Mistake 3: Trusting weighting to fix the sample

Weighting does not repair a flawed selection. You can only adjust for characteristics whose distribution in the target group is known, such as age, sex or education. The effect of self-selection stays invisible and therefore uncorrectable. The inquiry into the 2015 British general election opinion polls, set up by the British Polling Council and the Market Research Society, found that the main cause of the polling miss was unrepresentative samples. They contained too many Labour supporters and too few Conservative supporters, and the statistical adjustments applied to the raw data did little to correct this.

Mistake 4: Overlooking where and when people were asked

Where and when you ask helps decide who you reach at all. A survey on the high street on a Tuesday morning misses everyone who is at work or at school at that time. The same logic applies online: a call for participants posted in a subject group in the evening reaches a different crowd from an invitation sent by post.

Conclusion

Representativeness is not a badge you award yourself. It is a property of how the sample was drawn, and you have to be able to demonstrate it. For the vast majority of projects at university, in clubs or in small teams, it is neither achievable nor necessary. What counts is a clearly named target group, a selection method others can follow and an honest statement of what your figures cannot tell you. None of this costs anything, and together it makes your work more robust than any extra response would.

Where to go next

• Want to know how to draw a sample properly? Sample: method and size
• Planning the questionnaire for your study? Questionnaire design
• Unsure which method suits your question? Qualitative or quantitative
• Looking for an overview of the research process? Fundamentals of empirical research

Need a starting point for your own survey?

Ready-made questionnaire templates for university, clubs and teams save you from staring at a blank page. Browse the survey templates