What does Objectivity Mean? Definition and Example

Two graders read the same short essay. One gives it an A, the other a C. The essay did not change between the two readings.

Objectivity is one of the three main quality criteria in quantitative research and gets decided long before any analysis. Objectivity means that a result does not depend on who administers the instrument, who scores the responses and who interprets the values. Objectivity is secured through rules fixed before data collection begins: written instructions, a coding scheme for open-ended responses and interpretation thresholds that hold for every user. By the end you can say which of the three parts your own study covers.

📌 The key points at a glance

• Objectivity means the result does not depend on the researcher.
• Administration, scoring and interpretation are secured separately.
• Standardization is the strongest single lever for objectivity.
• No coefficient exists for objectivity as a whole.
• US testing standards treat the same idea under other headings.

Objectivity means the result does not depend on the person

Objectivity means that a test or questionnaire measures the characteristic it is meant to measure independently of the person administering it and the person scoring it, and that clear, user-independent rules exist for interpreting the results.

An older name for the same criterion is intersubjective comparability. The underlying question never changes: do several people working separately with the same instrument arrive at the same result? Where the answer is no, the study is partly measuring the person who runs it. The other two main criteria are reliability and validity.

A confusion hides inside the everyday word. Objectivity does not require researchers to hold no opinion about their topic. What it requires is that the opinion cannot move the result, because the procedure gives it no room. Someone with a hunch who still uses a standardized questionnaire is working objectively. Someone entirely open-minded who sorts open-ended responses by instinct is not.

Administration, scoring, interpretation: the three parts of objectivity

Objectivity breaks into three parts that follow the sequence of a study. The division traces back to Gustav Lienert and Ulrich Raatz and is still taught in that form.

The split has practical consequences. Each part can be met or missed on its own, and in a thesis each one is argued separately. A survey can be flawless in administration and still leave every judgment call open at the scoring stage.

Administration objectivity: what the instructions have to contain

Administration objectivity means the result does not depend on who runs the data collection. In practice, everything a respondent sees is written down in advance. That covers the opening text, the explanation of the response scale, the order of the items and the note about what happens to the data. Leave any of these to the moment and you produce exactly the difference between two sessions that the criterion exists to rule out.

Scoring objectivity: open-ended items need a coding scheme

Scoring objectivity means two people derive the same values from the same responses. With closed questions the problem barely comes up, since each option maps to a fixed numerical value. Once open-ended items appear in the questionnaire, a coding scheme has to exist before the first response is read: the categories, one definition each and one example each. Whether the scheme holds up shows only when two people apply it independently.

Interpretation objectivity: why norms are missing and what helps instead

Interpretation objectivity means different specialists draw the same conclusion from the same number. Norms from a standardization sample would be the precondition, and a self-built survey has none. Your own thresholds take their place, set before you look at the data and stated in the methods section. A mean of 3.4 on a Likert scale is neither good nor bad by itself.

How to build objectivity into your own survey

Objectivity is not added later, it is built in before the study starts. Each of the three parts has one measure a student project can realistically carry out, and one warning sign that it is missing.

The second row carries the most weight, because that is where the judgment calls live. A coding scheme written before the first response is read removes the most common line of attack at the defense. Anyone running a quantitative study without any open-ended items avoids the problem entirely and gives up the reasoning behind the numbers.

Objectivity in US testing practice, and how it gets measured

Objectivity is a heading from German-language methodology textbooks, where it stands beside reliability and validity as the third main criterion. US testing literature covers the same ground under different labels, which is worth knowing before the word appears in a methods section.

The Standards for Educational and Psychological Testing, published jointly by AERA, APA and NCME and available open access, build their foundations on validity, reliability and fairness. Objectivity is not one of them. What the German term covers is distributed across a separate operational chapter on test administration, scoring, reporting and interpretation, which is the same three-way split under another name.

What standardized administration and scoring look like in practice

NAEP, the federal assessment run by the National Center for Education Statistics, shows the machinery in the open. Field staff follow scripts to guide students through the sections and time the administration, which is administration objectivity in everything but name. On the scoring side NAEP develops explicit scoring guides, trains raters against them, and routes a share of already scored responses back to a second scorer to check consistency.

There is no coefficient for objectivity itself

No single number expresses objectivity, unlike reliability with its value between 0 and 1. Agreement, on the other hand, can be quantified: where two people score the same open-ended responses with the same scheme, their level of agreement is expressed as a coefficient, usually Cohen’s kappa. A boundary runs through here that many guides blur. The coding scheme secures objectivity, while the measured agreement is reported as inter-rater reliability and belongs to reliability.

Common mistakes with objectivity

Three mistakes show up again and again in theses, and none of them is statistical. All three happen in the text about the method rather than in the data.

The cause is the same every time: objectivity gets claimed for something it was never built to do. Once you know the three confusions, finding them in your own methods section takes minutes.

Confusing objectivity with the neutrality of the researcher

Objectivity is a property of the procedure, not a state of mind. Sentences such as “the study was objective because I approached the topic without preconceptions” therefore prove nothing. Objectivity is evidenced through the rules that limited the room for judgment: the questionnaire, the coding scheme and the thresholds set in advance. Holding a hunch is no flaw but the normal case in an empirical thesis.

Asserting objectivity in the methods section without evidencing it

A sentence such as “objectivity is given” appears in a great many theses and contributes nothing. Graded is not the claim but the description of the procedure. One paragraph does the job: what every respondent saw, by which rules responses became values, and who set those rules.

Reading high objectivity as a sign of a good result

An objective study can miss the topic completely. Objectivity says only that the result does not hang on the people involved, and nothing about whether the items capture the characteristic you mean. That is the job of validity. A standardized questionnaire with the wrong items reliably produces the same wrong picture.

Conclusion

Objectivity asks for the least arithmetic of the three criteria and gets decided earliest. Nothing about it improves after data collection, because the rules had to be in force beforehand. An hour spent on instructions and a coding scheme writes that part of the methods section for you.

Objectivity says the result does not hang on you. Whether you measured the right thing, it does not say.

Where to go next

• Want all three criteria side by side? Objectivity, reliability and validity compared
• Need the definition of the next term? What does reliability mean?
• Looking for the full set of criteria? Reliability and validity in research: quality criteria
• Want the whole process in view? Fundamentals of empirical research

Our reading tip: American Educational Research Association, American Psychological Association and National Council on Measurement in Education (2014): Standards for Educational and Psychological Testing. The volume is open access, and the chapter on test administration, scoring, reporting and interpretation is the closest US counterpart to the three parts described here.

Want every respondent to see the same questionnaire in the same order?

With empirio.ai, an online survey tool from Germany, you set the instructions, the item order and the response scales once and then share the survey by link. Administration objectivity is largely covered that way; scoring and interpretation stay your job.

Create a survey for free