Which of the following is the first step in the questionnaire design process?

If you want to find out something about a person, you’d usually just ask them. If you want to ask a few questions for a group of people, maybe you’d get together as a group. If you want to do research on the answers, you’d give them a questionnaire.

Questionnaires are a crucial part of research. There are many other tools that are used to find out about how people think, feel, and act, but the act of asking remains central to finding out what people explicitly think.

While questionnaires have likely been used for hundreds of years [1], the first recorded instance arose from the result of the work of Adolphe Quetelet, a French polymath, in 1835. He was interested in applying the same rigorous methodologies applied to natural science as to the humanities. By recording – through questionnaires (well, technically surveys) – the physical characteristics of soldiers, he essentially invented the field of sociology. This all goes to say: questionnaires can be powerful things.

Despite the lengthy and illustrious history of questionnaires, they are still not used necessarily in the right way. There remain various ways to carry out such work, and many pitfalls abound.

Below, we will define what a questionnaire actually is (including what separates it from surveys), and provide a guide to making one in the best possible way.

What are questionnaires?

Questionnaires are a set of written questions designed to gather standardized information about the opinions, preferences, experiences, intentions, and behavior of individuals, and can be devised for the purposes of a scientific study. Traditionally, they have been said to contrast with surveys in the sense that they do not collect mass data for further analysis, however the terms are largely used interchangeably these days (and many research studies also use them together).

While questionnaires provide a comparatively cheap, prompt, and efficient means of obtaining large amounts of information, questionnaire design is a multistage process that requires attention to a number of aspects at the same time to gather the information you seek. Why exactly is that?

Depending on the kind of information you aim to acquire, questions need to be asked in varying degrees of detail and in specific ways.

Given the same topic, it’s rather likely that different researchers will come up with different questionnaires that vary widely in their choice of questions, a line of questioning, use of open-ended questions, and length.

Question everything – what makes a good questionnaire?

Basically, well-designed questionnaires are highly structured to allow the same types of information to be collected from a large number of respondents in the same way and for data to be analyzed quantitatively.

Now how should you go about it? Planning and preparation are key. Although questionnaire design can seem simple at the surface, there are several components you want to make sure you get right. Before you know how to exactly phrase your questions, you need to define the goals and aims of your research, understand who you’ll be talking to, and design everything accordingly. Below, we go through exactly how to do this.

Six steps to good questionnaire design

#1: Identify your research aims and the goal of your questionnaire

What kind of information do you want to gather with your questionnaire? What is your main objective?

Ideally, there are already existing questionnaires that have been validated by published research that you can use (or maybe just to borrow a couple of ideas from). This can occur frequently within psychological research, as there is a broad range of research being carried out in a variety of different fields.

While this can be quite common, it’s not always the case. It might, for example, be rather difficult to find or reuse existing questionnaires for commercial applications. In this case, you might still be able to draw inspiration from pre-existing research, although greater care in the following steps will likely be needed.

#2: Define your target respondents

Clearly, you can’t test everyone – it’s rather plausible that there have to be certain restrictions with respect to the target audience of your questionnaire. The selection of groups is a key factor for maximizing the robustness of your study.

Another aspect to consider is whether you want to run multiple questionnaire sessions over a longer period of time with a single group (longitudinal design), or if you want to present your questionnaire once to two or more groups (cross-sectional design).

While the former allows you to analyze how the questionnaire results of the group change over time, the latter delivers insights into differences among groups.

#3: Develop questions

Smart questions are the cornerstone of every questionnaire. To make them work, they have to be phrased in a way that prevents any misunderstandings or ambiguities.

It’s often a lost cause trying to analyze data from a questionnaire where people have mixed things up, selected incorrect answers or haven’t been able to read or understand the questions at all.

It makes a significant difference whether you want to hand a questionnaire to children, adults, or maybe even elderly participants. It’s important to consider the cognitive, attentional, and sensory competencies of your target group – handing out long questionnaires with a huge amount of questions in small letter print and complicated phrasing might be too taxing for many participant groups.

Additionally, remember to avoid jargon or technical language – the text needs to be fully understood by anyone completing the questionnaire.

#4: Choose your question type

There’s a wide variety in how to phrase questions. In explorative questionnaires, you will find mainly open questions, where participants can fill in any answer (this makes sense whenever you try to gain an understanding of the topics associated with your research question).

By contrast, quantitative questionnaires primarily include closed-questions, which have been predefined by the researcher either in form of multiple choice answers or rating scales (such as the Likert scale).

Here’s one example:

Open question:

“What did you like about the webinar?”

Closed question:

“The webinar was useful.”

[  ] Strongly agree

[  ] Agree

[  ] Cannot decide

[  ] Disagree

[  ] Strongly disagree

As is usually the case, both types of questions have benefits and drawbacks that are worth considering in order to come up with a solid questionnaire design that does the trick for you.

Besides open and closed-format questions, there are several other types of questions that you can use in your questionnaire.

#5:  Design question sequence and overall layout

After optimizing each question separately it is time to improve the overall flow and layout of the questionnaire.

Are there transitions from one question to the next? Are follow-up questions placed correctly? Are skip-rules implemented (if needed) so that participants can skip questions that do not apply to them?

#6: Run a pilot

This stage is crucial for evaluation and optimization purposes. Any questionnaire should be handed to a representative sample of your target audience before you go further with it.

During piloting, you can identify issues in readability and understanding, in phrasing and overall arrangement. It could be helpful to discuss the questionnaire with pilot participants to better understand their experience. Also, keep in mind to evaluate your pilot data statistically to make sure that the analytic procedures of interest truly can be applied to the data.

I hope this post helps you set out your questionnaire or survey design. If you’d like to learn more about the fundamentals of experimental design, then download our free guide below.

References

[1] Gault, R. (1907). A History of the Questionnaire Method of Research in Psychology. The Pedagogical Seminary, 14(3), 366-383. doi: 10.1080/08919402.1907.10532551

What is the first step of questionnaire design?

Which are the first and last steps in questionnaire design process?

What are the five steps in designing a questionnaire?

What are steps of designing good questionnaire?