Methods
The selection of research and testing methods is always planned case-specifically. The designing of the data collection questionnaire is a multi-stage and iterative process, where the draft questionnaire is tested preferably several times and with several methods. The work is carried out in close cooperation with an expert on the topic of the survey.
Our procedure is essentially based on the cognitive model of the question-answer process.
Cognitive model of the question-answer process
From the perspective of cognitive psychology, the person answering questions in the survey carries out various cognitive tasks. Thinking is a flexible process, so the formulation of an answer consists of going back and forth between different phases. According to the general model, the respondent's first task is to understand and interpret the question. As the second task, the respondent recalls from memory the relevant information needed for responding after interpretation. Thirdly, the respondent formulates and adapts the answer to the provided response alternatives. Finally, the respondent assesses the effects or suitability of the information given before responding and decides whether to answer truthfully or not.
For enterprise respondents, tasks related to organisational factors are also included in responding. The tasks relate to obtaining information about the enterprise’s systems, organising the response task and the enterprise’s internal arrangements in order to check and release reliable information.
Cognitive interviews
Cognitive interviews examine and test draft data collection questionnaires or individual questions and concepts. The usability of the questionnaire is observed alongside the testing of its contents during the interview. Interviews are individual interviews during which respondents are encouraged to think aloud while answering the questions. Additional questions (probes) are asked of the interviewee either (a) simultaneously when answering the question or (b) retrospectively after answering the question or after the entire form has been completed. The method can also be applied in web probing. In web probing spontaneous probes are not possible, because the probe questions need to be planned in advance.
The following can be ascertained with cognitive interviews:
- What does the respondent think is meant by the question?
- What do individual terms and concepts mean to the respondent?
- What kind of information recall is required from the respondent?
- What kind of answering strategy does the respondent use?
- Is the respondent willing to put enough effort into answering accurately and completely?
Expert panel
The data collection questionnaire assessment carried out by the expert panel examines the questionnaire systematically question by question and summarises the points that are problematic. The assessment is based on a classification based on a general model of the question-answer process. The expert panel comprises around five persons who are specialists in behavioural sciences, survey methodology or the questionnaire topic. Based on the questionnaire assessment, problematic questions are corrected before the data collection questionnaire is implemented and thus the quality of data can be improved.
Focus group interview
The focus group interview is a discussion facilitated by the interviewer to collect information on the topic under interest. The aim is to find out what viewpoints, opinions and attitudes are involved in the studied topic. The discussions can also be used to evaluate data collection tools or methods. Focus group interviews help researchers learn the vocabulary and ways of thinking of interviewees. They can also provide insights into the interpretation of quantitative results.
Focus group interviews can be used together with quantitative data (so-called triangulation). Then the data collected with different research methods can enhance the reliability of the results and provide a more versatile view of the topic under examination. The method falls somewhere between participatory observation and observing the natural group situation and a structured individual interview.
The focus group interview includes interaction within the group. It is thus closer to everyday life where opinions, attitudes and habits are formed, than individual interviews.