Kill the Frustration: Speed Up Questionnaire Sign off

Often Researchers leave signing off on a Questionnaires design to their client's (and hate the desperate last minute changes made by Top Management.) There are two key reasons, the first is the project really doesn't have the buy in until the last minute, and two, the executives have no idea what a good questionnaire is. To avoid this we've developed a primer to assist those signing off on last minute 'corrections'. 
This has been useful to so many clients.

Preparing to Review a Questionnaire

Before starting to design questions, clearly articulate what problem or need is to be addressed by the questions. Review with stakeholders why you're doing the evaluation and what you hope to accomplish by it. This provides focus on what information you need to be looking at and ultimately, on what questions should be used. 

The following will guide your review process and provide the tools you need to contribute quickly/effectively.

Types of Information Collected by Questions

Questions are geared to find out what people know, did, feel and think.

1. To find out what information they know, ask them to describe something, e.g., "Please describe ..."
2. To find out what they feel, ask them, e.g., "How do you feel about ...?" or "How did you feel when ...?"
3. To find out what they think, ask them for their opinion on something, e.g., "How could the .. be improved?"
4. To find out what they did, ask them to describe an activity they did.

The Two Types of Questions

1. Open-ended: No options are provided for the respondent to answer the question. They must think of their own response and describe it in their own words. If respondents have and take the time to reflect on answers to the question, you can get more meaningful information than from closed questions. Open responses are more expensive and the responses are generally reduced to categories of responses for measurement (ie. turned into closed responses, see below.)

2. Closed: The respondent is given a set of alternative choices from which he or she can choose to answer the question, i.e., "yes," "no," multiple choice, a rating, ranking, etc. Closed questions can usually be answered quickly, allowing you to get a get a lot of information quickly. However, respondents may rush through the questions and not take enough time to think about their answers, use a long lead in if necessary. Be cautious that your choices allow for the fact their a persons answer may not be include in your predetermined choices.

Questionnaire Introduction

Directions to Respondents

1. Include a brief explanation of the purpose of the questionnaire.
2. Include clear explanation of how to complete the questionnaire.
3. Include an estimation of the time it will take to complete the questionnaire.
4. Note conditions of confidentiality

Questionnaire Style

Considerations

1. Will the respondent be able to answer your question, i.e., do they know the answer?
2. Will respondents want to answer the question, i.e., is it too private or silly?
3. Will the respondent understand the wording, i.e., are you using too cultural-specific or technical words?
4. Are any words so strong that they might influence the respondent to answer a certain way?
5. To ensure you're asking one question at a time, avoid use of the word "and" in your question.
6. Avoid using "not" in your questions if you're having respondents answer "yes" or "no" to a question.
7. If you use multiple choice questions, be sure your choices are mutually exclusive and encompass the total range of answers.

Order of Questions

1. Communicate how many questions to complete, so as to motivate the respondent to concentrate during the entire interview.
2. Aid respondent motivation by starting with fact-based questions and then go on to opinion-based questions, e.g., ask people for demographic information about themselves and then go on to questions about their opinions and perspectives. This gets respondents engaged in the questionnaire and warmed up before more challenging and reflective questions about their opinions.
3. Aim to get respondents' commentary in addition to their ratings, e.g., if the questionnaire ask respondents to choose an answer by circling an answer or provide a rating, ask them to provide commentary that explains their choices

Conclude Pre-Testing

1. Pilot test your questionnaire on a small group of clients or fellow staff. Ask them if the form and questions seemed straightforward. Carefully review the answers on the questionnaires. Does the information answer the evaluation questions or provide what you want to know about the program or its specific services? What else would you like to know?
2. Time the length of the questionnaire, as length can have a huge impact on costs
3. Date and time stamp the questionnaire so as to keep track of all future versions.

You are now ready to review your questionnaire, grade and improve it.

Back to Best Practices Library