11 Tips to Create More Effective Surveys

By: Kayleigh P. Marques
May 1st, 2020 (updated July 23rd, 2024)

The purpose of creating a program survey is to collect the data you need to improve your decision-making, maximize impact on your service recipients, and improve your organizational performance. Therefore, it is important that you follow survey best practices to avoid wasting time and producing irrelevant data.

We’ve developed the following 11 tips to help you maximize your survey response rate and collect useful, unbiased, thorough, and actionable data:

1. Set Survey Objectives Before Designing Your Survey

Setting clear survey objectives is a crucial first step in the survey creation process. Your programs have specific goals, and your surveys should be designed to reflect and measure the achievement of these objectives. Establishing objectives upfront ensures that your survey is purposeful and aligned with your program’s needs, rather than being a collection of arbitrary questions.

Before you begin writing your survey questions, you and your team should determine what your objectives are. Consider the following:

Program Goals: What are your program’s overarching objectives? Clearly defining these goals will help you formulate questions that assess whether these objectives are being met.
Quality Evaluation: How are you going to evaluate the quality of your program? Identifying quality metrics will guide you in crafting questions that gather relevant data to assess performance and effectiveness. For example, you may want to ask your survey respondents about the quality of instructors, facilities, program training materials, marketing tactics, and more.
Desired Changes: What kinds of changes are you hoping to produce, and how will you measure these changes? Understanding the specific outcomes you aim to achieve will help you design questions that capture data on these transformations.
Critical Information: What is the critical information you need to help improve service delivery? Pinpointing essential data points ensures that your survey focuses on gathering actionable insights. Asking yourself this question will help you determine the minimum number of questions you must ask in order to get the information you need.

2. Use Program Performance Measures to Inform Survey Questions

Your surveys should be designed to help you evaluate the impact of your programs. Therefore, you need to determine how you’re going to measure this impact with the use of Performance Measures. Then, you can design your survey questions to collect the necessary data.

All performance measures fall into the following three categories:

How much are you doing? (e.g. How many people are you serving? How many tasks are you performing? How many people have you reached out to and successfully recruited?)
How well are you doing it? (e.g. How well are you delivering your services? Are your instructors certified? Is the program accessible to the maximum number of eligible people)?
Is anyone better off? (e.g. Is your program delivering the intended impact, no impact, or negative impact? Are there any tangible changes in the perception, behaviors, attitudes, and or circumstances of your program participants?)

For example, if you’re running a job training program, one of your “better off” performance measures might be “% of program participants who gain full-time employment within 3 months of program completion.” In this scenario, you should include a survey response related to the participant’s employment status and send follow-up surveys to monitor employment status at the appropriate intervals.

Go here to learn more about creating effective performance measures.

3. Keep Your Survey Questions Simple

Your survey data is critical to your continued success – don’t make survey completion any harder than it needs to be! Avoid using industry-specific jargon as much as possible, and write in a way that most people will understand. Many sources cite 8^th grade as the average reading level for adults in the United States, but you may want to conduct research on your specific community.

Language and culture matter! Make sure the language your survey is composed in is appropriate for the community. Make translations available for diverse program participants. This means using the appropriate spoken language (English, Spanish, etc.) and using the culturally appropriate words and phrases to communicate your ideas.

Finally, you may need to consider whether electronic or paper-based surveys will be effective at all. You may be dealing with literacy issues in your community. If so, be sure to offer alternative solutions for clients who may have difficulty reading and completing surveys on their own.

4. Don’t Ask Too Many Questions

Generally speaking, the longer a survey takes to complete, the fewer survey responses you will receive. For example, Survey Monkey found that survey completion rates are directly correlated with the number of questions. 10 question surveys were found to have an 89% completion rate on average, with the rate reducing by about 2 percentage points for each additional 10 questions.

To get a good response rate, ask only the important questions you need to meet your survey’s objectives, evaluate the quality of your programs, and evaluate impact.

5. Include Relevant Demographics

A lot of social programming is designed to promote equity in the community and close the opportunity gap between people of different races, genders, etc.

To apply an equity lens, data must be disaggregated to enable you to develop strategies that ensure that race and other factors do not predict one’s success while also improving outcomes for all.

If equity is an important part of your work, make sure you collect the appropriate demographic data you need to disaggregate program results and impact.

The following are examples of demographics (not exhaustive):

Age
Gender
Sex
Household Income
Ethnicity
Race
Education
Marital Status
Grade Level
Employment Status

Create Better Surveys and Simplify Your Data Collection.

Schedule a custom, private demo of Clear Impact Compyle.

6. Understand and Utilize the Right Question Types

Creating effective program surveys involves more than just asking questions; it requires knowing the right types of questions to ask to gather the most useful data. Different question types serve different purposes and can provide both qualitative and quantitative insights. Understanding these types will help you design surveys that yield comprehensive and actionable information.

Here are the types of survey questions you can use and when they are most appropriate:

Open-Ended Questions

Open-ended questions allow respondents to provide detailed, unstructured responses. They are ideal for gathering in-depth insights into participants’ thoughts, experiences, and feelings. Use open-ended questions to explore complex issues or to gain qualitative feedback that can reveal nuances not captured by fixed-response options.

- Example: “Can you describe your experience with our after-school program?”

Closed-Ended Questions

Closed-ended questions provide respondents with a set of predefined answers to choose from. These are useful for collecting quantitative data and for situations where you need specific, comparable responses. They are easier to analyze statistically and can quickly give you a sense of trends or patterns.

- Example: “Did you find the program materials useful? (Yes/No)”

Rating Questions

Rating questions ask respondents to evaluate something on a numerical scale, providing a measure of intensity or frequency. These questions are valuable for assessing satisfaction, importance, or likelihood.

- Example: “On a scale of 1 to 5, how satisfied are you with the program?”

Likert Scale Questions

Likert scale questions gauge the level of agreement or disagreement with a statement, typically using a 5- or 7-point scale. They are particularly useful for measuring attitudes, opinions, and behaviors.

- Example: “How strongly do you agree with the following statement: ‘The program has improved my child’s academic performance.’ (Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree)”

Multiple Choice Questions

Multiple-choice questions offer several answer options, allowing respondents to select one or more responses. These questions are versatile and can be used to gather a variety of data, from preferences to behaviors.

- Example: “Which of the following services have you used? (Check all that apply: Tutoring, Counseling, Sports Activities, Arts and Crafts)”

Picture Choice Questions

Picture choice questions enable respondents to select answers using images rather than text. These can be particularly engaging and are useful in surveys for younger audiences or for respondents with varying literacy levels.

- Example: “Which of these images best represents how you feel about our community center? (Show images of different emotions)”

Demographic Questions

Demographic questions collect information about the respondent’s background, such as age, gender, income, and education level. These questions are essential for understanding the characteristics of your survey population and for segmenting data to identify trends among different groups.

- Example: “What is your age group? (Under 18, 18-24, 25-34, 35-44, 45-54, 55-64, 65 and older)”

By understanding and appropriately using these different types of survey questions, you can design surveys that capture a rich mix of qualitative and quantitative data, leading to more comprehensive insights and better decision-making.

7. Avoid Biased Questions

Creating effective surveys requires careful attention to question design to avoid bias, which can distort data and lead to inaccurate conclusions. Furthermore, if survey respondents interpret your biased survey questions as intentionally manipulative or misleading, this may cause them to quit the survey early, leading to higher drop-off rates. It could also have a more permanent impact on trust in your staff and organization. Biased questions can take several forms, including leading questions, loaded questions, poor response choices, and others.

Importance of Avoiding Bias

Avoiding bias in survey questions is crucial for several reasons:

Accurate Data: Bias-free questions ensure that the data collected is accurate and truly reflective of respondents’ opinions and experiences.
Credibility: Surveys with unbiased questions are more credible and can be trusted by stakeholders, including staff, partners, and funders.
Effective Decision-Making: Reliable data allows for informed decision-making, leading to better program improvements and outcomes.
Enhanced Respondent Experience: When respondents feel that their answers are valued and not being manipulated, they are more likely to provide honest and thoughtful feedback.

Here’s a closer look at different types of biased questions, examples of each, and better alternatives to use.

1. Leading Questions

Leading questions prompt or encourage respondents to answer in a specific way by suggesting a particular response.

Biased Question: Don’t you think our service delivery is excellent?

- Yes
- No

Better Alternative: How would you rate our service delivery?

- Very poor
- Poor
- Average
- Good
- Excellent

2. Loaded Questions

Loaded questions contain assumptions that may not be true for all respondents, leading to biased answers.

Biased Question: Why do you prefer our service over other services in the community?

Better Alternative: How would you compare our service to other similar services in the community?

- Much better
- Somewhat better
- About the same
- Somewhat worse
- Much worse

3. Double-Barreled Questions

Double-barreled questions ask about two things at once, making it difficult for respondents to answer accurately.

Biased Question: How satisfied are you with Service A and Service B?

Better Alternative: How satisfied are you with service A?

- Very dissatisfied
- Somewhat dissatisfied
- Neutral
- Somewhat satisfied
- Very satisfied

4. Absolute Questions

Absolute questions force respondents to choose a definitive yes or no, which might not capture the nuances of their opinions.

Biased Question: Do you attend our tutoring programs frequently?

- Yes
- No

Better Alternative: How many tutoring sessions did you attend this year?

- 0
- 1-13
- 13-26
- 26-39
- 39-52

5. Poor Response Choices

Questions with limited or biased response options can prevent respondents from providing accurate feedback.

Biased Question: How well did we deliver our service to you?

- Somewhat well
- Generally well
- Extremely well

Better Alternative: How well did we deliver our service to you?

- Very poorly
- Somewhat poorly
- Not sure / somewhere in the middle
- Somewhat well
- Extremely well

8. Switch from Paper-Based Surveys to a Survey Software System

The traditional method of using paper-based surveys presents numerous challenges that can hinder accurate data collection processes. Paper surveys are prone to errors, such as illegible handwriting, which can lead to data inaccuracies. Additionally, the manual process of distributing, collecting, and inputting data from paper surveys is time-consuming and labor-intensive, often resulting in delayed analysis and reporting. Storage and retrieval of paper surveys also pose logistical issues, as they require physical space and are susceptible to loss or damage.

Switching to a digital survey system can mitigate these challenges and offer several key benefits. Firstly, digital surveys are more convenient for both the survey administrators and participants. They can be distributed and accessed via email, social media, or embedded links on websites, allowing for a broader reach and higher response rates. Digital surveys also eliminate the need for manual data entry, reducing the risk of human error and ensuring data accuracy.

Learn more about our Compyle data collection and survey software >>

9. Implement Real-Time Data Collection and Analysis

Monitoring survey responses as they come in allows organizations to gain immediate insights and act swiftly based on the data. This approach stands in stark contrast to typical methods, where analysis is only possible once all survey data has been collected, leading to delays and potentially missed opportunities.

One significant benefit of real-time data analysis is the ability to identify and address issues as they arise. For instance, if a nonprofit organization is running a survey to evaluate the effectiveness of a new community program, real-time analysis can highlight areas where participants are facing challenges or are dissatisfied.

Example Scenario: Imagine a nonprofit dedicated to providing educational resources to underserved communities. They launch a new after-school tutoring program and distribute a survey to gauge its effectiveness. With real-time data analysis, the nonprofit notices early on that students in a particular area are not attending sessions regularly. Upon investigating, they discover that transportation is a significant barrier. The organization can then quickly arrange for transportation assistance, ensuring higher attendance and better program outcomes. Without real-time data, this issue might not have been identified until the program concluded, resulting in lower overall effectiveness and participant satisfaction.

If you choose to alter your programming or services based on real-time data analysis, just be sure you note the date and details of any changes. This will be important context to provide once you conduct your final analysis of the survey data, conduct program evaluation, and consider further refinements.

10. Ensure Mobile Compatibility

Not everyone has access to a desktop computer or laptop with an oversized screen. Ensuring your surveys are friendly for mobile devices (like tablets and smartphones) is crucial for maximizing participation rates and improving the overall respondent experience. Mobile compatibility means that your surveys should be easily accessible and fully functional on various devices, ensuring that respondents can complete them conveniently, regardless of where they are or what device they use.

Mobile-friendly surveys must have a responsive design, meaning the survey layout adjusts seamlessly to fit different screen sizes and orientations. This adaptability prevents users from having to zoom in or scroll excessively, creating a smoother and more engaging experience. Ensuring that text is readable without zooming and buttons are large enough to tap easily are also key elements of a mobile-compatible survey.

Here are a few tips to make your surveys more mobile-friendly, no matter what survey software system you are using:

Simplify the Design: Use a clean and straightforward layout with minimalistic design elements. Avoid clutter and ensure that each question is easy to read and answer on a small screen.
Optimize Load Times: Mobile users may have varying internet speeds, so it’s essential to optimize your survey for quick loading times. Compress images and avoid using heavy graphics or unnecessary animations.
Use Touch-Friendly Elements: Ensure that buttons and interactive elements are large enough to be easily tapped with a finger. Avoid placing elements too close together to prevent accidental clicks.
Limit Text Input: Typing on a mobile device can be cumbersome. Instead of requiring lengthy text responses, use multiple-choice questions, dropdown menus, and sliders.
Test on Multiple Devices: Before launching your survey, test it on various devices and screen sizes to ensure compatibility. Pay attention to any issues that might arise on different operating systems and browsers.
Provide Clear Navigation: Make it easy for respondents to navigate through the survey. Use clear and visible navigation buttons, and ensure that respondents can easily go back to previous questions or save their progress.

11. Utilize Form Logic

Form logic, also known as conditional branching or skip logic, is a powerful feature that enhances the respondent experience by dynamically adjusting the survey based on their answers. By using form logic, you can ensure that participants only see questions relevant to them, reducing survey fatigue and improving data quality.

Benefits of Form Logic:

Personalization: Form logic allows you to tailor the survey experience to each respondent. For example, if a respondent indicates they have participated in a specific program, you can present follow-up questions that delve deeper into their experience with that program, while skipping irrelevant sections for those who haven’t participated.
Efficiency: By eliminating unnecessary questions, you can shorten the survey length, making it less time-consuming and more engaging for participants. This efficiency can lead to higher completion rates and more accurate responses.
Improved Data Quality: Targeted questions based on previous answers help gather more precise and relevant information. This precision reduces the likelihood of respondents providing random or inaccurate answers just to proceed through the survey.

Learn more about form logic and how to use it effectively here >>

Learn more about our Compyle Form Logic upgrade >>

Conclusion, Tips Summary, and Next Steps:

Creating effective program surveys is essential for gathering accurate and actionable data that can drive program improvements and better serve your community. Survey creation is both an art and a science, so you may want to study the research behind question development and survey structure. See our reading recommendations below for more resources to help you design effective survey questions.

Tips Summary

By following the 11 tips outlined in this blog post, you can design surveys that are focused, clear, and unbiased, leading to more reliable insights:

Set Survey Objectives: Clearly define your survey’s goals to ensure that every question serves a purpose.
Create Performance Measures First: Establish performance metrics to gauge the success of your program.
Keep Your Questions Simple: Simple, direct questions are easier for respondents to understand and answer accurately.
Don’t Ask Too Many Questions: Keep surveys concise to maintain respondent engagement and increase completion rates.
Include Relevant Demographics: Collect demographic data to analyze trends and segment responses effectively.
Understand and Utilize the Right Question Types: Use various question types appropriately to gather both qualitative and quantitative data.
Avoid Biased Questions: Design unbiased questions to ensure you collect reliable and accurate data.
Switch from Paper-Based Surveys to a Survey Software System: Transition to digital surveys for efficiency and real-time data collection.
Implement Real-Time Data Collection and Analysis: Utilize real-time data to make timely decisions and adapt your program as needed.
Ensure Mobile Compatibility: Design mobile-friendly surveys to reach a broader audience and improve response rates.
Utilize Form Logic: Use conditional branching to create personalized survey experiences that are relevant to each respondent.

Leveraging Compyle for Effective Surveys

Clear Impact’s data collection and service recipient management software system, Compyle, is a powerful tool that supports effective survey creation, analysis, and decision-making. Compyle offers digital survey distribution tools, allowing you to reach your audience effortlessly across various online platforms. Its real-time data collection, and powerful analytics tools enable you to track responses instantly and visualize data through customizable dashboards, facilitating timely decision-making. Our Form Logic tools allow you to create personalized and relevant survey experiences for each respondent, improving data quality and respondent satisfaction.

What Sets Compyle Apart from Other Survey Tools

What sets Compyle apart from competing survey systems is its ability to maintain respondent profiles and engage in longitudinal analysis. This means you can track survey responses from individual participants over time, providing valuable insights into trends and changes in opinions or behaviors. This feature is particularly useful for evaluating the long-term impact of your programs and making data-driven decisions for continuous improvement.

By utilizing Compyle, you can enhance your survey effectiveness, gather comprehensive insights, and ultimately improve your program outcomes. Whether you are a nonprofit, government agency, or philanthropic organization, Compyle provides the tools you need to create impactful and meaningful surveys. Sign up for a custom demo of Compyle here >>