For more than 30 years, surveys have been an integral component of the Council on Accreditation’s (COA) operations. Each year COA creates and distributes nearly 100 unique surveys, including surveys which are distributed to a variety of stakeholders of organizations pursuing accreditation or reaccreditation.
From that extensive experience, as well as COA staff members’ professional expertise in the area of survey design and implementation, COA has crafted 10 key steps to assist organizations with survey design. Careful planning and design will yield higher quality data and more statistically sound results which, in turn, can lead to better decision making.
- Purpose. Ask yourself some questions. What are you hoping to learn? Are there key decisions that you’d like to make based on this data? Is your goal to learn about satisfaction with services, interest in a new service, or to better understand the population you’re serving? Establishing the purpose or goal of your survey is the foundation upon which a quality survey is built.
- Survey Type. Once you’ve defined your goal, ask yourself what survey type is best to achieve your goal. Is it a traditional paper survey, or maybe a focus group interview? Consider the advantages and disadvantages of each before you spend time creating the survey.
- Who. Determine who you are going to survey. Will it be a distinct population or a cross-section of the population you serve? This question will likely be driven by your overall goal. If you’re hoping to learn about satisfaction with a particular service, then you may want to survey a random cross-section of the entire population. If your goal is to learn how you can expand services to a distinct population, then you will want to conduct a targeted survey.
- Sample. Think about how many people you will survey. What type of sample will you need for the results to be statistically significant and generalizable to the population for which you’re interested? Should you use probability or nonprobability sampling, voluntary or convenience sampling, stratified or cluster sampling? This is a complex question and one with which many organizations struggle. If you hope to make statistically sound decisions based on the data collected, then the best advice is to consult with a statistician regarding sampling.
- Question Type. What type of questions are you going to ask, and which format will give you the best information for decision-making purposes? Asking open-ended questions, such as an essay format, will yield rich data but is difficult to aggregate and analyze. Asking close-ended questions, such as picking from a list or picking multiple options for a list, is easier to collect and analyze but is not as information rich. Generally, most organizations opt for a hybrid of the two.
- Content. What is the content of your survey? What types of questions are you going to ask? Be careful in wording your survey questions so as not to ask a leading question or to direct the respondent toward a desired outcome. According to “The One Number You Need to Grow,” an article published in July/August 2001 by the Harvard Business Review, one of the most important questions a surveyor should ask to determine customer loyalty is, “Would you recommend this service or product to a friend, family, or colleague?”
- How Many. How many questions should you ask? The rule of thumb here is that concise is often better. However, length may depend on what information you are seeking or who you are surveying. A good recommendation is to go back after you’ve developed your questions to determine whether any can be eliminated because of redundancy or nonrelevance to the overall goal.
- Method. What is your method of surveying: mail, phone, and/or the web? Will you use one or multiple methods to reach the survey population? To maximize the return rate, pay attention to the method of surveying. Mail-in surveys typically provide good return rates but can be costly and time-consuming to implement. Web surveys are convenient but not necessarily accessible or simple to all users. And phone surveys, while accessible to nearly anyone, can have higher drop rates if the surveys are too long.
- Timing. When are you going to conduct your survey? How long will it last? Always consider holidays and peak summer vacation times, which may adversely impact return rates. You also want to encourage respondents to return the survey immediately by creating a timeframe that is long enough to get a high number of answers while minimizing procrastination.
- Analyzing and Reporting. How will you aggregate, analyze, and report your data? Who will do this? How will this data be used in your quality improvement efforts? It’s important to define this information upfront, before completed surveys begin coming in the door, so that reporting is simple and intuitive.
Printer-friendly version
