Part Five: Collecting Data through Surveys and Focus Groups
Surveys and group interviews, or focus groups, can provide critical information about how kids and families heard about summer meals programs, why they did or did not participate and their recommendations for improving outreach efforts. Surveys of program sponsors, site managers and volunteers can also provide a wealth of important information to help you evaluate your efforts. Staff and volunteers will have a perspective on why sites saw an increase or decrease in participation and the general satisfaction from participants. They can also provide insight as to the challenges they face in running or sponsoring a site, what convinced them to open or expand a summer site and what support organizations/agencies need to serve more kids.
Developing surveys is as much an art as it is a science; but there are several best practices and practical tips that can guide your efforts. This section provides those best practices and sample survey instruments that you can use or modify to gather information from a variety of stakeholders.
Best Practices in Data Collection Through Surveys
Use language your audience understands.
Be mindful to use terms that everyone can understand in your data collection tools. For example, some sponsors may not use the term outreach as you are intending it. Similarly, there are other terms that your team might use regularly that make less sense to parents, kids and representatives from sponsors and sites. Use terms that are easy to understand or clearly defined in your instruments and when possible, pre-test the surveys with members of the target group to make sure.
In some cases, the survey respondents may literally speak a different language, requiring you to translate your survey. Learn what languages are spoken by significant proportions of the students you will be surveying and arrange to have a translator present at the time of the survey if you can.
Strive for a high response rate.
When you conduct a survey, not everyone will respond and return the survey. To measure participation for a survey, you can look at the response rate, which is found by dividing the number of surveys returned by the number of surveys that you distributed. While there is no magic number for an acceptable response rate, you will want to get as high a response as possible — ideally above 50 percent.
There are several ways to increase your response rate:
- Provide clear instructions for completing your surveys and provide a rationale for taking the trouble to fill them out. In many instances, people will provide feedback when they believe completing the survey will contribute to making a program run better for them and others in similar situations.
- Keep the survey short. This makes it more likely that people will start to fill it out, and more likely that they will finish.
- Follow-up with reminders before your deadlines approach.
- Be polite. Remember to say “please” and “thank you.”
- Whenever possible, provide incentives, e.g., gift certificates for parents or children to a local grocery store or a chance to win something.
- People often forget about completing surveys if they are not asked to complete them on the spot. Consider having the surveys done all at once or in a group where someone is present to collect the surveys.
Consider distributing a survey at an event where your target audience will convene, such as a sponsor training.
Make sure survey intentions and instructions are clear to all.
If your survey is being administered in person or over the phone, it is important that the people administering the surveys understand the purpose of the survey and why you are asking each question. This will allow them to effectively answer any questions that participants ask about the surveys.
It is also important to make sure those taking the survey understand why you are asking survey questions. When dealing with young people, it is important to read the instructions of the cover page of the surveys, insure confidentiality and anonymity provisions, and remind them that participation is optional. For example, they should be told that they can skip any questions for any reason, and that no one at the site will connect them with their survey responses.
Ideally, you should give each survey participant an envelope in which to put their survey to insure privacy, and then collect the envelopes. The surveys are then ready for forwarding to those who will do the data entry and analysis.
Seek permission to conduct surveys with kids.
Prior to starting your survey, you will need to get approval to administer surveys and questionnaires to children. This can be a lengthy process, as there are bureaucratic safeguards for collecting data from groups who cannot be expected to make decisions for themselves, including youth under the age of eighteen. Plan plenty of time to get the permission of the body that administers the sites you want to survey, e.g., the local School District or Department of Parks and Recreation, Director of Boys and Girls Club, etc.
Other survey tips:
- Pre-test surveys for functional purposes, checking links and making sure that any skip logic takes your intended path.
- For all surveys, we recommend that you begin with an opening letter explaining who is collecting the information, that those who fill out the survey will remain anonymous, that filling out the survey is optional and what will be done with the results. If possible, read the letter aloud.
- If budget permits, consider providing a small incentive for completing a survey.
Tips for Conducting a Successful Focus Group
While surveys allow you to get the opinion of individuals, focus groups allow you to create an opportunity for structured conversations in which groups of people hear what others are saying and have the opportunity to react.
The surveys in this toolkit can also be used as prompts for a focus group or as the basis for a questionnaire that you can deliver to children, parents or professionals. If you choose to have a focus group, we recommend the following best practices:
- If possible, make sure at least two people are present to run the focus group, one to take notes and one to lead the discussion. This will make it easier to manage the group and write down everything that people say.
- Aim to have between six and 12 people in your focus group, as larger numbers can become overwhelming.
- It is important for the focus group leader to not to interrupt people, and to encourage everyone to share their opinion and not interrupt others or dominate the conversation.
- Design each focus group to get the point of view of a specific type of participant. For example, it would be most effective to have a focus group of only teenagers who did not go to summer meals, only parents of young children who bring them to summer meals or only 10 and 11 year olds who go to summer meals once or twice a week.
- If possible, ask permission of the group to record the conversation and ask everyone to identify themselves before they speak, so when you go through the transcript you can more easily follow the conversations that occurred.
- If you do record the focus group, it is important to test the sound equipment beforehand and make sure you have sufficient batteries, electrical hookups, etc.
- It is likely that the best time to conduct a focus group for summer meals participants will be during a meal. Ask the site if you can use a side room for the focus group so outside noises will be less distracting and fewer people will overhear the children.
- Ask everyone to consent to be in the focus group, and make it clear to everyone that everything said within the group should remain confidential.
- Encourage everyone in the group to speak but do not force anyone to address any given question. At times, it may be useful to go around in a circle with everyone addressing the same question, but this approach should not be utilized for every question.
- Arrange your questions so you ask one question and then several sub-questions relating back to the original question.
- After the focus group is over, both the facilitator and the transcriptionist should meet as soon as possible to discuss what they learned and to write up their results.
- If you want to understand the perspectives from family members who do not send their children to free summer meals, you can sometimes work with a local community group to identify or convene people for the focus group and offer small incentives such as gift certificates to a local grocery store to entice people to participate.
Before you can collect information from someone, you need to obtain informed consent. Informed consent is when people know why you are collecting information from them and why it would be in their interest to participate. While informed consent is discussed a great deal when it comes to medicine — you would not want an optional operation with dubious benefits that you had not agreed to — it is also important when working with people for any kind of research.
If you are giving a survey, gathering a focus group together or administering a questionnaire, informed consent might include:
- Introducing yourself and explaining what you are doing and why.
- Describing the questions you will ask.
- Explaining what will be done with the information you gather.
- Assuring confidentiality and anonymity.
- If giving the survey, reading the introductory page aloud.
- Giving people the option to participate and respecting the decisions of those who decide to not take part in your survey, questionnaire or focus group.
Learning more from site sponsors can help you identify strategies for addressing barriers to serving more young people, opening more sites, staying open for more of the summer or offering additional activities. You can also learn what they think has caused them to be successful and how other sponsors can replicate that success.
Many organizations or agencies working with summer meals programs have developed surveys and/or other processes for systematic feedback from leaders at the sponsor-level. The survey below is based on tools developed by Brandeis University Heller School, the Washington State Summer Meals Workgroup, End Hunger Connecticut!, Share Our Strength and the Arkansas state government.
Before you finalize a survey, decide how much time you have, how many sponsors you seek to reach and the importance of getting in-depth information from them. In-person or telephone surveys provide an opportunity for getting more detail but take considerably longer than surveys to administer. Surveys conducted on the internet, such as SurveyMonkey, can produce large numbers of responses, but offer no opportunity to have longer discussions on topics that seem important. It is not advisable to use mail surveys, which typically have low response rates.
Also, consider involving the administering state agency in planning for the survey. They may plan to collect information from sponsors as well and you can combine efforts. Or, since sponsors are familiar with their agency contacts, having communication come from the agency may increase participation and trust in the process.
Staff or volunteers at sites that provide free summer meals can provide important perspectives on the operation of the program, how families hear about the food, whether the meals are popular and, if summer meals are part of other programmatic activities, the extent to which including the meals is advancing the goals of the other program(s).
This tool is based upon one used to study New York City outreach efforts. We recommend asking the questions in person or on the telephone at a random selection of at least ten sites. To promote openness and honesty in responses it is important to ensure respondents that their answers will remain confidential. Many summer meals staff are invested in the programs continuing as-is — keep that in mind as you interpret survey results, particularly if you see results that seem excessively positive.
Tool Seven: Survey/Questionnaire for High School and Late Middle School Students Who Did and Did Not Receive Summer Meals for Use During the School Year
This tool is based primarily upon a student survey piloted in the New York City school system in 2011. If feasible, student surveys should be administered during the school year to get information from both students who did and did not participate in summer meals programs. This can be invaluable in learning why those who have not participated in summer meals have chosen not to do so.
By offering separate versions of the survey for those who have and have not partaken in free summer meals to each student, researchers can avoid (a) skip patterns that are often misunderstood by students taking surveys and (b) singling out those who ate free summer meals the previous summer and possibly causing embarrassment. Verbal and written instructions should be given to all young people explaining the survey procedure. This survey can be given out during homerooms, during lunch in the cafeteria or at other times that work well for the school you are visiting.
Tool 7: Sample Student Survey Template for Use During The School Year In Schools That Have a High Proportion of Students Receiving Free or Reduced-Price Meals
- Survey for students who did NOT participate in summer food programs.
- Survey for students who DID participate in summer food programs.
As noted at several points in this toolkit, it is important to insure that students understand why they are being asked to answer questions and to understand the extent to which their answers will — or will not — be confidential. This letter can be modified to go at the beginning of the surveys you conduct. The goal is to explain why it is important to take the survey, that their answers are confidential and that taking the survey voluntary. Ideally, there will be someone in person in the room where the survey is administered to reiterate these points and to answer questions.
Decisions about participating in summer meals are often made by parents or other family members. Experience has shown that it is not reasonable to rely upon parents returning surveys after they leave the site so, if possible, the surveys should be handed out and collected directly at the sites where their children are receiving the meals.
Also when possible, surveys should be handed out and collected by people who do not operate the site in order to promote greater honesty. In many cases, this will not be possible so it may be useful to promote confidentiality through steps such as asking people to put completed surveys in an envelope and seal it before handing it in.
The parent survey below is based primarily upon a questionnaire used at summer meals sites in New York City in 2011.
 Maman, S. (2011, Feb. 1). Planning and Facilitating a Focus Group Discussion, Health Behavior and Health Education 753, Gillings School of Global Public Health, UNC Chapel Hill.