Update from 4/9/2012: We agreed to a method that FOTC would use to analyse and prioritize the possibilities. Within that they would also identify other initiatives that they had related to the themes identified from the focus groups. No meeting next week as they’ll take that time to weigh and decide.
Update from 4/2/2012: Three focus groups were facilitated, segmented by job function within Friends of the Children. They were all very lively indicating a culture of having permission to speak openly. All sessions were recorded then sent to the transcriptionist leading to 25 to 30 pages each. Next, Susan entered the data into spreadsheets. From that answers to each question were coded by like themes, then each individual answer was coded to confirm the themes. This was an iterative process requiring two or three passes.
The data provided is qualitative and there are some general guidelines when interpreting qualitative data. We can do a lot with qualitative data; the cautionary point is to not to try and quantify this information. Doing so is a typical tendency but qualitative data “points to” areas that need addressing rather than quantifying them.
Resist counting the number of people who make comment or rely on the numbers present in the data. It’s not that a frequent similar comment doesn’t indicate an area to pay attention to. It’s just that the way the data is collected (open-ended) and the small number of participants does not allow for valid quantitative conclusions or comparisons. An issue brought up by two people can be a bigger issue than one brought up by seven people if the two people have an in-depth understanding of the issues and represent the voices of many others not in the room. If it’s important to quantify any of the qualitative findings, then follow up with a survey that includes the bigger population from which the focus group participants were drawn.
The questions asked were open ended questions. The responses could have been anything. One could respond in different units than another (for example when asking about how much you drive one could report the number of hours and another could report in the number of miles). It’s important not to get hung up on any one answer. Responses could be outliers. Also try to resist the natural tendency to identify the people that answered the questions to protect their anonymity and reduce any biases you may have toward that person or their opinions. That said, it’s important to consider the type of participant that provided the response. Was it a team leader or staff member? It will help contextualize their response.
The beauty of a focus group is in the snowball effect it engenders. One person’s comment can trigger another person’s thinking (and commenting) leading to another’s comment. But there could also be some influencing, especially by those with strong opinions. A skilled facilitator can minimize this effect by setting ground rules upfront and often asking, “So what do others think?” In general, the group is still smarter than the smartest individual within the group.
Look for convergence of the data. Sometimes this is called triangulation. This happens when the same information comes from multiple people, from people in different roles, from the responses to multiple questions, or from related themes emerging throughout the data set. For example, at FOTC there is a habit of being all inclusive with everyone, i.e., giving everyone permission to provide input. There seems to be a related pattern about being flexible with interpreting polices (giving people permission to interpret policies uniquely). This thematic pattern points to a culture at FOTC that may be important to recognize in order to address other areas of convergent data.
Qualitative data is valid for a number of reasons. It is valid based on convergence. It is valid from having been obtained by a professional facilitator. It is valid from the collaborative manner in which the questions were established. Finally it is valid from saturation. For example, a theme that emerged frequently throughout the groups was one in which the number of children that each Friend works with is a trade off with the quality of work that a Friend can do with each child. It got to the point that there was no new information revealed about this topic with each subsequent time it was brought up by someone–it had reached a point of saturation.
Susan recommended that we acknowledge the answers in public by announcing the questions that were asked and the thematic responses provided. She also recommended that we provide a plan to state how we are dealing with each (whether it is with the Lean team, another initiative, or if it is off the table for other reasons). Lastly, she recommended that we contextualize the data in trying to put it to good use. In other words, consider the focus group data in the context of other information that FOTC already has about the issues as well as past history in addressing the problem, any politics that may enter in, and of course budget and resource realities.