Learn Tips and Tricks to Program Design and Evaluation
Check us out every 2 weeks for a blog about all things evaluation!
Introduction to Needs Assessments
“Our program is meeting its stated goals, but our clients are still struggling!”
Have you ever experienced this phenomenon? Your program works as designed – your clients are experiencing the outcomes you’ve decided were important to measure. However, your clients are still struggling.
Don’t worry. You are not alone!
Needs assessments are critical tools in the toolbelt of any nonprofit or government organization that is responsible for designing programs that serve the needs of a group of people.
A needs assessment is an evaluation tool that helps fill gaps between what is currently happening and your desired outcome. For example, let’s say you manage a nonprofit that serves individuals experiencing mental illness. Your typical program consists of offering free counseling to those who need it. Your clients attend counseling sessions every week and meet their socioemotional outcomes; however, they still cannot maintain a living wage job, so they struggle with maintaining safe and stable housing and obtaining groceries every week.
Currently happening: counseling.
Desired outcome: to live independently.
How do we know what we are missing? It is clear that counseling does not lead your clients to lead independent lives, so what else needs to happen?
Here is where a needs assessment is crucial.
A needs assessment is a type of evaluation that collaborates with the community (your clients or potential clients) to determine their challenges and how to solve them. There are several methodologies to achieve these results.
Surveys, interviews, focus groups, and observations are typical methodologies to glean information for the needs assessments. The key is to gather as much information as possible (from diverse methodologies) about what your population is struggling with and what they think they need to overcome that challenge. Oftentimes, we assume that we are the experts and know what is best – but in fact, the community is the expert in their own experience. Too often, we ask their challenges without asking what solutions they hope to see. Communities often know exactly what their challenges are and exactly what will best support them – they need someone who will co-create those types of programs with them.
When surveying, interviewing, or facilitating focus groups, collect demographic information. Then, compare responses across various demographics. You want to know if all or most people in a certain demographic are answering in certain ways. Are all of your white clients having a different challenge than all of your Black clients? Perhaps a solution that would work in your white American community would not work as well in your immigrant community.
When designing your needs assessment project, consider the following:
1. Create an ad hoc advisory board. This board will help you draft questions, recruit study participants, and make sense of the results. Having an advisory board (made up of people from the community) ensures you make the right assumptions based on the information you are given. When analyzing data from your perspective, you will likely design a program that looks vastly different than when someone with lived experience analyses the same dataset. Experiences matter and are crucial in the design and analysis process. Your job is not to have the answer but to elevate the voices of those who have the answers.
2. Triangulate your data by engaging in multiple research methodologies. Use surveys, interviews, focus groups, and observation together. When you have multiple methodologies and the same information is being repeated across each method, you know it is an important feature in your dataset. Stagger your methods so they work with and not against one another. You could conduct a series of interviews or focus groups first to glean key information from important stakeholders, then use that information to design a survey to go out to the masses. Or you could design a survey to go out to the masses to see where people are in general terms, then use those survey responses to design your interview or focus group questions. Either strategy is excellent, depending on the goals.
3. Consider who you engage in each type of methodology. Understand that you obtain rich, in-depth details about someone’s life with interviews. A focus group gathers details (but not in-depth) on multiple people simultaneously. Focus groups are excellent when gathering data from multiple perspectives; remember that the data may not be as detailed as you need. Sometimes, it is a good idea to follow up a focus group with a few in-depth interviews with people from the focus group. Additionally, the questions you ask will determine your methodology. You don’t want to ask sensitive or potentially embarrassing questions in a focus group. Finally, you want the most detailed and rich information from those most important to the issue. Consider interviewing those you think are key to the issue and not necessarily the most important based on their title. For example, when determining what barriers someone with a mental illness has to employment, you may want to ask someone unemployed rather than asking a CEO who makes no hiring decisions their perspective.
Regardless of the topic, there are some key questions I always like to ask that help set the stage.
1. What does an average day look like for you now?
2. What would your dream day look like for you?
3. What is missing between your average day and your dream day?
4. What are some of the biggest barriers you face regarding (organizational mission)?
5. What would that look like if you could design a solution that would alleviate some of your biggest challenges?
Overall, needs assessments provide rich information on gaps between what you are currently doing and what needs to be done based on the perspective of the community you serve. Multiple methodologies and strategies will help you on your journey. The key to remember is community engagement – do not make decisions without community involvement.
If you are interested in conducting a needs assessment but don’t know where to start, contact me at Jodie@ChangeAmplifiers.Com
How Evaluation Supports Program Design and Implementation
As nonprofit leaders, we continuously design or modify programs to better serve our clients. The question becomes: “How do we design programs so they meet the direct needs of our clients?”
The answer to this lies in the program design, implementation, and evaluation cycle. This cycle begins with conducting a “needs assessment.” Based on the “needs assessment,” the program can be designed, and funding can be secured. Once funding is received, program implementation can begin. Finally, at regular intervals, program evaluations occur.
The “needs assessment” is the foundation of this whole process. A “needs assessment” engages potential clients to hear their biggest challenges and potential solutions directly from them. The key to this phase of the cycle is engaging the community. So often, nonprofit organizations skip this step because they think or assume they know the community's needs. The community tends to know what they need – they just need to be empowered to ask for it.
Let’s simplify the example of housing. A nonprofit organization can assume that if there is a housing crisis, people need housing – so they design a program that builds houses. However, after building homes, the organization finds no one buys them. Why?
When asking the community what they need, they say their biggest need is employment to afford housing. Building new houses won’t solve the problem because families do not earn enough to buy a home. However, a program offering employment training services may lead to families having the ability to stabilize their housing situation. This example helps demonstrate that the community is the expert in knowing what they need. So, just ask!
Now that you know what the community needs, it is time to design the program. Since the community knows what they need, create an advisory board of community members to guide the program design, clarify issues that may arise, and build buy-in from the community. Ensure you draft a program design – often in the form of a Theory of Change or Logic Model (more to come on that in March!)
Your advisory board and you have designed a program that could work for your community. Now what?!
Securing funding is a critical component of the cycle. As nonprofit leaders, you fully grasp the importance of funding. It is more difficult to secure funding for a new pilot program than for an established program with results. There are several grant-makers, however, that will support pilot programs. Often, smaller, family-owned foundations will support pilot programs if they are passionate about the program's goal. Another funding avenue is “major gifts” fundraising. Individual philanthropists may feel passionate about your program and will donate substantial financial capital to support it. You may have to apply for multiple grants or seek donations from multiple philanthropists to secure enough funding to implement your program. This is where your program design (your Theory of Change or Logic Model) will come in handy. Grants (and philanthropists, to an extent) want to see that you’ve thought about what your program will do and how it will do it. Use your Theory of Change or Logic Model to work for you.
Finally, a major donor who is particularly passionate about your new program has come forward. Now is the time to implement the program. Implementing programs is what we do every day – we know we need to hire staff, recruit clients, and begin performing services.
After conducting services for a year, you are interested in determining if your program is working. Now is the time to conduct an impact evaluation (more to come in April!) It is best practice to evaluate the program annually to determine if modifications must be made. Additionally, having results is a plus when it comes time to apply for more grants or seek more donations! You can conduct the evaluation internally, using these blogs to offer support and guidance, or hire an external evaluator with expertise in conducting needs assessments and evaluations. Tips and tricks on working with an external evaluator will come in June!
In summary, evaluation is critical to the program design and implementation cycle. It ensures you are designing programs that truly meet the complex needs of the communities you serve; it allows you to see your impact, and, of course, it helps you secure additional funding. Use this blog series over the course of this year to develop your evaluation program! Reach out to me at Jodie@ChangeAmplifiers.Com for questions about all things evaluation!