Youth programs have enormous potential to create social change for individuals, families and communities, but how can you make sure you are evaluating youth programs effectively? Evaluating youth programs is critical to satisfying external funders and policymakers who want to ensure resources are being deployed effectively. Internally, program administrator’s evaluations identify effective practices in concrete terms and provide on-the-ground staff with details and data that can drive continued improvement and success.
Researchers suggest that you need to 1) connect your efforts to outcomes, and 2) make sure the youths themselves are involved in the evaluation process
Effectively evaluating youth programs means assessing both program implementation and outcomes. While effective assessments should be anchored to outcomes, identifying key metrics and collecting data about program implementation, i.e., process evaluation, is important to connect the specific efforts you’re making to the desired outcomes for individual youth.
Strong process evaluations look at what specific services and activities were planned, what was actually delivered, and who participated. They also ask whether program participants were satisfied and capture specific feedback from those participants.
Outcomes evaluations focus on capturing data about specific changes in the target population. For example, what knowledge, skills, behavior, or beliefs have participants gained or changed? Have participants changed their education, work, or health status? Outcomes evaluations look deeper into how those changes reflect the stated goals of the programs. They also seek to identify whether those changes are a result of the specific program activities and services.
An evaluation that focuses only on process won’t demonstrate your real, concrete impact. At the same time, focusing exclusively on outcomes limits your ability to identify actionable feedback for program improvement. Effective evaluations emphasize outcomes and make sure to capture key process data to drive refinement and improvement on the ground.
Programs that anchor their evaluations in outcomes while collecting key process data benefit immensely from systems that integrate data collection into daily program management. Systems like Social Solutions’ ETO product suite are specifically designed with outcomes-centered evaluation in mind. They also make sure that evaluation data is being captured throughout the lifecycle of a program, not just during the course of the evaluation. That not only ensures accuracy, but also allows your actual evaluation design to be data-driven and thoughtful.
Increasingly, research on evaluating youth programs is encouraging youth participatory evaluation (YPE). This approach seeks to engage the youth being served in the evaluation process as more than just subjects. YPE provides a model that allows young people to research the issues that impact their lived experience and develop useful knowledge about their community. The goal of YPE is to create organizational spaces and opportunities where youth can meaningfully participate in decisions that affect them.
While participatory evaluations are not new in the human services sector, inclusion of children and adolescents in those evaluations is a fairly recent development. As the Act for Youth Center for Excellence explains:
Youth Participatory Evaluation has emerged over the last several years as a new field of inquiry, influenced by positive youth development and participatory evaluation methods and approaches. Its foundation has been laid: definitions have been established; core principles of youth participation to guide practice have been delineated; and articles, journals, and books have been written, exploring its dimensions and impacts. Yet the actual practice of bringing adolescents to the evaluation table, as full partners rather than subjects of study, is still relatively uncommon.
Closing the gap between YPE theory and practice can substantially improve the quality of your evaluation. If you haven’t studied or been trained on YPE, you might be surprised to discover that it allows you to integrate youth participation in every aspect of evaluating youth programs. With the right structure and support, young people can be involved in developing research questions, identifying the sample set, recruiting participants, creating data collection instruments, gathering data, interpreting and analyzing findings, presenting findings to stakeholders, and making recommendations for changes.
It doesn’t take much imagination to see how involving youth participation in these activities can provide critical insight and context for any evaluation and program.
YPE researchers have identified common challenges to most YPE projects. Overall, they have found that the main difference between successful YPE initiatives and unsuccessful ones is the extent to which stakeholders are aware of the challenges and take them seriously. Underestimating the significance of the challenges, failing to deal with them directly, and/or hoping that the challenges would somehow resolve themselves and go away consistently limits the success of YPE efforts.
While there can be a number in smaller issues in any given evaluation, the research suggests three high impact “leverage points” where those challenges can be addressed with broad impact.
- Acknowledge the Inexperience and Concerns of Many Adults.
Especially when it comes to program evaluations, most adults have less experience sharing power with youth. That’s not necessarily a huge failing, but more a recognition that youth-adult partnerships are less common. Lack of experience leads to lack of understanding and often makes YPE uncomfortable at first. Ignoring that discomfort will immediately create barriers to successful YPE implementation.
Remember, the goal of YPE approaches is to create a partnership between adults overseeing programs and the youth that those programs serve. Building that partnership means proactively addressing the inexperience of the adults involved.
To build successful partnerships for evaluating youth programs, start with the assumption that the adults want to partner with youth, provide opportunities for youth and adults to get to know each other, and create situations where the adults can directly observe the competence and commitment of young people.
- Adapt to the Complexities of Youths’ Lives
When young people fail to follow through or are less than enthusiastic about participating in program evaluations, many adults assume they lack interest or dedication. The truth is that young people’s lives are extremely busy and they often don’t have control over things that adults can take for granted.
Young people who want to be involved in YPE may lack reliable transportation, need to care for siblings, have little control over work and school schedules, and have time-intensive academic, extracurricular, or service commitments.
Plan ahead for youth to have periods of disinterest or limited availability. Divide tasks to make sure youth have access to adult support and that the two groups can back each other up. And encourage adults to be patient with youth and look for opportunities to encourage, train, and support youth participation.
- Respond to Existing Structures and Norms within Organizations
Evaluation projects regularly create added stress for organizations that need to continue carrying out program services and activities. Often, this can be compounded when the organizational structure and norms within an organization are more rigid or structured than those of the evaluation team. Organizations like schools may be built on strict adherence to hierarchy, tradition, and established structures
Evaluation coordinators need to be prepared to respond to these existing structures and norms. They should expect a certain degree of push back from the organization they are evaluating and plan time and resources for addressing organizational concerns as they arise.
As youth are involved in the evaluation process, they should also be prepared for youth to question or resist those structures and norms. Especially when youth want to discuss concerns around issues of race, ethnicity, and social justice, those questions can be seen as challenging authority.
Being mindful of organizational culture, seeking to change, adapt, or accommodate structures and norms as part of the evaluative process, and creating opportunities for youth to make meaningful decisions within organizational constraints can help resolve tensions that arise. On a practical level, making sure youth evaluators have space of their own (like a dedicated conference room or office) can make a huge difference in the way they engage and how they are viewed by the organization.
Evaluating youth programs effectively can require a great deal of effort and resources. But by centering on outcomes, tying those outcomes to efforts, and involving youth in the evaluation process, you can make sure your efforts and resources leads to powerful impact data and ongoing improvement and success!