What is Program Evaluation?
Throwing a group of professional evaluators together in a room and asking, “What is program evaluation?” would likely result in a lively discussion and a variety of answers that, while not necessarily contradictory, would demonstrate the depth, diversity and complexity of this field of study and practice.
Indeed, there is a vast literature base that explores the concepts, applications, philosophical foundations and controversies surrounding the ways in which program evaluation is defined and how related tools are used. Our hypothetical group of evaluators might debate the strengths and limitations of data types (e.g., qualitative vs. quantitative data); approaches to categorizing the various program evaluation methods; whether the much hailed randomized control trial is truly the gold standard of designs; ethical issues surrounding the use of control groups; or even the nature of reality and what is truly possible to measure, interpret and demonstrate through the use of evaluation methods. And this would only scratch the surface of the thousands of articles and books that comprise this complex subject area.
A Starter Definition
Yet, funders are increasingly driving nonprofit directors and program managers towards reporting requirements and evaluation techniques that measure and provide stronger evidence of program outcomes.
Before you can learn how to evaluate a program, it would be helpful to have an answer to the question “what is program evaluation” So, where do we start if we are new to this area? Perhaps a good beginning is to work with a definition of program evaluation that can help us ground a bit of our thinking. Here is one such definition:
Evaluation is the systematic application of scientific methods to assess the design, implementation, improvement or outcomes of a program (Rossi & Freeman, 1993; Short, Hennessy, & Campbell, 1996).
Let’s unpack this program evaluation definition a bit.
- The first thing we note is that program evaluation should be systematic. This means that we need to be thoughtful, deliberate, structured and rigorous in our approach.
- Notice also that methods is plural. This means that we have a lot of tools (i.e. program evaluation methods) in our evaluation toolbox to choose from. We need to be careful in the selection of the tools that will best serve our needs. Which ones we choose will depend greatly on what we hope to accomplish through an evaluation effort.
- Thirdly, we see examples of things that might guide our selection of tools; that is, what we want to assess within our programs. Examples here include a focus on the structure of our program design, its manner of implementation, areas for possible improvement, and, of course, the measurement of outcomes or impacts.
So, as we begin our journey, we start with an appreciation of the importance of these three operational areas: 1) being systematic – the “how,” 2) using scientific methods – the “what,” and 3) assessing something specific about our program – the “why.”
What Small Steps Can I Take to Explore Program Evaluation Methods
While this definition provides us with a preliminary answer to “what is program evaluation,” it says nothing specific about the actual program evaluation methods we might use or the approach we might take. This is because our chosen approach is greatly dependent on the questions we seek to answer through an evaluation effort.
So, as a program manager or nonprofit leader, how might I learn how to evaluate a program?
- As discussed, start with defining your questions. Determine what you want learn and what purposes this learning might serve.
- Once the questions are specified, explore existing evaluation literature in the same or a comparable program area where similar questions are being answered. This can help you think more critically about the nature of your questions and how others have sought to answer them.
- Finally, consider spending a little time with an evaluation professional who can help you think through your best options based on what you want to accomplish and what your program might be ready for.
Taking these three steps in relation to the evaluation definition above, can greatly assist you in moving confidently toward an appropriate and meaningful evaluation effort.
Rossi, P. H., & Freeman, H. E. (1993). Evaluation: A systematic approach (5th ed.). Newbury Park, CA: Sage Publications, Inc.
Short, L., Hennessy, M., & Campbell, J. (1996). Tracking the work. In Family violence: Building a coordinated community response: A guide for communities.