Evaluator Karen Walker, PhD

Karen Walker, PhD, is a University of Virginia professor of research methods and qualitative/quantitative community-based research design and an independent program evaluator with significant experience in the nonprofit sector. The Robert Wood Johnson Foundation (RWJF), the largest U.S.-based philanthropy devoted exclusively to improving the health and health care of all Americans, contracted with Walker through Public/Private Ventures to evaluate Children’s Futures, one of its long-term programs. Children’s Futures is an initiative to improve the health and well being of children in Trenton, N.J..  In response to inadequate data collection by the programs throughout the first five years of the initiative, which hampered both program monitoring and operations and the evaluation, Walker oversaw the implementation of ETO Software. 

ETO Motivations

To successfully improve the health of Trenton’s children and document progress to RWJF, Children’s Futures needed to design and maintain a database that would support its overall initiative, which involved the work of a large number of public and nonprofit agencies throughout the area. Walker and RWJF needed Children’s Futures to better manage the types and collection methods of the data it would use to ultimately measure its positive impact on children’s lives and demonstrate the program’s social value. The foundation and Children’s Futures agreed to implement  ETO Software and leverage its proven processes to support broad, consistent data collection and management so it could measure and report accurate overall outcomes.

Success Hurdles

The biggest challenge for RWJF/Children’s Futures was facilitating the cultural change required so that multiple agencies would collect and manage data using a new or different tool. Also, the lack of an extant, cohesive data collection process called for a highly flexible solution that would provide needed adaptability and enable nearly immediate generation of reports to demonstrate value. Moreover, as the independent evaluator hired by the foundation to produce useful information about the initiative's implementation and outcomes, Walker needed to maintain critical business relationships with the agencies providing services even as she worked alongside those same agencies to implement the ETO performance measurement solution.

Lessons and Achievements

Due to a visible commitment to ETO by the foundation and Children’s Futures, Inc., the agencies involved in the Children’s Futures initiative are now fully interacting as they all move forward with the ETO implementation, which is strategically leveraging the extant data collection activities of the agencies. ETO’s easy accommodation of additional data elements is enabling the building of a system of consistent measurement for the long term. Any cultural resistance to new ways of working is dissipating as the agencies realize the value of increased access to meaningful and reliable data.

Children’s Futures, Inc. has assumed responsibility for the implementation and continuing administration of ETO, and is focused on identifying service agency priorities, generating funder reports, and conducting site visits to derive the greatest value from ETO.

The evaluator is now focused on her primary role with improved access to the data that enable her to objectively and accurately assess program efficacy. Such data include extensive demographic data, which add critical dimensions to measures of program efficacy, as well as case-weighted data that allow for a more detailed understanding of staff effectiveness.  ETO’s reporting capabilities are also contributing richly to the substance of Walker’s program evaluation. Moreover, ETO’s flexibility allows Children’s Futures to easily modify the tool to capture additional types of data, adding more substance and depth to the evaluator’s work.