Exploring New Horizons: Building Internal Evaluation Capacity and Systems
Leaders of an environmental education group recognized the need to document their participants’ outcomes, after a long reliance on anecdotal evidence. There were many challenges: None of the staff possessed evaluation experience, some resisted the notion of “testing” participants, the program’s remote site had poor access to technology, and it would be impractical to evaluate all the program’s participants. Assistance was sought to develop the group’s first internal program evaluation system. The project involved designing and piloting a pre-/post-questionnaire along with feasible sampling, data collection and analytical methods.
“This project was a good first step for us. We learned a lot about the organizational capacity needed for evaluation. It helped staff better understand what evaluation is and why we need to do it. The preliminary survey findings will help reduce staff resistance to data collection.”—Tracey Weiss, Executive Director, Exploring New Horizons
WuYee Children's Services: Rationalizing In-house Program Evaluation
Leaders of this agency’s family resource center sought assistance to develop more effective program evaluation tools and methods. With improved internal monitoring systems, leaders felt that staff and other internal stakeholders—as well as funders and other external stakeholders—would gain greater understanding of the program’s impact on its clients. Not only would better data help the program attract and maintain financial resources, it would save valuable staff resources by streamlining and/or eliminating some of the existing evaluation methods.
"I really appreciate all the dissecting work of our evaluation forms and processes. It's been eye-opening to dive in and untangle things [and develop] something much more manageable. Thanks for helping us with this hard work and leaving behind an improved program.”—Lena Yu, Joy Lok Manager, WuYee Children’s Services
YMCA of the East Bay: Creating Internal Program Measurement Systems
Leaders of the outdoor school at the Y’s Camp Arroyo sought help to develop their first program performance measurement system to better understand their program’s effectiveness. The small staff had no experience collecting participant outcomes data but wanted to start using a pre-/post-program survey. Given their lack of evaluation experience and limited resources, we weren’t sure what would be possible. Certainly, any new tool must be easy to administer and practical to analyze and make sense of. Staffers were happy with the short questionnaire and feasible strategies we developed and piloted. After the project, they continued using them to learn what participants were learning and how their attitudes were changing.
“After several years of successful results, I continue to contract with Eleanor. She provides a breadth of knowledge, solid research skills and expert analysis. In all her interactions, she is friendly, cooperative and flexible without compromising superior outcomes.” --Ginni Ring, Executive Director, CASA of Alameda County, formerly with YMCA of the East Bay