Evaluation Techniques for Difficult to Measure Programs: For Education, Nonprofit, Grant Funded, Business and Human Service Programs

Evaluation Techniques for Difficult to Measure Programs demonstrates the weaknesses of poorly crafted outcome measures and provides the reader with techniques to strengthen programs and provide clients with the quality services they deserve. Programs with difficult to measure outcomes provide inviting environments for weak evaluations and this book illustrates why typical evaluation methods result in less than stellar results. Examples from difficult to measure programs are used to present techniques that can make any evaluation more rigorous. This book will guide the reader in overcoming inappropriate measures, false perceptions and misconceptions that plague many evaluations.

This book provides a new perspective on program evaluation that engages difficult to measure programs, and the aspects of developing an evaluation plan that usually result in a less than stellar result. Agencies settle for “Good enough” because people are not knowledge able enough of evaluation processes to develop something that is more robust. Unfortunately, it is easy to sell a weak evaluation to people who do not know the difference. This modern day Emperor’s New Clothes behavior does little to strengthen the program.

Every program manager and Director likes to have a report that tells them that what they are doing is having a tremendous positive impact on their clients. Usually, if you ask them to describe the benefits to the client, they will instead describe the activities that take place in the program. Others will tell you how much better off the client is because of the program, but many of them cannot give any substantial evidence that the change was a result of the program.

Program evaluation has developed as a focused field of practice that has continued to evolve through fad, fashion and a sound application of scientific measurement and analysis. Just counting the numbers of clients served left human service staffs and funding bureaucrats wondering if the expected results were being achieved through the program efforts. One could justify asking for additional funding if additional clients were to be served, but the question of effectiveness of treatment was never answered by statistics of numbers of clients served. Programs resulted in things being different, but were they better? If they were better, was the improvement worth the investment? Could the same change take place with fewer resources?

The purpose of this book is to take the reader beyond describing what should be done and through the meaningful questions of why. Why conduct a program evaluation? Why do clients actually need the services? Why do the services actually reduce the needs? Why do staff and managers believe the program is actually working? Why do staff members resist efforts to evaluate their program? This book uses examples of difficult to measure programs to show techniques that can make any evaluation more rigorous. It explains why typical methods fall short and it explains why many staff members settle for less than stellar measurement techniques. Focusing on overcoming inappropriate measures and perceptions provides the basic framework for this book.

This book covers the evaluation process in depth and provides details on communication and relationships issues that are only touched upon in other texts. Pretending to have a good relationship might well result in pretending that the data are accurate. Techniques for developing trust and communication vary depending on the audience. Each audience, and their particular needs, is discussed within the frameworks of planning, data collecting and reporting.

"