The Chicagoland Evaluator: Member feature

Using evaluation to inform policy: Tips for evaluators an interview with Dr. Anne Farrell, Director of Research, Chapin Hall at the University of Chicago

For this edition of The Chicagoland Evaluator, Dr. Rafiqah Mustafaa, Evaluation Coach at Planning, Implementation, Evaluation Org interviewed Dr. Anne Farrell, Director of Research at Chapin Hall at the University of Chicago. Below, Rafiqah shares Dr. Farrell’s valuable tips on how readers might conduct rigorous translational research and evaluation that can be useful for decision-making.

1. Deeply examine the theory of change. This is an essential component of any project because it helps the evaluator or researcher to understand the logic behind the policy or program being examined. To understand this logic, Dr. Farrell suggests asking questions like what is this program/policy trying to accomplish? And who is involved in implementation?These answers provide a realistic idea of how the program operates, the outcomes that matter, and where our work can have the most impact. Too often, measurement of implementation is ignored, making it impossible to know what components are responsible for change (or lack thereof).

2. Seek the perspectives of various stakeholders including the policy’s intended beneficiaries and frontline workers. Inquiry into the effectiveness or impact of a policy or program involves an informed, collaborative dialogue. As evaluators, we have perspectives that shape our inquiry; however, there are many things we don’t know, or can’t know, because of our particular vantage points. It is important that we understand the perspectives of the beneficiaries, frontline workers, and other stakeholders, and use those perspectives to inform our evaluation process. In particular, equity-oriented evaluation efforts need to tap the perspectives of community members and individuals who are intended to benefit from initiatives and programs.

3. Study in depth the policy and its related regulations. Policy-oriented evaluators need to understand deeply the policies of interest in order to align evaluation to inform policy. To do this, evaluators can ask questions like what are the regulations associated with this policy?And how are the intended beneficiaries affected when laws, regulations, or appropriations change?  By answering these questions, evaluators are better positioned to provide insightful analyses and relevant recommendations for decision-makers.

4. Focus on collecting data that is needed and useful. Dr. Farrell shared that she has become a measurement “minimalist” as she’s evolved in her career. Very often, heads of agencies, frontline staff, and intended beneficiaries are required to collect and/or report data that is never actually used in evaluation. Policy-oriented evaluators need to have a clear understanding of the purpose of each data point. Knowing this, we can use a more intentional approach to measurement that focuses on collecting quality, useful data, and minimize demands on the individuals involved in the data collection process.

This conversation with Dr. Anne Farrell highlighted that while evaluators bring to their work a range of interests, skillsets, and approaches, it is important to consider how we can be intentional in shaping our work to maximize its utility for policy and practice.