CLARITY OF PURPOSE IS SO IMPORTANT
Everything always comes back to purpose.
Have you been part of evaluations, where 6-12 months in, you’re starting to uncover some really important learnings… but you can’t quite recall exactly what you set out to explore when you started, and now you’re overwhelmed with choices about what to do with what you’ve learned… and sometimes you don’t end up doing anything with the learnings?
Or perhaps the opposite, where 6-12 months in, the learnings that are starting to emerge are really not meeting your expectations, and you’re wondering if this whole evaluation thing was a waste of time and resources?
Whilst there are many types of evaluations, one evaluation cannot evaluate everything. A good evaluation is purposely designed to answer the specific questions of the intended users, to ensure it can be utilised for its intended use. It’s critically important to ensure the evaluation, and all those involved in it, remain clear about its intended use by intended users.
A simple taxonomy that I find helpful is one proposed by Huey T. Chen (originally presented 1996, but later adapted in his 2015 Practical Program Evaluation).
Chen’s framework acknowledges that evaluations tend to have two main purposes or functions – a constructive function, with a view to making improvements to a program; and a conclusive function, where an overall judgement of the program’s merit is formed. He also noted that evaluations can be split across program stages – the process phase, where the focus is on implementation; and the outcome phase, where the focus is on the impact the program has had.
The four domains are shown below:
- Constructive process evaluation – provides information about the relative strengths and weaknesses of the program’s implementation, with the purpose of program improvement.
- Conclusive process evaluation – judges the success of program implementation, eg, whether the target population was reached, whether the program has been accepted or embedded as BAU.
- Constructive outcome evaluation – explores various program elements in an effort to understand if and how they are contributing to outcomes, with the purpose of program improvement.
- Conclusive outcome evaluation – provides an overall assessment of the merit or worth of the program.
This simple matrix can serve to remind us of the purpose of the particular evaluation work we are doing at any given time. It is simple, and there are of course nuances, where you may have an evaluation that spans neighbouring domains, or transitions from one domain to another, but despite its simplicity, I have found it a useful tool to remind me about the focus of the current piece of work or line of enquiry.