ARE WE NATURAL EVALUATORS?
Think about the last time you bought a car, chose where to live or decided which breakfast cereal to throw in the trolley (or in the online cart if you’re isolating like we have been for the past few days) … without overtly realising it, you quite possibly followed the logic of evaluation.
For those interested, you can read more about Michael Scriven’s work on the logic of evaluation here, but to paraphrase, evaluative thinking has four key steps.
- Establishing criteria of merit
- Constructing standards
- Assessing performance
- Evaluative judgement
Let’s apply this to buying your next car.
Establishing criteria of merit
What criteria are important to consider in buying a car? Maybe we have a firm budget, so any car we’re going to consider will need to perform well against that criteria. Maybe we’re climate conscious, so we will place more value on a car that has a better emission rating. Maybe our family has just expanded, so need to find a car that will accommodate a minimum number of people. Maybe we don’t like white cars, maybe we’re looking for a car with low kms, maybe we’re only interested in cars made by a certain company, because we believe them to be a safe and trust-worthy company.
The list of possible meritorious criteria is extensive, and quite dependant on who needs to make the evaluative judgement. Two people shopping for a car at the same time may have quite different criteria.
Constructing standards
This is where we define how well any potential car needs to perform against the various criteria we identified earlier. If our budget is a firm $10,000, then any car that comes in over budget is not going to score well against that criteria. Some of us may even write a list of our must haves and nice to haves, and include our standards in that list. Maybe our next car must be within our budget, must be able to carry five people, must a be no older than 10 years old. We’d also, if possible, like to find a purple car, that’s located within 50km of where we live, so we don’t have to travel too far to collect it, and has had less than 2 previous owners. We now have a list of criteria that are important to us, and we have set standards against each of those criteria.
Assessing performance
Now’s the point at which we assess how any contenders stack up. Regardless of whether we’re scrolling through an online marketplace for cars, browsing various websites, or physically walking through car yards – we are assessing how each car we review compares to our defined criteria. Maybe we find a purple five-seater car within budget, but it’s 200km away. Maybe we find a blue five-seater car within budget and within 50km, but it’s 15 years old. In the same way that our programs, policies, products and people don’t perform ideally against all criteria – our potential cars are the same. We will take note in our heads, or some of us will actually take notes in a notebook or prepare a spreadsheet, where we track how each car performs. We may find that one or two of the criteria we determined were important to us, were simply unable to be assessed because data wasn’t available. We should learn from this, and perhaps rethink our criteria and/or standards for next time.
Evaluative judgement
Now we need to make our evaluative judgement, which will inform our decision. Which car performed the best? Often not all criteria are equally important, and we apply different weightings. Maybe we determine that blue is pretty close to purple, so despite the car not being the right colour, it’s still a contender. We need to determine how we will synthesise all we have learned from our evaluation activities, and make an evaluative judgement. (For those interested in more about how to integrate or synthesise multiple criteria, Jane Davidson has a nice chapter on Synthesis Methodology in her book Evaluation Methodology Basics).
We often work through these steps fairly quickly in our heads, especially if the decision is about what cereal to buy, or what to have for dinner. Are all the kids going to be home? Does one child have a particular allergy or dietary preference? Is the budget tighter this week because we needed to spend extra on fuel? We might not go to the effort of making a spreadsheet, or getting evaluation support to make these decisions… but we do employ some logical evaluative thinking more often than we realise.
