Evaluation shouldn’t be just the add-on at the end of a project – it’s a core skill for enablers of change like ourselves! So let’s explore one framework for evaluation that’s been around for sometime – Bennett’s Hierarchy.
In this blog post we’re going to explore what Bennett’s Hierarchy is and give you a few pointers for using it.
To set the scene for this we head back to the 1970s when Claude Bennett worked as a specialist in evaluation with the extension service in the US Department of Agriculture. He wanted to make sure that extension was collecting information that was able to illustrate the effectiveness of extension, and help extension improve its services to clients. So he developed seven categories of criteria for evaluating extension programmes based on what he called a ‘seven link chain of events’. He put these into a hierarchy – literally in his article describing this – as a diagram of seven steps.
So let’s go through each of these seven levels.
Starting at the bottom we have level 1 and this is inputs. This is resources that are required to run extension programmes, usually things like time, money and extension personnel.
Next up is level 2 and this is the activities – the extension activities – these could be workshops, field days, or resources produced.
Then for level 3 we have participation. This is a common thing for extension programmes to monitor – we count how many people turn up to our workshops or field days or how many saw our work on Twitter or other social media. This is what people like reporting and it’s generally easy to do so!
Level 4 is reactions – that is – the response of the participants to the activities. It may be that they get more interested in finding out about something, that they learn how to do something – it’s the immediate response.
Next we have level 5 – and this is where it starts to get really interesting! This is KASA level change; Knowledge, Attitude, Skills and Aspirational change. This is getting towards the heart of extension – we want people to be able to change and these are the precursors to practice change which is the next level.
Yes! Level 6 is practice change. This is the nitty gritty of extension – it’s why we do what we do. This is the adoption of a new practice, technology, or skill.
And finally we get to level 7 which is the end results. This is the big picture outcomes, the social, economic, environmental or cultural change driven by the practice change.
So there you have the seven levels of Bennett’s Hierarchy – what do we do with them? Well Claude had some pretty good tips for thinking about evaluation using the hierarchy. He pointed out that when you’re thinking about evaluation of extension, the higher up the steps you can gather evidence, the stronger the evidence of impact. So the hierarchy forces you to ask questions about what level will you aim to get to. How high in the hierarchy can you gather evidence?
Bennett was also a realist too though, because he also pointed out that the higher in the hierarchy you go, the greater the difficulty and cost of obtaining the evaluation data, especially hard data. So what’s your budget like and what is realistic to try and gather data on?
These questions are often overlooked when it comes to evaluation, so it’s good practice as an enabler of change to have the chance to work this through and decide what is possible, and why, when evaluating a change programme. This means you can be clear about the change you can track and the amount of time and effort required.
In fact Claude went on to use his Hierarchy as a programme development tool and a programme evaluation tool. Essentially you can use the hierarchy as a way of workshopping the ultimate aims of a programme, and work down the levels to figure out what is needed at each level. Then you can use the hierarchy to figure out what data you can gather to determine how well the programme is going. There’s a few resources that cover this process that we’ll put links below.
So there you have it – one useful framework for evaluation! There are others, but this one has been around for a while and used extensively in agriculture. We’ll be looking at others down the track.
So you’ve read our thoughts, now we’d like to hear yours! Add a comment below and tell us your thoughts about the evolution of extension. We don’t want this to be just a one-way conversation – join in by sharing your thoughts and ideas with us!
References and links
Claude Bennett (1975), Up the hierarchy https://www.joe.org/joe/1975march/1975-2-a1.pdf
Or click here for the higher quality scanned document
Rockwell and Bennett (2004), Targeting Outcomes of Programs: A Hierarchy for Targeting Outcomes and Evaluating Their Achievement https://digitalcommons.unl.edu/aglecfacpub/48
Really useful info John. I use a slightly different model for evaluations but there are some elements in this blog article I can apply to improve the method I use. Thanks!
Thanks Jane… I’m curious, what model do you use?
Hi John. Yes very useful for designing and evaluating. The keys are considering what practice changes are required to achieve the end results and working back to what KASA changes will lead to that and in turn what how many people to attend the activities will lead to the required KASA changes. We can then evaluate the # of people and their reactions to activities to help improve the activities and assess their changes in KASA before and after their involvement in the activities. Then assess the short-term and longer-term KASA changes and practice changes leading to the end results… Read more »
Thanks for your insights there Roy! I’ve heard that approach described as walking down and then back up a staircase, which is a great way of integrating the planning of an activity (or project) and then later evaluating it.
Thanks, I have been using Bennett’s Hierarchy since the 1990’s and it has been an indispensable part of effective project management since then. I think success is about first accepting that level 5 KASA achievements are the most useful for planning and delivering project achievements, and secondly winding that effectively into organisational systems like MERI, program logic and others to clearly communicate what teams plan and deliver.
Great, thanks for sharing that Noel. I agree that it’s an indispensable tool – so handy and applicable across a wide range of activities and projects!
Very useful, we use Log Frame in M&E where in Activity, Output, Goal and Impact are measured.
Thanks for sharing this.
Indeed the Log frame is an excellent approach and we will cover that in future episodes.
Thanks John and Denise – yes I’ve been using the Bennett’s Hierarchy in my extension activities since the 1980’s – it is a great framework which underpins program development and evaluation. And yes, I have a copy of the original paper as well – worthwhile going back to from time to time.
Thanks Graham… good to know you still find it useful.
Thanks for this article, a refresher for me from when I worked with the Dept Agr Victoria.
Now in semi-retirement I’m using Bennett’s as a way to plan and evaluate a program in social change philanthropy. The similarities with agr extension are that we are wanting people to change attitudes and behaviors, on an evidence based rationale.
Is anyone aware of Bennetts being used in Social Change philanthropy?
Good to hear from you Ian and I hope semi-retirement is going well for you!
I’m sure Bennett’s approach could be applied to social change philanthropy, but I can’t say I’ve heard of any examples sorry.
I recall reading some material from The Rensselaerville Institute in the US many years ago. You might find some of their publications useful.
John, are you very familiar with using the Program Development side of Bennett’s hierarchy? I’m doing a retrospective analysis of a program, and still trying to wrap my head around how this might fit in the with the Program Performance side of the hierarchy…
Elya, ideally the program would have been developed using the hierarchy, as that then makes the evaluation so much easier.
Personally, I prefer to use a diagrammatic form of the Theory of change approach where you start with the desired longterm outcome. You then work backwards determining which outputs, activities (interventions) and resources are required. Identifying the indicators at this stage helps inform the ongoing monitoring and evaluation.
The key is asking why would a particular intervention change the system, and looking at related factors.
This is very similar to Rockwell and Bennett’s Targeting outcomes of programs approach.
Thanks John – and yes ideally these things are designed as such. But I’m working on a 5yrs post program (i.e. 5yrs restrospective) analysis… so I don’t have the luxury of working backwards..
Thanks for the reply – I’ll delve a bit deeper into the Theory of Change
My commiserations – trying to do the ‘monitoring and evaluation’ after the end of a program or project, when the desired outcomes and measures of success were not produced at the beginning, is always going to be questionable.
I am a company HR safety director. I report to my CEO from my evaluation that I offered 4 educational sessions in March for employees in our manufacturing plant. This is an example of which level of Bennett’s Hierarchy?
Hi Bob, that sounds like an Activity item to me, level 2.
“85% of attendees at the safety training sessions could correctly identify OSHA Safety Guidelines for operating a fork lift,” is an example of which level of Bennett’s Hierarchy?
That sounds like a Skill change to me, so Level 5.
I think that needs a bit more explanation – it is a knowledge change, not a skill change, and you got there by putting resources into an activity (educational sessions) that involved staff to get a particular reaction (increased knowledge of safety guidelines.
The big question is what did you expect them to actually do (practice change) in response to knowing the safety guidelines? And was that to reduce accidents or breaches of the guidelines (the outcome)?
It is a common problem in ‘extension’ that people carry out activities in isolation from the rest of the planning and operations hierarchy.
Well picked up Dave… it is indeed a knowledge change – apologies!
“Three years after establishing the new mandatory monthly safety training sessions, our annual lost time accident rates per year have reduced from 200 hours per year to 24 hours per year.” This is an example of which level of Bennett’s Hierarchy?
That sounds like a good example of Practice change to me, so Level 6.
Can I use the Bennett Hierarchy Model in evaluation of E-governance programs.
Yes indeed you can, as the same principles will apply.
“Three years after establishing the new mandatory monthly safety training sessions, our annual lost time accident rates per year have reduced from 200 hours per year to 24 hours per year.” This is an example of which level of Bennett’s Hierarchy?
Safety Analysis
Impact
Learning
Resources
Hmmm, what do you think Rick? Suggest an answer I’ll comment again.
Hi John. I am looking for suggestions on how to better capture “soft” evidence of level 5 (KASA) change. I have previously used an end-of-workshop survey with participants asked to self assess their level (1-5 scale) before and after the workshop. Questions have included terms such as understanding, ability, importance and confidence in relation to the workshop topic to try and distinguish between KASA segments. I found that the response rate dropped progressively through the series of questions which I have interpreted as meaning participants did not distinguish or understand the difference between the questions. I am considering trying to… Read more »
Hi Andrew! Great questions! We’ve had some success at incorporating KASA level questions into end-of-workshop surveys but not all the time. For example, the project team might decide that in August all activities/events being run will have a slightly longer feedback form that includes KASA level questions, but then they are dropped for Sept/Oct, and picked up again in Nov. This can work really well and can reduce survey fatigue. It also means you can be explicit and say ‘In August we’re keen to know more about how this project has had an impact, so for that reason, we’ve got… Read more »
As always, great piece of information. Evaluations are really something to be practices each step along the way of “practice change”, it’s unfortunate that we still have “Project Managers” leading projects that are all about “change” who have no idea of the psychology of adoption and think that installing technological tools is all that’s required. Keep up the good work John !
Thanks for your encouraging words Andres, and we too look forward to a change in attitude of some project managers!
I just love these logical frameworks, with justifiable steps that be challenged or unwrapped. The Rockwell and Bennet reference has been a mainstay to me for planning and carrying out operations, with a very clear system of monitoring and assessment that can be assigned to various levels of managers and program/project staff.
Then I can use methods like the community based social marketing to identify discrete activities and stakeholders, and the SELN policy tools to effect change.
Thanks Dave, I too like the Rockwell and Bennett reference and in particular their staircase analogy of going down the steps of Bennett’s hierarchy to develop a program and then going up the steps to assess its performance.