Are the Kirkpatrick Four Levels Outdated?
By Jim Kirkpatrick, PhD
A few weeks ago, Don Kirkpatrick, originator of the four levels of evaluation fifty years ago, was curious as to what learning professionals thought of them. In a demonstration of bravery, on the ASTD National LinkedIn group site, he asked, “Are the four levels outdated?” He received a large number of responses. I thought I would include a few in this article, and add some comments at the end.
Hi Donald - I definitely don't think it’s outdated. However, applying all levels across the new age technology enabled industry (ITES) that I work in seems a challenge. For me specifically to go beyond level 2 is near impossible due to the prevailing high attrition we face. Often times the results are skewed due to the above concern. We cannot even think of correctly measuring the ROI as well.
Looking forward to see how this discussion progresses.
Hello Donald,
The model looks at training from the outside, not from the learners' point of view, and in my opinion, that is the fallacy about it. It does not take into account the diverse learning strategies of different individuals. For many, an idea, after it germinates in the mind, it takes a little time to turn into a sapling and it is difficult to keep track of that mental process in the learner. I shall be happy with an “aha” at level 1 as I wouldn’t trust that the learner knows best and he will work out the strategy for harvesting his learnings.
Professor Kirkpatrick, your four level evaluation model may be 50 years old, but it is still the best, most concise way to look at the impact of training. This model remains the industry standard today, because it works.
It may be challenging to evaluate the effect of training on level four, results, but this is exactly what training and development professionals must do. It is more critical now than ever to show organizations the powerful impact that investing in employee development has on the bottom line.Dr. Kirkpatrick
The model is still useful but it reflects a 20th century model of "Top-down" and "Inside-out" thinking.
If we look at this from the learner's perspective, we would have a different model that is based on "Outside-in" and "Bottom-to-top"thinking:
1. How quickly can I learn?
2. Is the information credible?
3. Is the information applicable to me?
4. Did it help me accomplish what I needed to do?
5. Am I better off because I did it?
6. Will I be even more effective the next time?
I suggest a 21st century model (or at least a June of 2009 model would be:)
1. Results
2. Sustainability
3. Future effectiveness
Thank you for a great gift with your model. I do not mean to be unappreciative.Dr. Kirkpatrick,
Is it outdated? No. Is it effective in today’s learning environment? By itself...no. To determine whether there is training impact or not, certain behaviors have to take place. And these behaviors take place both before, and after the learning event itself. If you can influence those behaviors on both ends then you will see the results of training begin to work. Otherwise you can sit back and just expect the usual12-15% of training to stick.
While most of the comments were very supportive, they did echo the belief that while the four levels – reaction, learning, behavior, results – are still sound, their implementation has not been.
The first comment pointed out the difficulty in applying the four levels because of rapidly advancing technology, remote operations, and employee turnover. These do offer a challenge, but certainly not insurmountable. At SMR, we work with companies to actually leverage new technologies to streamline evaluation at all four levels in order to better assess remote employees and increase retention. The business partnership model seems to be the best method to get and keep training aligned with the needs and operations of the business.
The next two comments suggest that the four levels have been too trainer-centric. Unfortunately, the way they have been typically implemented makes that statement true. Along with that, the entire training industry has been self-serving. Consider the questions on a Level 1 reaction sheet. They are typically about how did we do rather than what was the reaction of the learner. Similarly, evaluation typically stops at level 2, as we believe our jobs are done when employees have learned.
Do you feel any of these questions relate to ones you ask yourselves? I suspect the answer is “yes”. If so, I encourage you to look at and inquire about the way SMR uses many of these suggestions to make sure that the four levels – and all of our models – focus on business results, and the critical behaviors that training graduates need to perform in order to bring about those results.
Here are some specific actions you can take to begin to create a powerful business partnership model, and increase the impact training brings to the bottom line:
- Sign up for an SMR Evaluation Program or other training-related programs. Click on the link to our websites to see what we offer. Note: our programs are very experiential, and designed so that you will walk away with tools and techniques you can immediately implement.
- Contact me or one of my SMR colleagues to set up a customized webinar to share with your team to link your needs with our models.
- Conduct an “Impact Study” going through all four levels with a high impact program you are about to roll out. Presenting your work as a compelling chain of evidence will go a long way to making a strong business case to your business partners.
- Contact us for more information about a customized in-house program to get your entire department moving ahead with this.
