For many years, the idea of calculating ROI in corporate training was considered as valid and modern. At first glance this idea makes sense: if all HR gurus want HR professionals "to speak the language of the business" than what is more "businesy" than bringing the idea from the world of accountants and bankers and implement in the HR realm?
Well, while we as professionals are all for using metrics that connect HR practices with business goals and objectives, in this article we try to explain why ROI and Kirkpatrick model of learning evaluation have better and more useful alternative which is more aligned with the trends of digitalisation and transparency.
ROI? The Kirkpatrick Model? Not so fast!
The problem is not in the Kirkpatrick model per se. Any model of learning evaluation - Kirkpatrick-Phillips, Bern, Tyler etc - CAN NOT evaluate how money spent on training impact performance. They are not useless and based on common sense - they just can't establish a link between what employees learned and impact on business.
That's why in HR textbooks we can find something called "the black box phenomenon".
Often HR books present HRM inputs then HRM outputs (or just something like "actually we don't know what happens here") somehow linked to "Firm performance".
A good example is a textbook of Michael Armstrong, now on its 15th edition. The author can't answer a very simple question of how HRM practices impact performance. Yes it is still a black box, as you can see in the picture.
70 years ago, when Kirkpatrick presented the idea of measuring learning, this black-box approach was working. Today we should get rid of it.
The reason is simple: most companies have everything they need to have more valuable results:
- Data (lots of data),
- Model of Human Capital Evaluation, developed by CIPD and CIMA,
- Methods of Applied Statistics.
This is a more modern and useful approach and most companies already have enough data to use it.
Data
Digitalisation allows to collect a lot of data on employees. Although there should be constraints in what data should be harvested, when it comes to learning the more data us better. Microlearning and spaced learning allow to track every step any employee takes to master a competency.
This allows the data on how competencies are developed over time. This information can not be monetised, however this data is what called output in CIMA/CIPD model.
CIMA/CIPD model
A bit about two organisations that initiated the framework that can be used for learning evaluation.
- CIMA is a UK based professional body offering training and qualification in management accountancy and related subjects.It is the largest management accounting body in the world with 115,000 members and 6,500 CGMA students in 2020.
- The Chartered Institute of Personnel and Development (CIPD) is a professional association for human resource management professionals. It has more than 150,000 members worldwide.
Not a long ago these two organisations joined efforts and developed Human Capital evaluation model using the International Integrated Reporting Framework. In the Framework Human Capital is one of six types of capital companies should prepare for Integrated Reporting. The model has four levels:
- Inputs,
- Activities,
- Outputs,
- Outcomes.
For us, the most interesting part are Outputs.
Data collected in the learning process are outputs that are links between investment in learning and business results.
Similar outputs can be obtained when L&D and HR professionals provide compliance training, for example AML in banks and insurance companies and work safety in manufacturing companies. In this case we can link training effectiveness with performance by analysing how much was saved by the organisation for not paying fines or reducing workplace incidents. In this case lower number of incidents and fines are "outputs" that impact performance.
And now about statistics
Correlation is a well-developed notion in statistical analysis. Basically, correlation measures the degree to which two variables move in relation to each other.
Correlation is the measure of how two or more variables are related to one another.
If X is a progress in developing Leadership capabilities (output) and Y is a one of the Organisation performance metrics (outcome) then we can establish if these two variables are dependent.
Essentially data allows to understand if there is a correlation between learning and organisational effectiveness.
Deciphering the black box of corporate learning
So, yes, HR professionals can decipher the black box with the following sequence:
1) Identify the required professional standard.
2) Develop a learning programme that covers knowledge and skills, described in the standard.
3) Identify and codify behaviours (for example in the form of questionnaires).
4) Conduct learning and evaluation.
5) Measure change in behaviours on the job.
6) Using the data on how the knowledge gained was understood and applied, measure progress in skills and competencies.
7) Calculate the correlation coefficients between this progress and KPI and/or profit.
This sequence is simple but not easy and it is actionable.
Really, ROI is a bit outdated in corporate learning.