Phillips ROI Model: The 5 Levels of Training Evaluation (2024)

Throughout the first four levels of the Phillips ROI Methodology, data helps identify reasons for positive or negative ROI in the resulting fifth level. This model suggests that the final ROI of the training program is a result of a series of events:

• Transfer of knowledge and skills (in level 2)
• Application of learned knowledge and skills (in level 3)
• Business impact of training (in level 4)

Assuming the program clears level 1, each next level evaluated under the Phillips ROI Model checks if the change was a result of the program or various external factors. 

To show the calculation of ROI under Jack Phillips’ model, let’s reiterate our previous example of 50 software developers receiving training on a new programming language. 

Let’s put the use case through the Phillips ROI model and see how to calculate the ROI of this training program.

Consider these precautions before following the Phillips ROI Model for training evaluation

The Phillips Model presents a complex and sensitive process. Make sure the following conditions are met before you put the process to use:

• Conduct a needs assessment for your training program. Don’t use the ROI method when no needs assessment data is available.
• Include one or more strategies to isolate the training effect. 
• Base the estimates only on the most reliable and credible sources.
• Calculate costs and benefits with a conservative approach. 
• Don’t compare the training ROI with other financial returns until it’s absolutely essential.
• Don’t isolate the management from the ROI calculation process.
• Be cautious in deciding what’s measurable and what’s not. Take input from business teams on inclusion/exclusion of sensitive factors.

Step 1 – Gather data before the training starts

Before software developers start their training, they fill out a questionnaire to assess any trends over 6 months until the start. The questionnaire also examines their pre-training knowledge in concepts related to the new programming language. This will help understand how the training program helped different types of learners.

Step 2 – Gather data after the training ends

Post-program data comes from performance records of the company and/or through follow-up questionnaires of the following types:

• Participants fill questionnaires 
• The company assesses their performance 
• Subordinates, peers, and supervisors/seniors of participants fill in observational surveys 
• Other relevant internal/external groups.

Collect this data for all levels of the model as required. 

Step 3 – Isolate effects of lessons from the training program

The Kirkpatrick Model assumes improvements come from the training program. The Phillips ROI Model attempts to find other causes for training results including the training impact. Some of the possible factors that contribute to improved post-training performance include:

• Recent performance bonuses
• Competitive conditions in the company/department (for example, were there any impending promotions after the training program?)
• Any marketing/employer branding programs
• Any other seasonal/non-seasonal programs that offered incentives to participants

To isolate the effect of the training program, some techniques include:

• Trend line analysis
• Control groups
• Forecasting models
• Training impact estimation from participants, supervisors, experts, and customers.

Step 4 – Extract monetary benefits of the program from the data

After filtering the data and isolating training program effects, the monetary dimension lets us move one step closer to ROI.

In the example of training on a new programming language, we assume a profit of $5,800 per developer from the new project that employs 50 trained developers. This profit margin considers the effect of only the training program; overall profit figures may be higher due to other factors

Total profit from the training program = $5,800 / developer X 50 developers = $2,90,000.

Step 5 – Add up all program costs

This is a straightforward calculation that brings together all costs for the training program. The total program costs will be a summation of the following categories:

• Costs to design, develop, and administer the program (including payment to training staff and their travel, lodging, and miscellaneous expenses)
• Training venue costs
• Training material costs (physical paper, electronic hardware, training, and evaluation software, and other administration costs)
• Travel, lodging, meal, and miscellaneous costs of participants
• Salary of participants and their employee benefits for the time they will devote to training
• Other administrative costs incurred during the training program.

In the example of training on a new programming language, let’s assume the total program costs were $2,34,900.

Step 6 – Calculate the return on investment (ROI)

ROI (%) = (Net benefits of the training program / total program costs) X 100

  = (($2,90,000 – $2,34,9000) / $2,34,900) X 100

  = ($55,100 / $2,34,900)  X 100 = 23.457%

Step 7 – Find the intangibles

A training program will have more than monetary benefits alone. Such intangible benefits are often more valuable than the hard data coming out of the ROI calculation process.

You can identify such abstract measures at any stage of the ROI calculation process through discussions with the project’s sponsors. 

• during the needs assessment 
• when you convert data into monetary values (step 4 in this section)
• in a separate follow-up evaluation. 

Each type of program will have its own type of intangible benefits. The table below mentions the common categories.