Kirkpatrick
The Four Levels
s Reaction s Learning s Behavior s Results
All about Kirkpatrick
In 1959, Kirkpatrick wrote four articles describing the four levels for evaluating training programs. He was working on his dissertation for a Ph.D. when he came up with the idea of defining evaluation. Evaluation, as according to Kirkpatrick, seems to have multiple meanings to training and developmental professionals. Some think evaluation is a change in behavior, or the determination of the final results.
All about Kirkpatrick (continued)
s Kirkpatrick
says they are all right, and yet all wrong. All four levels are important in understanding the basic concepts in training. There are exceptions, however.
Kirkpatrick: Evaluating Training Programs
s What s How s How
is quality training? do you measure it? do you improve it?
Evaluating
The reason for evaluating is to
determine the effectiveness of a training program. (Kirkpatrick, 1994, pg. 3)
The Ten Factors of Developing a Training Program
1. 2. 3. 4. 5. Determine needs Set objectives Determine subject content Select qualified applicants Determine the best schedule
The Ten Factors of Developing a Training Program
6. Select appropriate facilities 7. Select qualified instructors 8. Select and prepare audiovisual aids 9. Co-ordinate the program 10. Evaluate the program
Reasons for Evaluating
Kirkpatrick gives three reasons why there is a need to evaluate training: 1.To justify the existence of the training department by showing how it contributes to the organizations objectives and goals.
Reasons for Evaluating
2. To decide whether to continue or discontinue training programs. 3. To gain information on how to improve future training programs. (Kirkpatrick, 1994, pg. 18)
The Four Levels
s Reaction s Learning s Behavior s Results
The Four Levels represent a sequence of ways to evaluate (training) programs.As you move from one level to the next, the process becomes more difficult and time-consuming, but it also provides more valuable information. (Kirkpatrick, 1994, pg. 21)
Reaction:
is the measuring of the reaction of the participants in the training program. is a measure of customer satisfaction. (Kirkpatrick, 1994, pg. 21)
Learning:
is the change in the participants attitudes, or an increase in knowledge, or greater skills received, as a result of the participation of the program.
Learning
The measuring of learning in any training program is the determination of at least one of these measuring parameters: s Did the attitudes change positively? s Is the knowledge acquired related and helpful to the task? s Is the skill acquired related and helpful to the task?
Behavior
Level 3 attempts to evaluate how much transfer of knowledge, skills, and attitude occurs after the training.
The four conditions Kirkpatrick identifies for changes to occur:
s Desire
to change of what to do and
s Knowledge
how to do it
s Work
in the right climate for (positive) change
s Reward
When all conditions are met, the employee must:
s Realize
an opportunity to use the behavioral changes. s Make the decision to use the behavioral changes. s Decide whether or not to continue using the behavioral changes.
When evaluating change in behavior, decide:
s When s How s How
to evaluate
often to evaluate to evaluate
Guidelines for evaluating behavior:
s Use
a control group s Allow time for change to occur s Evaluate before and after s Survey/interview observers s Get 100% response or sampling s Repeat evaluation, as appropriate s Consider cost versus benefits
Results
Level 4 is the most important and difficult of all - determining final results after training.
Evaluation Questions:
Increased production? s Improved quality? s Decreased costs? s Improved safety numbers? s Increased sales? s Reduced turnover? s Higher profits?
s
Guidelines for evaluating results:
Use a control group. s Allow time for results to be achieved. s Measure before and after the program. s Repeat the measurements, as needed. s Consider cost versus benefits. s Be satisfied with evidence if proof is not possible.
s
Case Study #1
INTEL CORPORATION
Intels Compromise of the Kirkpatrick Model
Intel uses the four-level model as an analysis instrument to determine the initial training needs and design of its training program; as well as using the model for evaluations.
Intels Compromise of the Kirkpatrick Model
Their uniqueness of using the model is in the fact that the designers of the training program worked backwards in the analysis of the training, starting with Level Four.
The Model
This implementation of the Kirkpatrick Model stands as vivid testimony to the versatility of the model as a training tool, and in developing fledgling training programs.
The Model
It also reflects the open-mindedness of the senior executives at Intel for their infinite use of the model and the use of the genius and visions of Kirkpatrick.
How Intel applies the analysis to their training program
Level Four Determine the organizations structure and future needs. Level Three. Change the environmental conditions and employee conditions to improve business indicators.
How Intel applies the analysis to their training program
Level Two. Design a training program that would ensure a transfer of deficient skills and knowledge. Level One. Use a questionnaire, according to their skill level, that would instruct and inspire training participants.
How Intel applies evaluation to their training program
Level One - Questionnaire. Level Two - Demonstrate competency, create action plans through group simulations. Level Three - Follow-up to determine if action plans were met (specific steps to implement concepts of what was learned). Level Four - Ongoing process of tracking business indicators.
Case Study #2
ST. LUKES HOSPITAL
St. Lukes is unique s Evaluation
of outdoor-based training program, not classroom. s Results analyzed statistically to determine the significance of any change. s Evaluation led to recommendations for future programs.
The New Questionnaire
Used before attendance in the program. s Used 3 months after completion of the program. s Used again 6 months after completion of the program.
s (Communication showed statistically significant improvement, and Group Effectiveness showed statistically significant change.)
Kirkpatricks 4 Levels of Evaluation are:
Level 1 - Reaction: how participants reacted to the program. s Level 2 - Learning: what participants learned from the program. s Level 3 - Behavior: whether what was learned is being applied on the job. s Level 4 - Results: whether that application is achieving results.
s
Post-test Questions
(1) Name three ways evaluation results can be measured. (2) Do all 4 Levels have to be used? (3) Do they have to be used in 1,2,3,4 order? (4) Is Kirkpatricks method of evaluation summative or formative? (5) Which developmental view does Kirkpatrick use? (discrepancy,
IF YOU THINK TRAINING IS EXPENSIVE, TRY IGNORANCE. and, remember, the definition of ignorance is repeating the same behavior, over and over, and expecting different results!
