n
o g
i
t
n
i
a
n
u
i
l
a
a
r
v
E fT
Chapter # 7
o
What is Evaluation?
Evaluation is systematic continuous process of
investigating, monitoring and supervision of
preplanned activities to ensure that everything is
going on as per desire. It is process of
determining the effectiveness of training.
It is a program which undertaken to see
whether everything is done according to
predetermined
organization
plans,
policies and strategies.
It is the function of organization manager.
What is Training Evaluation?
Accessing the effectiveness of the training
program in terms of the benefits to the trainees
and the country.
Process of collecting out comes to determine if
the training program was effective.
From whom, what, when and how information
should be collected.
Objectives of evaluation
There are different objectives for evaluation, those
are very important and necessary of a successful
organization, discussed in below:
Explain why evaluation is important
Identify and choose outcomes to evaluate
Identify how to measure outcome
Discuss validity threats and experimental designs
Discuss issues to consider in evaluation design to
improve the ability to make inferences.
Contd.
1) Primary Objectives:
To collect data that will serve as a valid basis for
improving the management training system and
maintaining quality control over its components.
1) Secondary Objectives:
a) To ensure that its instructor is conducted in a manner
consistent with the system as planned and designed.
b) To provide a basis for instructor in service
management training and upgrading.
c) To provide data which are usable in effecting
revisions of the instructor for training course.
Contd.
d) To increase the effectiveness of the training program
while it is going on.
e) To increase the effectiveness of the management
training programs to be held time.
f) To help participants to get feedback for their
improvement and to increase their efficiency.
g) To find out to what extent the management
objectives are achieved (out come of training)
Contd.
Some more objectives of training are:
i.To develop human resources for mechanization- an
indispensable condition for increasing the agricultural production.
ii.To increase awareness about judicious use of available
machinery and power.
iii.To transfer the proven technology from research institutes to
the actual users.
iv.To help farmer / technicians / engineering students / extension
workers etc. in the mechanization.
v.To upgrade social and economic condition of their rural youth
by generating the employment further.
vi.To help industries especially small sector in transplanting the
research design of tested equipment / machines into industrial
design for the production of quality machines.
vii.To encourage the energy conservation in agriculture through
various training programs.
Purpose of the Evaluation
The purpose of evaluation is to determine whether or not
the training achieved the desire objective.
An evaluation strategy builds Organizational capability
to increase the performance and business value of training
investment. - Stated purpose, May 24, 2006 (
www.designedtoinspire.com)
The purpose of the evaluation can be apportioned into
two assortment:
1.Summative evaluation: collecting data to access
learning and other criteria.
2.Formative evaluation: collecting data to access how to
the make the program better.
Criteria for Evaluation
Criteria should be based on training
objectives- all objectives should be evaluated.
Criteria should be relevant (uncontaminated on
deficient), reliable, practical and they should
discriminate.
Criteria should include reactions, learning
(verbal, cognitive, attitudes), results and ROI
Criterion deficiency, relevance and contamination
Outcomes Related to Training Objectives
Methods of Training Evaluation
There are several methods for giving training to the managers
and other workers of the organization. They are as follows:
1. Observation
2. Ratings
3. Trainee Survey
4. Trainee Interviews
5. Instructor Survey
6. Instructor Interviews
7. Reaction
8. Learning
9. Behavior
10. Results
11. Before & after test scores
12. Before & after performance
measure
13. Experimental & control
groups
14. Delayed treatment of control
groups
15. Modified critical incident
method
16. Over the shoulder evaluation
Contd.
17. Performance analysis
18. Time series evaluation
19. Cost benefits analysis
20. Organization development approach
21. Job related approach
22. Training centered approach
23. Learning centered approach
24. Managerial approach
25. Complaint files consultation
26. Participants enjoyment of program
27. Volume of material learnt
28. Achievement of training goals
29. Assessing the benefits of training
Evaluation Method
To create a favorable climate for learning , creature comforts such as
good food and accommodation may help. But if you want to be in the
Good Learning Guide as well as in the Good Food Guide
you will also want feedback on some of the questions listed below:
Evaluation methods are tools for change. If they are simply used as score
sheets or as pats on the back, evaluation tools can lead to complacencyrather than to change and innovation. End of course feedback can play an
important part in designing and evaluation strategy-but it is by no means
the only part!
When to evaluate? This page lists some of the ways in which you can
generate rich insights, creative ideas, evaluative discussions and other
useful responses at or near the end of a course (summative evaluation).
Many of these ideas can also be used for formative mid-course evaluation.
See the article below on Facilitated Student Feedback for some
detailed advice on formative evaluation.
Issues in Evaluation: You will also notice plenty of messages (in the lefthand column) that will remind you of some of the wider issues that are
relevant to end of course evaluation and to evaluation as a whole.
Evaluation of Training
1. Trained or untrained
2. Subjective or not
3. Sincerity
4. Timeliness
5. Strictness
6. Use of modern aids
7. Relations with course outline
8. Relations with course objectives
9. Supply of reading materials
10. Supply of necessary tools for training
Kirkpatricks four levels of training evaluation
This grid illustrates the basic Kirkpatrick structure at a glance.
Level
1.
2.
3.
4.
Evaluation type
(what is measured)
Evaluation description and
characteristics
Example of evaluation tools and
methods
Relevance and practicability
Reaction
Reaction evaluation is how the
delegates felt about the
training or learning
experience.
Happy sheets, feedback forms
Verbal reaction, post training
surveys or questionnaires
Quick and very easy to obtain
Not expensive to gather or to
analysis
Learning
Learning evaluation is the
Typically assessments or
Relatively simple to set
measurement of the
tests before and after the
up; clear cut for
increase in knowledge
training Interview or
quantifiable skills Less
before and after
observation can also be used. easy for complex learning
Behavior
Behavior evaluation is the
extent of applied learning back
on the job implementation
Observation and interview over
time are required to assess
change, relevance of change and
sustainability of change
Measurement of behavior
change typically requires
cooperation and skill of line
managers
Results
Results evaluation is the effect
on the business or
environment by the trainee
Measures are already in place via
normal management systems and
reporting the challenge is to relate
to the trainee
Individually not difficult;
unlike whole organization.
Process must attribute clear
accountabilities.
Successful Evaluation Tips
1
Reaction
Measurement
focus
Trainees
perceptions
What did trainees think of
this training?
2
Learning
Knowledge/
skills gained
Was there an increase in
knowledge or skill level?
Level
3
Worksite
Behavior implementation
4
Results
Impact on
organization
Questions addressed
Is new knowledge / skill
being used on the job?
What effect did the training
have on the organization?
Example to the mentioned tips
Level 1: One way to assess trainee reactions
and attitudes is to use a questionnaire.
Level 2: Written or performance tests can
assess change in knowledge / skills.
Level 3: Post-training testing or observations
can assess use of skills on the job:
Level 4: Quantifiable measures are often used
when assessing organizational impact.
Evaluation of participants:
Important to evaluate participants are some
reasons, these are:
1.Regularity
2.Attentiveness
3.Preparation of assignment
4.Preparation of home tasks
5.Participation in discussion
6.Participation in brain storming
7.Contributing new ideas / thoughts
8.Use of training time
Benefits of Training Evaluation
1. Knowledge of results
2. Determining the level of training groups
3. Determining the efficiency of training institution
4. Determining level of training success of
participants
5. Deciding the training results
6. Determining the subjectivity and objectivity of
training courses
7. Follow-up of trainees toward organization
developments.
The evaluation process
Figure: The process of evaluation of training
Thank
you
