The document discusses various models for evaluating the effectiveness of training programs, including the Kirkpatrick and Phillips models, and focuses on applying return on investment (ROI) analysis to evaluate the impact of e-learning initiatives. It provides details on calculating costs and benefits of training to determine ROI percentages and discusses challenges in isolating the effects of training on business results. The goal is to help training professionals communicate the value of learning interventions to business leaders using the language of financial performance and returns.
The document discusses various models for evaluating the effectiveness of training programs, including the Kirkpatrick and Phillips models, and focuses on applying return on investment (ROI) analysis to evaluate the impact of e-learning initiatives. It provides details on calculating costs and benefits of training to determine ROI percentages and discusses challenges in isolating the effects of training on business results. The goal is to help training professionals communicate the value of learning interventions to business leaders using the language of financial performance and returns.
The document discusses various models for evaluating the effectiveness of training programs, including the Kirkpatrick and Phillips models, and focuses on applying return on investment (ROI) analysis to evaluate the impact of e-learning initiatives. It provides details on calculating costs and benefits of training to determine ROI percentages and discusses challenges in isolating the effects of training on business results. The goal is to help training professionals communicate the value of learning interventions to business leaders using the language of financial performance and returns.
The document discusses various models for evaluating the effectiveness of training programs, including the Kirkpatrick and Phillips models, and focuses on applying return on investment (ROI) analysis to evaluate the impact of e-learning initiatives. It provides details on calculating costs and benefits of training to determine ROI percentages and discusses challenges in isolating the effects of training on business results. The goal is to help training professionals communicate the value of learning interventions to business leaders using the language of financial performance and returns.
Download as PPT, PDF, TXT or read online from Scribd
Download as ppt, pdf, or txt
You are on page 1/ 37
Training:
What is the Impact
on the Bottom Line?
Robert J. Gray QA 7504 September 22, 2001 1
Order of Presentation Introduction The Need to Evaluate Training from a Corporate Perspective What is e-Learning? The State of the e-Learning Business Evaluating the Effectiveness of Training Kirkpatrick Model Phillips Model
Applications of e-Learning Evaluation Using ROI
Conclusion
Introduction The New Corporate Realities of the Global Economy
International competition New technology Emphasis on productivity, performance, and return on investment Needs of the new learner are challenged New leaders with expanded competencies Rethinking, restructuring, and reengineering of education and training is required
Introduction Corporations spend millions of dollars for training Corporations are concerned about profitability Training departments are cost centers Training gets cut during expense reduction programs Imperative to quantify the value of training
The Need to Evaluate Training
from a Corporate Perspective $58.6B budgeted for training in 1997 Top challenge determining the impact of training on financial performance How are you helping the business?
The Need to Evaluate Training
from a Corporate Perspective The Language Barrier Training Focus Training content
Business Executive Focus Business results
Staying on budget
Maximizing profit
Building competencies The latest training techniques
Solving customer problems The latest competitive threat
The Need to Evaluate Training
from a Corporate Perspective Change of Focus Customers
What are they trying to
accomplish? Products What solutions do customers demand? Operational How do we ensure high level of Efficiency productivity? Measurement How do we track our performance, return, opportunities to improve? 7
The Need to Evaluate Training
from a Corporate Perspective Paradigm Shift From
To
Event oriented
Process oriented
Domain is the classroom
Domain is the organization
Focus on learners
Focus on workplace
Analyzes instructional needs
Analyzes performance needs
Develops training interventions Implements training solutions
What is e-Learning? Enterprise Learning? Electronic Learning? Everywhere Learning? Excellent Learning? Experience Learning?
What is e-Learning? Characteristics
Dynamic Up-to-date and relevant Real-time learning environment Highly available Flexible Learner control
10
State of the e-Learning Business Web-Based Education Commission Report to the President and the Congress Recommendations Make Internet available and affordable Provide training and support to educators and administrators New research framework on how people learn High quality online educational content Sustain funding e-Learning agenda
11
State of the e-Learning Business U.S. Corporate e-Learning Market 1997 - 2003E $11,415
Dollars in Millions
$12,000 $10,000
$7,113
$8,000 $6,000
$4,053
$4,000 $2,000
$234 $550
$2,222 $1,114
$0 1997 1998 1999 2000 2001 2002 2003 Year 12
State of the e-Learning Business e-Learning Projected Market Share 90% Percent Market Share
80% 70% 60%
e-Learning
50% 40%
Classroom Learning
30% 20% 10% 0%
1999
2003 Year 13
State of the e-Learning Business
Revenue in billions
$7.0
Growth of the U.S. Internet-Based Training
Market by Training Product $6.1
$6.0
$5.3
$5.0 $4.0
$4.1
$3.0
$3.1
$2.7
$2.0
$1.7
$1.0 $0.0
$0.9 $0.4 $0.1
1998
$0.2
1999
IT Training Soft Skills Training
$1.4
$0.6
2000
2001
Year
2002
2003 14
Evaluating the Effectiveness of Training Level 1 Reaction
Subordinates Identify Changes in Work Climate Identification of Other Influencing Variables Customer input
Factor out non-training variables
Influence on purchase decisions
18
Evaluating the Effectiveness
of Training: The ROI ProcessTM Converting Data to Monetary Values Tool
Purpose
Convert data to profit
contribution or cost savings Calculate cost of quality
Determine unit contribution to profit or
cost reduction Convert quality improvement into dollars
Use wages and benefits for the
value of time Historical tools
Convert time saved into dollars
Experts
Provide estimates
External databases
Research, government, industry estimates
Participants estimates
Learner estimates based in experience
Supervisors estimates
Supervisor estimates based on observation
Manager estimates
Manager estimates based on observation
HRD staff estimates
HR professionals estimates based on
observation
Can be used for specific variables
19
Evaluating the Effectiveness
TM of Training: The ROI Process Cost Calculation Elements Program design and development Program materials Instructor preparation and delivery time Facilities Travel, lodging, meals for participants Salaries and benefits Administrative and overhead costs Needs assessment and evaluation costs
20
Evaluating the Effectiveness
TM of Training: The ROI Process Formula 1: Cost/Benefit Ratio (CBR) CBR = Program Benefits/Program Costs Formula 2: Return on Investment Percentage (ROI%) ROI(%) = Net Program Benefits Program Costs
X100
21
Evaluating the Effectiveness
TM of Training: The ROI Process Soft Benefits Increased job satisfaction Increased organizational commitment Improved teamwork Improved customer service Reduced complaints Reduced conflicts
22
Evaluating the Effectiveness
TM of Training: The ROI Process Issues with the ROI ProcessTM Use of estimates Soft data Intangible benefits Use of industry studies and vendor information Ability to isolate training effects Lack of standards 23
Evaluating the Effectiveness
TM of Training: The ROI Process Caveat Emptor Estimating costs of training projects is a black art only slightly more precise than reading tea leaves or bird entrails. These spreadsheets offer a frameworkor just a framework for a frameworkfor estimating costs. The only thing we guarantee about them is that they are wrong for your particular project. But they can suggest ways you can create spreadsheets to estimate costs for your own project.
24
Applications of e-Learning Evaluation using ROI Internal Benefits Which of the following benefits does your company expect to gain from using e-Learning for training internally? Le ss trav el time and costs Better conve nie nce and se rvice M ore training about company products and procedures M ore access to company information Better-quality training Training outside work hours New custome rs New re ve nue source 0%
10%
20%
30%
40%
50%
60%
70%
Percentage response
80%
90%
100%
25
Applications of e-Learning Evaluation using ROI Early Returns When do you expect to see a return on investment from e-Learning?
More than 2 years
6% 1 year to 2 years 15%
Already using ROI
46%
6 months to 1 year 17% 3 to 6 months 8%
Within 3 months 8%
26
Applications of e-Learning Evaluation using ROI Cisco Systems Task Force Key Metrics Categories Satisfaction Penetration and Usage Effectiveness Relevance Cost
27
Evaluation using ROI
Effectiveness Measures Pre-performance and post-performance per offering With and without intervention performance Time to completion Time to competency Change in ability to perform job responsibilities against time spent learning
28
Applications of e-Learning Evaluation using ROI Do the Cisco measures evaluate ROI? Pre-test and post-test scores per offering NO Time to completion NO Time to competency NO Change in ability to perform job responsibilities against time spent learning YES 29
Applications of e-Learning Evaluation using ROI Six Significant Research Findings about e-Learning ROI impact analysis driven by clients funding projects Level 1 Reaction higher for classroom instruction e-Learning as effective as face to face for Level 2 - Learning Levels 1 through 5 can be applied to e-Learning ROI studies focus on cost reduction alone Evaluation can result in savings of money and time
30
Applications of e-Learning Evaluation using ROI Phillips - Issues with e-Learning evaluation Difficult to gather data from remote learners Lack of a framework for data collection Lack of standards Process models Operating standards Approaches to implementation 31
Conclusion ROI evaluation is in its early stages Use is somewhat limited Difficult to obtain data Use of soft data Lack of standardized methods and criteria Difficult to integrate data from many sources Companies are skinning their knees 32
Conclusion New evaluation methods Time to competency Time to market Time to understanding
33
Conclusion Success Stories Reduced injuries and damages Reduced time to train Reduced time develop new hires Reduced distribution time of training materials Reduced time to market Reduced certification time
Created online certification
programs Used website learning Used online discussion topics Taught sales skills and product knowledge online Developed online resource center Reduced training expenses 34
Conclusion e-Learning is growing rapidly An attractive solution to reduce costs Bottom-line benefits have not been conclusively demonstrated Need standard tools and methodologies
35
Conclusion Undeniably, e-Learning cuts the costs of travel, facilities, administrative overhead, duplication of effort, and more importantly, the opportunity cost of people away from the job in times of great need. There's no doubt that e-Learning can be rolled out fast. The time required to roll out a new product globally can shrink from months to hours. But all the cards aren't in yet. e-Learning is too new to have produced hard evidence of learning gains. e-Learning's top-line upside is speculative; its bottom-line savings are on 36 more solid ground.
