ROI - WhitePaper
ROI - WhitePaper
ROI - WhitePaper
Table of Contents
Introduction ................................................................................................................................................. 2 What is ROI measurement for Training? ........................................................................................................... 3 ROI as process ......................................................................................................................................... 3 ROI as Perception ..................................................................................................................................... 3 Benefit and Cost Aspects of ROI ................................................................................................................. 4 Why do we want to measure ROI? .................................................................................................................. 4 Show me the value ................................................................................................................................ 4 Alignment with significant goals.................................................................................................................. 5 ROI in Human Capital vs. Non-Human Capital ROI ....................................................................................... 6 Inhibitors to ROI measurement .................................................................................................................... 6 Does ROI have any Value, anyway? ........................................................................................................... 6 Measuring training effectiveness ..................................................................................................................... 7 Criteria for an effective ROI Process ................................................................................................................ 9 The Process Model ........................................................................................................................................ 9 Identify the Returns .................................................................................................................................... 9 Identify the Investment ............................................................................................................................. 10 Step 1: Create an ROI Measurement Plan .................................................................................................. 10 Step 2: Collect Data................................................................................................................................ 11 Step 3: Isolate the effects of training .......................................................................................................... 12 Step 4: Convert data to monetary value ..................................................................................................... 12 Step 5: Calculate the ROI ........................................................................................................................ 13 Scenarios .............................................................................................................................................. 14 Sources of Data and Data Collection methods................................................................................................. 15 Sources of data can include: .................................................................................................................... 15 Data Collection methods .......................................................................................................................... 15 Appendix 1: factors in Return and Investment .................................................................................................. 17 Return (Benefit) factors ............................................................................................................................. 17 Investment (Cost) factors .......................................................................................................................... 17 Appendix 2 - Other ROI Models ................................................................................................................... 19 Stufflebeam CIPP.................................................................................................................................. 19 Kaufman's Five Levels of Evaluation ........................................................................................................... 21 CIRO (context, input, reaction, and outcome) .............................................................................................. 22 Alkins UCLA model ................................................................................................................................. 23 References ................................................................................................................................................. 24
Introduction
This paper has been prepared to provide to members of CEdMA some basic introductory information and ideas to assist in the preparation of their own customised ROI processes. CEdMAs clients are typically purchasers of IT software and hardware, and CEdMA members are responsible for the provision of training to these clients. The question this paper addresses is How do we help customers understand and justify for themselves the need to invest properly and comprehensively in training, and how do we present the comparative benefits of different approach es to training? Implementing some form of measurement process is as important for managing training programmes and investment, as it is for any other project requiring significant financial investment by a business. Training programs consume resources (ie they take peoples time and cost money), but they are also critical to maximising the return on investment in other programs or products (e.g. the effective introduction of software systems requires users to be able to use the systems effectively if the potential benefit of the software systems is to be realised in practice), as well as generally improving the productivity of the workforce. If ROI is to be successfully managed and measured, it is important that the process be included early in the planning cycle for the training programs. The purpose of this paper is to provide a background to Return on Investment (ROI), and to document a simple approach to predicting or calculating the Return on Investment, in financial terms, from training programmes. This paper is not intended to be a detailed analysis of ROI statistical measurement techniques, but a tool on which you can start to build simple predictors or measurement tools, and then make them as sophisticated as you wish. It should also be understood that ROI from training cannot always be seen in isolation from other sources of ROI. It is often the case that an overall project ROI measurement has to be taken, and the evaluation methodology must allow the customer to identify the interconnected and relative values of the various sources of benefit. For example, in a project to upgrade software, the ROI can come from a combination of factors: The user and their organisation will see increased productivity purely from use of new and enhanced functionality in the hardware and software or better performance of the application or system. This is not enough, however, and must in turn be enhanced by Reducing as much as possible the time taken to achieve these benefits, and achieving further increases in productivity due to training on the use of the new functionality (compared to the productivity the users might experience without the training). And in some cases Avoiding or minimising a negative productivity impact due to the typical decrease in productivity during the early days or weeks of using new software (as people have to learn new and changed facilities in order to regain their previous level of performance) In cases like this, the training not only has its own pure ROI (i.e. the increased produc tivity of the trained user, compared to the untrained user), but it is also a key component in realising the ROI of the software implementation or upgrade. It is up to you and your client to decide how to separate these factors or whether to roll them all together. As well as demonstrating how training can affect the success of software or hardware implementations, the ROI exercise may also be used to evaluate the different approaches to training.
3
For example, an ROI exercise may show some comparisons of expected costs and benefits from purely classroom training, from purely e-learning, or from a blended approach.
ROI as process
ROI measurement is the process of collecting and analysing this performance data, and translating this into a measurement of real financial benefit to the organisation. This benefit is then compared to the cost of creating this benefit through training and measurement. In many cases, ROI measurement can be linked to data collected and analysed for the purpose of Training Needs Analysis (TNA). If detailed TNA studies are done prior to the training, the data from these studies can be compared to the feedback and performance data acquired after the training takes place. In addition, the TNA is likely to highlight the expected benefits and results from the training. In this case, the change in performance may be more accurately determined.
ROI as Perception
So, what actually is ROI on training? It can be considered to be a perception on the part of the client of how valuable the training has been in achieving their perceived goals; and these perceptions will vary depending on whom you talk to. For example: The Board may see a big picture of how the training affects th e companys ability to achieve its corporate goals The finance department may be looking to see how training stacks up financially against other ways to invest the companys money, and whether the training, as carried out, was financially more effective than alternative forms of development The business unit manager may be solely concerned with the impact on performance and productivity in achieving the goals of their department The training and development manager may be concerned with the impact training programmes are having on the credibility and status as the training function within the company and its ability to secure investment in the future to drive further business performance enhancements With all these potentially different viewpoints, one of the first things you need to consider with your client, is what the client actually considers is a return on investment, and which views of success are critical to the measurement process. Hopefully, there will be a balance of these viewpoints, leading to an overall value judgment based on actual measured results.
5
and much more damaging, than the cost of training. For example, various studies in various situations and circumstances have found some interesting statistics: Untrained users take up to six times longer to perform the same tasks. o (From TBA) Training enhances employee retention. A Louis Harris and Associate Poll says that among employees who say their company offers poor or no training, 41% plan to leave within a year. Of those that say their company offers excellent training, only 12% say they plan to leave. Studies show that in-house training costs 73% more than outsourced training. o (from TBA) A four-year study by the American Society of Training and Development shows that firms who invest $1500 per employee in training compared to those that spend $125, experience on average: 24% higher gross profit margins and 218% higher income per employee! Just a 2% increase in productivity has been shown to net a 100% return on investment in outsourced, instructor-lead training o (from Training ROI. Avatech Solutions.)
These examples are all well and good, but these are generic and non-specific indicators; how well do these apply to any specific customer situation, or to a specific industry? If a customer is going to make a decision to invest in training, they may want more specific supporting evidence than this; hence the need to be able to offer a pre-training ROI analysis to give the customer some comfort that this might in fact work in their situation.
One of the keys to gaining commitment has to be to allay these fears and concerns, by demonstrating a process that can be carried out, and by taking a realistic approach to the costs and overheads, and the expectations of the client. In the right situations, the insights into the effectiveness of the training can provide important strategic benefits to the company such as: Measuring contribution of HRD Setting priorities Focusing on results Altering managements appreciation and perceptions of training
By carrying out some pre-training ROI analysis, it should be possible to demonstrate that the there is a likelihood of a positive ROI on training, and thus allay some of the fear, as well as encourage the investment in the training.
7
For access to the full text of this article, see the references section below; this is the conclusion: My experience has shown that most senior executives have more faith in gut feeling, than in numbers. The numbers are input, but the decision is broader than that. Results from an Information Week survey reveal that "More companies are justifying their e-business ventures not in terms of ROI but in terms of strategic goals. Creating or maintaining a competitive edge was cited most often as the reason for deploying an application." To some people--me included--the traditional concept of training ROI is obsolete. Astute training managers employ business metrics, not evaluation levels, I believe. Business unit managers value time more than ROI. Major decisions are based on descriptive business cases, not pro forma budgets. Senior executives tend to be more interested in the top line (dramatic growth from new markets and innovation) than the bottom line (the accounting fiction of profits). The Net has changed everything. So, you need to take into consideration some of these other factors that are extremely difficult to quantify, yet have a significant impact on the perceived success of training and learning; good luck!
Evaluation Levels (modified from the Kirkpatrick 4-level model) Evaluation Description Characteristics Level Measuring Reaction and Identifying Measures participants reaction to the Level 1 Planned Actions program, and outlines specific plans Did they like for implementation of learning to the it? job. Level 2 Did they learn? Level 3 Do they use it? Measuring Cognitive Learning and Retention Measures skills, knowledge, or attitude changes as a result of the training. Measures actual changes in behaviour on the job, and specific applications of the training material.
8
Level 4 Did it impact the bottom line? Level 5 What is the return on learning investment? Identifying business results from the training Measures the business impact of the training. (eg measures changes in output, quality, costs, time, productivity or other business metrics) Compares the monetary value of the results with the costs for the program.
In most organisations, measuring training effectiveness at all Levels 1-4 is not feasible for all participants and for all projects. In practice, the organisation must set targets for the scope of evaluations at each level, and for the critical training courses or programmes which have most importance to the success of the organisation. Typical targets for measurement activities at each level are: Level Level Level Level Level 1: 2: 3: 4: 5: 100% of participants/courses provide effectiveness data 50-70% of participants/courses provide effectiveness data 30-50% of participants/courses provide effectiveness data 10-20% of participants/courses provide effectiveness data 5-10% of actual ROI is measured, and extrapolated to the overall program.
From this, we can see that most organisations will initially only fully calculate ROI in detail on a sample of the courses and participants. It is therefore essential to identify those courses and participants who are: Representative of the overall population Taking part in important or high impact programs Able to accurately assess the impact of training on their jobs Amenable to providing the full depth of data required Note: This model describes ROI as level 5; please bear in mind that it is not necessarily appropriate to carry out detailed analysis at all levels up to this, in order to derive an ROI; it may be enough for the client to know that an 80% pass mark on a technical test (Level 2) is good enough to justify the investment, assuming some numerical value can be put on the passing of the tests. During the lifetime of the training programme and evaluation, the data can be collected at various times immediately after training, Level 1 feedback should be easily accessible within a reasonably short time, it should be possible to measure the level 2 performance change it will take longer for level 3 data to be meaningful, as this typically requires some period of practical experience, applying the learning to the job, and gaining some fluency with the skills learned Once level 3 data can be reasonably collected, level 4 data can be either derived from the level 3 data immediately in some cases, but in other cases it will require a further waiting period to assess the new level of business operations performance, especially where the new levels of business performance can only be measured after the new skills have been applied for some time (eg a helpdesk technician may be able to apply new customer care skills after
9
some practice (level 3); but it may take some further time before the impact of these new skills on customer satisfaction can be gauged (level 4)). Note: there are other models for describing ROI (see Appendix 2), such as: Kaufman's Five Levels of Evaluation The CIRO (context, input, reaction, and outcome) Approach Stufflebeams's CIPP (context, input, process, and product) Model Alkins' UCLA Model
10
Beware that, sometimes, benefits can be hidden; for example, consider the business benefits of certification for regulatory compliance in the ROI calculation; it may be that compliance is seen as a pure overhead cost of doing business; but in this case, training can reduce the cost of compliance (and the costs of non-compliance), and provide benefits in enabling new business activities, or better quality of customer service resulting from the compliance-training etc etc. Examples of such factors may be How can you quantify competitive advantage resulting from a more highly trained and competent workforce, resulting in shareholder value? o Competitive advantage is probably seen differently by every organisation; what constitutes competitive advantage may perhaps be better seen as a compilation of factors, such as quality, cost, customer satisfaction; but there may be intangible elements such as company image in the market which amounts t o more than the sum of the individual components of competitive advantage. How can you quantify the value of improvement in managers long term decision making resulting from improved techniques in situational analysis and decision making? o This may require some quite sophisticated analysis; maybe this can be determined by offering simulation exercises, or role/game playing, and deriving a score from that.
Note: you should try to estimate beforehand what a reasonable ROI is. Carrying out an expensive exercise to show a 300% ROI is great; unless comparable training exercises in the clients industry for equivalent training typically show 750%! Or even better, how does the client typically view 300%? Is this good in their circumstances, or is it lower than their typical ROI as determined from previous training?
11
The Data Collection Plan State the objectives of the training / learning State the objectives of each phase of data collection at each evaluation Level Identify any previously used metrics, values or methodologies used by the client, and determine their suitability for the current exercise Select the appropriate evaluation methods Identify the audiences who will be surveyed for data i. The learner ii. The manager (focussed on levels 3 and 4) iii. The analyst (for analytically determined performance-change data based on available performance measurement data) Set the timing for the data collection i. Pre-training data collection (benchmarking the current situation) ii. Post-training data collection (comparative data (apples to apples comparison)) 1. immediately post-training (initial reaction and assessment) focussed on levels 1 and 2 2. at a later date when the effect of the training has had time to make itself felt in the learners job performance focussed on levels 3 and 4 Allocate responsibilities for data collection and analysis The ROI Analysis Plan (This is a continuation of the Data Collection Plan, capturing information on the key items needed to develop the actual ROI calculation.) List Significant Data items (usually Level 4 or 3) to be collected i. Benefit Factors ii. Cost Factors Methods to isolate effects of the learning/training from other influences Methods to convert data to numerical values Intangible benefits Other influences Communication targets
12
4. Carry out the data collection at the levels 1-4 indicated above
13
Cost of quality calculated and quality improvements converted to cost savings or increased profitability Cost savings (salaries and overheads) in reductions in participants time in completing projects. Internal or external experts may be able to estimate the values of the performance improvements gained. Participants or their supervisors/mangers can estimate the cost savings or value of increased productivity
Having calculated the direct financial value of the performance enhancements, it is also necessary, wherever possible, to estimate the value of the more intangible benefits, such as: Increased job satisfaction, and the benefits of increased staff retention and reduced recruitment costs Increased organisational commitment Improved teamwork Improved customer service Reduced problems and complaints Reduced conflicts To calculate the cost of the training program, ensure you include: Cost of external training services purchased Cost of training materials supplied Cost of internal training staff involvement Cost of facilities used for the program Travel, accommodation and other incidental costs Salaries and benefits of the participants Administrative and overhead costs of the training function. The costs of carrying out the ROI on the training program. (see Appendix 1 for more suggestions on defining the benefits and costs to be measured)
BCR uses the total benefits and the total costs, and are expressed as a ratio (eg 10:1) For example: If you gain a benefit of $1M in 12 months, and the cost of training is $250K for the same period, the BCR is 4:1
14
x 100
In ROI, the costs are subtracted from the total benefits to produce net benefits, which are then divided by the costs. This shows the net benefits to the company after the costs are covered. For example: If you gain a benefit of $1M in 12 months, and the cost of training is $250K for the same period, the net benefit is $750K; divided by the costs of $250K (and multiplied by 100) this equates to a 300% ROI
3. Break-even time Break-even time in months = Investment Benefit For example: If you gain a benefit of $1M in 12 months, and the cost of training is $250K for the same period, the break-even time is $250k divided by $1M, times 12 months = 3 months It is possible for business benefits to last more than 1 year, but in most cases, the effect of training is more difficult to assess over longer periods, so typically 1 year benefits are calculated. Longer term programs can be measured over multiple years. Calculating ROI requires that business results data must be converted to monetary benefits. It is important to be able to allocate financial value to results such as Improved productivity o Time saved o Output increased Enhanced quality Reduced employee turnover Decreased absenteeism Improved customer satisfaction
x Period in months
Scenarios
ROI calculations can be used in 2 ways; To identify the returns on a specific training programme, and compare these to expected results To examine alternative approaches, and their respective expected costs and benefits, e.g.: o blended vs. purely classroom
15
o o multiple course programme vs. single course limited scope vs. extended scope (e.g. addressing a few goals, or addressing a larger number of goals)
The first use is a relatively straight forward analysis of costs and benefits, using the models and tools provided. The second use however can also be accomplished using the tools. In this case, you can use the tools to provide a number of different views of the ROI data relating to the different approaches, and compare the results by applying the estimated data to the final ROI calculations.
16
Focus groups An extension to the Interview technique, involving in depth discussion as well as individual feedback; especially valuable for subjective quality judgments covering a group of participants Action plans (or Performance contracts) and Program assignments To measure the actual performance of a participant in carrying out a pre-determined task(s), or to assess the participants understanding of what actions would be taken in performing the task(s) Performance data monitoring To capture specific performance of the participants against measured performance criteria
17
18
Communications costs to reach internal and external audiences promotional programmes mailers and flyers poster design and production conference calls and meetings PR o Costs incurred in the evaluation exercise Time and effort used o Who Training Project team Training programme managers Instructors Senior management time and attention Training administration, records administration Learners/students Learners managers (e.g. time spent contributing to the evaluation) o What activities Training Needs Analysis and Programme design Materials development and testing Accreditation design and development Communications and training Evaluation and ROI calculation o
Both of these items are amenable to financial cost calculation. Note: Time and effort should take into account both: Actual salary and expenses of staff Opportunity cost of not performing other tasks *** Note, some items of infrastructure costs, such as implementations of Learning Management Systems, may be so large as to make for great difficulty in allocating costs to a specific programme. In these cases you should decide whether to: allocate a nominal share of shared facilities or infrastructure cost to any particular programme of learning calculate an actual cost of shared facilities or infrastructure cost where it has been in operation for long enough to make a reasonable calculation allocate the full cost of this infrastructure, if it is implemented purely for the purpose of supporting the project In all these cases, the basis of calculation should be clearly stated and understood.
19
Stufflebeam CIPP
Stufflebeam considers evaluation as the process of delineating, obtaining and providing useful information for judging decision alternatives The CIPP model of evaluation was developed by Daniel Stufflebeam and colleagues in the 1960s, out of their experience of evaluating education projects for the Ohio Public Schools District. Stufflebeam, formerly at Ohio State University, is now Director of the Evaluation Centre, Western Michigan University, Kalamazoo, Michigan, USA. CIPP is an acronym for Context, Input, Process and Product. This evaluation model requires the evaluation of context, input, process and product in judging a programmes value. CIPP is a decision-focused approach to evaluation and emphasises the systematic provision of information for programme management and operation. In this approach, information is seen as most valuable when it helps programme managers to make better decisions, so evaluation activities should be planned to coordinate with the decision needs of programme staff. Data collection and reporting are then undertaken in order to promote more effective programme management. Since programmes change as they are implemented, decision-makers needs will change so the evaluation activities have to adapt to meet these changing needs as well as ensuring continuity of focus where appropriate in order to trace development and performance over time. The CIPP framework was developed as a means of linking evaluation with programme decisionmaking. It aims to provide an analytic and rational basis for programme decision-making, based on a cycle of planning, structuring, implementing and reviewing and revising decisions, each examined through a different aspect of evaluation context, input, process and product evaluation. Stufflebeam viewed evaluation in terms of the types of decisions it served and categorised it according to its functional role within a system of planned social change. The CIPP model is an attempt to make evaluation directly relevant to the needs of decision-makers during the different phases and activities of a programme. In the CIPP approach, in order for an evaluation to be useful, it must address those questions which key decision-makers are asking, and must address the questions in ways and language that decisionmakers will easily understand. The approach aims to involve the decision-makers in the evaluation planning process as a way of increasing the likelihood of the evaluation findings having relevance and being used. Stufflebeam thought that evaluation should be a process of delineating, obtaining and providing useful information to decision-makers, with the overall goal of programme or project improvement.
20
There are many different definitions of evaluation, but one which reflects the CIPP approach is the following: Programme evaluation is the systematic collection of information abut the activities, characteristics, and outcome of programmes for use by specific people to reduce uncertainties, improve effectiveness, and make decisions with regard to what those programmes are doing and affecting (Patton, 1986:14). Stufflebeam sees evaluations purpose as establishing and providing useful information for judging decision alternatives; assisting an audience to judge and improve the worth of some educational programme or object; assisting the improvement of policies and programmes. The four aspects of CIPP evaluation (context, input, process and outputs) assist a decision-maker to answer four basic questions: 1. What should we do? This involves collecting and analysing needs assessment data to determine goals, priorities and objectives. For example, a context evaluation of a literacy program might involve an analysis of the existing objectives of the literacy programme, literacy achievement test scores, staff concerns (general and particular), literacy policies and plans and community concerns, perceptions or attitudes and needs. 2. How should we do it? This involves the steps and resources needed to meet the new goals and objectives and might include identifying successful external programs and materials as well as gathering information 3. Are we doing it as planned? This provides decision-makers with information about how well the programme is being implemented. By continuously monitoring the program, decision-makers learn such things as how well it is following the plans and guidelines, conflicts arising, staff support and morale, strengths and weaknesses of materials, delivery and budgeting problems. 4. Did the programme work? By measuring the actual outcomes and comparing them to the anticipated outcomes, decisionmakers are better able to decide if the program should be continued, modified, or dropped altogether. This is the essence of product evaluation. The four aspects of evaluation in the CIPP model support different types of decisions and questions (see Figure 2).
Type of decision
Planning decisions Structuring decisions
21
Process evaluation Product evaluation Implementing decisions Recycling decisions Are we doing it as planned? And if not, why not? Did it work?
Since the introduction of Kaufman's four-level OEM model, many researchers have used it as a viable framework for evaluation. Others, though, have found it restrictive and have attempted to modify and/or add to it. Kaufman, et al. (1996), for example, later added levels of impact that go beyond the traditional four-level, training-focused approach which they felt did not adequately address substantive issues an organization faces. Such modification to the model resulted in the addition of a fifth level, which assesses how the performance improvement program contributes to the good of society in general as well as satisfying the client. Kaufman's Five Levels of Evaluation (Kaufman et al., 1996) Level Evaluation 5 4 3 2 1b 1a Societal outcomes Focus Suggested Levels*
Organizational Organizational contributions and payoffs. output Application Acquisition Reaction Enabling
Individual and small group (products)utilization within Micro the organization. Individual and small group mastery and competency. Methods', means', and processes' acceptability and efficiency. Availability and quality of human, financial, and physical resources input. Micro Process Input
22
A key benefit of using the CIRO approach is that it ensures that all aspects of the training cycle are covered. Context Evaluation here goes back to the reasons for the training or development event or strategy. Employers should look at the methods used to decide on the original training or development specification. Employers need to look at how the information was analysed and how the needs were identified. Inputs Evaluation here looks at the planning and design processes, which led to the selection of trainers, programmes, employees and materials. Determining the appropriateness and accuracy of the inputs is crucial to the success of the training or development initiative. If, for example, the wrong types of learners were chosen to attend a customer care National Vocational Qualification programme this would be a waste of time and money for the organisation. Reactions Evaluation methods here should be appropriate to the nature of the training undertaken. Employers may want to measure the reaction from learners to the training and to assess the relevance of the training course to the learners roles. Indeed assessment might also look at the content and presentation of the training event to evaluate its quality. Outcomes Employers may want to measure the levels at which the learning has been transferred to the workplace. This is easier where the training is concerned with hard and specific skills - this would be the case for a train driver or signal operator but is harder for softer and less quantifiable competencies including behavioural skills. If performance is expected to change as a result of training, then the evaluation needs to establish the initial performance level of the learner. In addition to evaluating the context, inputs, reactions and outcomes to training and development, employers must continuously measure the costs. A cost/benefit analysis is usually conducted prior to committing to any training initiatives. Costs must be monitored to ensure that they don't scale over budget.
23
24
References
Jack Phillips Center for Research (JPCR) Online evaluation of company environment for assessment and ROI readiness The Human Capital Return on Investment from Global Learning Alliance (in association with Knowledge Advisors) How to Maximize the ROI of Learning by Dean Spitzer, IBM The Bottom Line on ROI The Jack Phillips Approach (Canadian Learning Journal Vol. 7 No. 1, Spring 2003)
Jay Cross A Fresh Look at ROI, ASTD Learning Circuits online magazine http://www.franklincovey.com/jackphillips/inde x.html
http://www.franklincovey.com/jackphillips /satest.html
http://www.glaworld.com/documents/2F1D3F1 4-1968-45399DC1213B0B4DB724/Human%20Capital%20R OI.pdf http://www.ibmweblectureservices.ihost.com/ser vlet/Gate/Component?action=load&customer=ib m&offering=sngl&itemCode=ltu2153f http://www.learningdesigns.com/page_images/LDOArticleBottomLin eonROI.pdf http://www.learningcircuits.org/2001/jan2001 /cross.htm
Jack Phillips - Return on Investment in training and performance improvement programs (ISBN 0-88415-492-0) Jack Phillips - HRD Trends Worldwide Shared Solutions to Compete in a global economy (Chapter 9) (ISBN 0-88415-356-8) The CIPP approach to evaluation (Bernadette Robinson)
http://hub.col.org/2002/collit/att-0073/01The_CIPP_approach.doc
25
2004 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Itanium is a trademark or registered trademark of Intel Corporation in the U.S. and other countries and is used under license. XXXX-XXXXEN, 07/2004