Human systems and aircraft maintenance

Air Transport SAFETY Human Systems and Aircraft Maintenance Nick M C D O N A L D fTesearch in normal operational environments indicates that the formal requirements for the (~ircraft maintenance system are frequently not fully or effectively implemented in practice, 7his gives rise to a pervasive 'double standard" with no effective means of reconciling official ( ~nd unofficial operational practice. A systems strategy for human-centred management in (~ircraft maintenance is being developed in a number of European projects, which address i,;sues of design, planning, operational practice, monitoring, change and competence, Ine primary elements go\'erning the European aircraft maintenance system include the Joint Aviation Requirements (JAR) and the Air Transport Associat on's (AT/\) specifications for maintet ante documentation. JAR 145 I he framework of JAR 145 regulations i; built around the philosophy of granti G approval to maintenance organisatons which haxe an adequate manage~ent system to ensure safe operations. I hus tl~e regulator only indirectly reguhtes the safety of the operation - fine responsibility is on operational and c uality management to ensure safety. In [articular this devolution of responsi[~ilitv for safety and airworthiness is c \pressed through the requirements to c esignate an accountable manager and t ~ hax'e an independent quality system xlqch inchldes a feedback system to t tnstlre corrective action. lhe manufacturer lhe principle specification relevant to rmintenance documentation is ATA s:>ecification 100 wlnicln sets out guidelines for the manufacturer for producing maintenance documentation, such t lat "the Aircraft Maintenance Manual ( \ M M ) shall p r m ide the necessary proo-dures to enable a mechanic who is t nfamiliar uith the aircraft to maintain tie aircraft properly." Manufacturers are also particularly concerned to casure that documentation is comprehensive accurate and up to date. The maintenance organJsation fhe Maintenance Exposition Document is compiled in compliance with the regulatkms of the European Joint A\iation ALithorities governing maintenance organisations (JAR 145). TIx, document contains fine company's formal information on Maintenance Management (i.e. roles, responsibilities, accountabilities), Maintenance Procedures (Line, Light and Base), and Quality S;'stems Procedures. The maintenance technician -[he maintenance teclmician operates a set of approvals to cindertake maintenance work on ,,,pecified aircraft types, and to certify that the work lnas been carried out according to fine correct procedures. The basis of this is fine certification of competence in compliance with fine requirements of IAR 66. Llnder How does the organisational system work in practice? fhe ADAMS project p r o v i d e d fine opportunity to study the functioning of four European aircraft maintenance r • organisations. Unles, otherwise stated fine research discussed below is reported in McDonald et al. [1] Quality and safety Auditing practices ear} widel 3 hehveen different maintenance organisations and national aviation authorities. While all organisations place emptnasis on effecfive auditing of documentation, some audit fixed facilities and resources, but few attempt to audit h o w work is actu- I li AIR & GPAC[ alh' done. There are no comnx~n auditing standards. Few quality-reporting systems work as they should, particularly in dealing witil lmman factors information. Some organisations are only starting to implement a quality discrepancy reporting system, and either it onh' covers part of their operation or man\ technicians are not aware of its existence. For some it is not seen to be sufficiently independent of fine disciplinary process to be fulh' trusted. For others there is a large rob Lime of reports wlnictngive rise to a backlog and long delays on responding. Part of fine problem is reported to be getting managers to take responsibility for dealing with reports when they have other more pressing matters to attend to [2]. Organisations are not learning from tlneir incidents. Following incidents or accidents, it is critical to future safety tlnat organisations learn from what has happened and implement change to pre~ent similar incidents occurring. This is particularly true of fine lmman and organisational factors which contribute to incidents. It is hard to find hlfornlation on cases ulnere learning and change has occurred. The Case Stud\ on "Organisational learning from incidents" [2] demonstrates that, often, despite focused efforts to solve fine problem, incidents may ha\e to occur several times before effective change Inappens. Organisations are rareh, systematic in their follow-up to non-technical aspects of incidents, specifically the implementation of recommendations, the monitoring of their effectiveness in addressing fine problems tinev are EkiROt't ° VO[. 3 • hJo ~,,4 2001 HUMAN SYSTEMS designed to change and ensuring that knock-on problenls are a\ aided. Planning and organisation For many companies, the traditional functional organisation with an established hierarchical structure and areas of specialisation dictates a top-down process of plalming and organisation. For example, the Engineering department o\'ersees the higher order and long term planning and produces the maintenance schedule (MS) for each aircraft. The Planning depal:tment receives the MS from the Engineering department and produces, certifies and dispatches work-packs required to accomplish scheduled maintenance. The scheduling section in the Production area then receives the work-packs fi'om the Plalming Department and further breaks the packs down into the daily work. On completion of the checks on the aircraft the Planning department then audit and maintain the work-packs and any other records for the aircraft. This top-down sy'stern tends not to be flexibly responsive to the needs and problems of the production system because strong organisafional boundaries and little opportunity for feedback prex ent the effective co-ordination of work. Some organisations are trying to moxe to a more process-based organisafion that in effect calls for the break-down of traditional departmental barriers in whicln the overall planning process is crossfunctional. Thus, while engineering, planning and material departments still exist, the Fqannim'o process inx'olves the integration of these to oversee the planning of long term and day to day maintenance activities. The planning and coordination of daily work take place within production control centres located in the hangars. The make tip of the producLion control centres brings together fnnctions previously carried out in planning, materials and engineering. Quality of documentation In most of the organisations studied, the quality of the documentation available to the maintenance technician (especially through microfilm readers and printers) fell well below basic ergonomic standards. Even where modern CD ROM systems were available they were not often used. Training was rarely pro\'ided in their use. Oll tile other hand every technician will admit to using 'black books' - unofficial documentation. This documentation is not available to scrutiny or inspection because of its illegal status. vestigators and ttle authorities, it is diti:i cult to know whether the ieccmlnleixtations from hlvestigations ha\e been implenlented and, if so, whether file\ have been effective I q. Implications of the evidence C o m p l i a n c e with procedures Maintenance engineers completed 286 questionnaires after riley had completed a task. The questionnaire sought primarily to discover the normative level of deviation fi'om task procedures, as well as inquiring into the reasons behind this non-conformance. 34"i, of respondents reported not-following the official procedure for the task. The most common reason gi\'en was that there was an easier way than the official method (45%) followed bv 43% saying there was a quicker way. A number of factors which were related to increased likelihood of non-conformance were identified. Those individuals who consulted the manual but did not follow the official method were significantly more likely to report that: • the task card was unclear; • the necessary steps to complete the task were unclear; • to hat'e employed guesswork or trial and error; • to report that the maintenance history was desirable but unavailable. Major incidents Increasingly, e\idence froill major incident and accident enquiries ([3], [4], [5]) is implicating failures at an organisational and regulatory level. Of particular concern are situations where there ha,'e beena series of incidents exllibiting similar underlying organisational problems, while tl~e immediate characteristics of the incident might be quite different. For the official in- What inferences about the i]ornlativt. system of aircraft maintenance can bu marie fronl these disparate obser\ ation, about how it is practised7 .Ge\eraI gel> eralisations stand oLIL; • for many organisations the top-dowil nattlre Of their planning s\stem, moans that tl~ev tend to be reiati\el\ unresponsive to the S]lort-ternl requirements ('d production. there b, littk opportunity for feedback and organisational boundaries inhibit ettc'cti\c coordination; • while virtually ever\one admits usin<4 unofficial documeniation, official documentation is not presented in a uax tl~at meets the needs of the u s e r ; • bv COlllmon admission, work is routiilelv done not according t~> the requirements of the maintenamc manual. This is particularl> the case when task requh'eillents are nol clear; • few organisations svstematicall\ attempt to 1110nitor how work is actually carried el_It, being inole COllcernoct to enstlre Cleatthe ctc)¢tllllentaStrategy -... z- ,I/[ l)c~iun //I '~ i l)laillliil~ ~.ilid { ,uganixutiOil '\\ V Ctmi>,,i L ~ J ", J T ()pcili,-:it , \ -\ 1M,,,/iZii;7-1 /" )11~11 ] pcrfornlanCu ./// # ( "ompdcncc systems approach managemen t, Figure I. A to h u m a n centred ,, Air Transport. . . . tion is signed off in order; • quality svstems do not, in general, provide an effective method of gathering feedback, and, even when they do this, they do not provide a means of ensuring corrective action; • there is no transparent system (either within organisations or involving official national agencies) for demonstrating effective response to incidents through implementing recommendations designed to prevent similar incidents happening again. The double standard Putting this evidence together leads inexorably to the following conclusion: • there are hvo parallel systems of work in operation - an official one, and the way in which work is actually being carried out. These two may overlap to some degree; however, there are considerable areas of divergence; • there are no currently effective mechanisms for reconciling discrepancies between these two systems, whether in terms of immediate feedback and adjustment, auditing how work is performed, quality reporting, or response to incidents. ['his can only be described as a 'double ;tandard'. The 'official standard' for task ~erformance has a strong paper trail ram the manufacturer to the maintelance organisation and back, through inditing, to the national authority. The actual standard' relies on unofficial doclmentation and informal work practices. /re should not infer that this system is nherentlv unsafe. It certainly lacks the ransparency which might give an inde,endent observer confidence in the svsera. Thus, technicians believe that there fften are better and quicker ways of toing task than those they understand o be officially sanctioned by the manufi. It is not very difficult to find instances ~f where this is the case when one examnes task performance and documenta:ion in detail. However it is also possible 'o highlight instances where unofficial nethods are pursued in apparently ~nappropriate circumstances. Fhus, in so far as unofficial methods may ~e inherently worse than official methods (which come with the authority of . . . . . . . . . the designers) the system mav be inherently unsafe. In so far as such methods may be better than the official methods, the system may be much safer through using the experience and judgement of professional technicians. Either way, it is clear that the system does not allow for effective learning so that the system as a whole can be made both more safe and efficient. We should be cautious, howev- er, because even if we believe in the effectiveness of unofficial checks and balances in maintaining system safety; this may be true only for a stable system. When such a system is subject to externally produced" changes (new technology, change in organisation or personnel) the implications for safety will be verx difficult to predict, and minor, apparentlv innocuous changes in technology, organisation or personnel may' haxe profound consequences for safety. A systems approach One of the main conclusion of the above is that a systematic approach to ensure feedback and learning throughout the maintenance system is essential. We can express the requirements of a HumanCentred Management System as a set of core assumptions as foliows: • design, plan and organise operations to the best possible standard; * adapt and adjust to deal with unforeseen situations, and learn from experience; • creating the conditions for optimal task performance is necessary for safet}; reliability and efficiency. Components of this generic approach a r e illustrated in.f)ik,un ' 1 and the following paragraphs outline how various RTD initiatives, which ha\'e been developed in response to this analysis, address the elements of the diagram Design for maintainability l h e de\elopment of design for main- tainability guidelines and the integration of these guidelines into design tools and standards is one of the main strands of the work-programme of a new project under the CS(; Aeronautics programme - ADAMS-2 - Human centred svsterns for aircraft dispatch and main- . . SAFETY tenance safety. This is due to run between 2001and 2004. Planning and organisation Many of the key findings and recommendations of the ADAMS proiect were consolidated into a Guide to HumanCentred Management in Aircraft Maintenance. This includes best practice guidelines for human factors in maintenance and an outline of how these topics relate to the requirements of JAR 145. A case stud}, on flexible planning illustrates some requirements of the effective coordination of maintenance operations. Operational performance Developing an in-depth analysis of the task and professional skills of maintenance operations is one of the basic requirements underlying the possibility of developing more effective humanfactors interventions. This has been a core task of an IST-funded project with the acronym AITRAM, as well as comprising an important strand of ADAMS2. In the latter the development of a methodology for the assessment of critical professional skills will enable svstematic comparison of different occupational groups. Cognitive task analyses of key maintenance tasks will complement this work. This will lead to a generic understanding of task and professional skills requirements, which will feed into design guidelines, auditing and event analysis methodologies and training specifications. Monitoring and change Several parallel initiatives address the requirements for monitoring and change. ADAMS-2 will develop a set of demonstrator quality auditing methodologies at the level of task performance, professional skills and organisafional reliability systems. A separate project is developing a continuous improvement system spanning both the maintenance organisation and the aircraft manufacturer (AMPOS - Aircraft maintenance procedure optimisation svstem - Esprit programme, 1999-200]). This comprises an IT-based methodology for processing and managing cases. An organisationaI framework for managing the improvement process has been developed in AIR & SPACE EUROPE • VOL. 3 o N o 3/4 - 2001 HUMAN SYSTEMS both organisations. P,esults so far have demonstrated that e\en simple cases invol\e complex solutions - organisatianal mechanism need to be in place to ensure that recomn~endations are cffectirol\ implemented and reviewed. One of the achievenaents of ADAMS xxas to develop an 'Occurrence reportins form' and error taxonomy. This will be further developed in ,,'\D,A\'IS-2 as a comprehensive prototype event-management system. This will incorporate a risk-awareness and management s\'stern built around incident reporting and risk management prototype databases. The objective here is not just to support the processing of individual events but to facilitate a strategic approach to risk nlallagelllen t. Competence Training to de\relop human factors coin- petence is, of course, one of the elenlents which is critical to successful implementation of the different components of a human-centred management. Building on the initial findings of the ADAMS project, initiatives to develop, implement and evaluate training haxe been pursued at three levels: • a core human factors training programme adaptable for training for training technicians, trainers, supervisors and managers has been completed in the STAMINA project (Leorlardo programme); • a cost-effective approach to c?nsuring effective Imman factors training requires the integrating human factors into technical training. AITRAM (IST programme)is developing a \irtua] rea]Jt\ environmeilt to demonstrate how this should be done; • de\'eloping competence in-depth for hunlan factors specialists is critical to ensuring that the opportunities for change are taken, and that human factors programmes are effecti\ ely managed. The STAMP project (Leonardo programme) is developing accredited training caurscs for stlch specialists. Strategy for c h a n g e If Ilum<m factors pragrammes are to be Illtrrl2 SLiCCOSSfUiit, the i:uture tilan they ha\e been in the past, then they need to offer a strategic vision not only of what benefits thv can bring to the organisation, but also of how to achieve these benefits. A major strand of the ADAMS2 project is to de\'elop an organisational model of the implementation process far the tools and methodologies develaped in the projects described here. In parallel with this a strategic cost-benefit model will be developed. Both the organisational and cost-benefit models will be tested and elaborated through implementation case studies. Conclusion The ADAMS project has documented major deficiencies in how the aircraft maintenance system manages non-technical (htullail factors) informatiori. Fhese deficiencies can be characterised as a double standard - an official way of working, which is publicly ackm;wledged, and an unofficial way in which much maintenance work is actuall\' ed under the EU Esprit programme. Project no. EP 29053. The STAMINA Project (Safety Training for Aircraft Maintenance) w a s funded under the EU Leonardo Programme. Project number IRL/97/1/31013/P1 / 11I.'La/ FPC. The STAMP Project (Specialised Training for Aircraft Maintenance Professionals) is funded under the Fk Leonardo Programme. l'roiect number IRL/001 B/F/PP-IlC*.217. References (1] McDonald, N, Corrigan, S., Daly, C & Cromie, S., Safety management sys terns and safety culture in aircraft m a i n t e n a n c e organisations Safely Science 34 (2000) 151 176 (2) McDonald, N Led), Hcmnc]n Centred Management for AircraH Maintenance, ADAMS project report Department of Psychology, Trinity College Dublin, 1999. done. Breaking down this double standard will require a comprehensive sys- (3) Air Accident Investigation Branch (AAIB), Report on the incident to BALI 1-11, G BJRT over Dialect, Oxfordshir~ on l0 June ]990, HMSO, London 1992. tems approach. Projects building on .\DAMS are developing new tools and methodologies to support a co-ardinated strategy to improve design for maintainabilitv, the planning and organisa- (4) Air Accident Investigation Branch (AAIB), Report on the incident to AirbL~s A320-212, G-KMAM at London Gatwi< k Airport on 26th August ]993. HMSC) London, ]995. tion of work, monitoring of operational performallce, and cllange in nlahltenance systems and operations. • (5) Air Accident Investigation Branch (AAIB), Report on the incident 1< Boeing 737 400,G OBMM, r~cJr Daventry, on 23 February 1995 HMSO London, ] 996 Acknowledgements (6) Smart, K., Presentation to Europ{~an Transport Safety Council q:enleren<:c on a c c i d e n t investigation ETSC Brussels, ] 997 The ADAMS Project (Human Factors in Aircraft l)ispatch and Maintenance Safety) was funded under the European Commission Brite-Euram programnle. Project number BE95-1732. The ADAMS 2 Project (Human Centred S\'stems for ,\ircraft Dispatch and Maintenatlce Safety) is funded under the European Commission Campetifive and Sustainable Growth programme. Project number GRD1-201)0-25751. The AITRAM Project (Advanced Integration training in Aeronautics)is tended under the European Commission Information Society Technalogies pro- gralllnlc. Project ntlrllber IST-Ic)9c)- 12241. The A MPOS Project (Aircra ft Maintenance Procedure Optimisatian System)is fund- About the author: Nick McDonaldworksin the Aerospace PsychologyResearchGroup in the Department of Psychology of TrinityCollege, Dublin. nmcdonald@tcd ie http://www.tcd,ie/Psychology/aprg/index,html