Chapter 1

INTRODUCTION TO PERFORMANCE AUDITING

1.1. Definition and Scope of Performance Auditing
INTOSAI’s Performance Audit Guidelines, ISSAI 3000, 1.1 states:
‘The full scope of government auditing includes regularity and performance audit’, and
‘Performance auditing is concerned with the audit of economy, efficiency and effectiveness and
embraces:
(a) audit of the economy of administrative activities in accordance with sound administrative
principles and practices, and management policies;
(b) audit of the efficiency of utilization of human, financial and other resources, including
examination of information systems, performance measures and monitoring arrangements,
and procedures followed by audited entities for remedying identified deficiencies; and
(c) audit of the effectiveness of performance in relation to achievement of the objectiveness of
the audited entity, and audit of the actual impact of activities compared with the intended
impact.’

Performance auditing is based on decisions made or goals established by the legislature,

and it may be carried out throughout the whole public sector.
Definition of Performance Audit
The ISSAI 3000, 1.1 defines a performance audit as ‘an independent examination of the
efficiency and effectiveness of government undertakings, programs or organizations, with
due regard to economy, and the aim of leading to improvements’. These three terms are
often confused. The overriding objective of any activity is effectiveness. It is also often the
easiest to audit.
Effectiveness is a measure of how well an audited activity achieves its objectives. These
objectives may be specifically stated or they may be the outputs of the activity. For example,
the principle objective of hiring a vehicle to move files from one building to another is that
the files are moved to the other building. If this happens then the procedure appears to be
effective. Effectiveness generally involves not just producing some sort of deliverable but
doing so in a way that optimizes the expenditure of public monies; considers all applicable
regulations and other requirements; processes and reports on financial transactions
accurately and manages the human resource elements of the activity appropriately.
The other elements of performance audit are logically, subsets of effectiveness, and are the
objectives of the activity or program that are often unspoken. Any expenditure on an
activity should be done in the most efficient manner. The program outputs being produced
using the least amount of resources as possible to produce outputs of the required standard.
It would not be efficient for example to hire a ten-ton truck to move a box of files from one
place to another if there were smaller, cheaper vehicles available, even though it would
achieve the desired result.
 Economy is concerned with costs of input. In the example above if it is possible to hire
two identical vehicles to move the box of files, and management chooses the more
expensive of the two, then this is not economical.
The other elements that frequently arise as components of a performance audit are
compliance with procedures and internal control requirements. These are often the
domain of a financial statement audit. However, any activity has the objectives of
complying with applicable laws and the procedures of the organization, managing the
resources of the organization passing through its financial system in a secure manner and
providing relevant and accurate financial reports.

The INTOSAI Performance Audit Guidelines, ISSAI 3000, 1.5 also stipulates ‘that
standards concerning ‘environmental considerations’ and ‘equity requirements’
should also be taken into account in performance auditing.’

The performance auditor may also be expected to address concerns relating to equity and
ethics while assessing the effectiveness of a program/activity. Equity in the context of
program management relates to fairness and impartiality in the use of public funds.
Ethics in managing public expenditure includes the qualities of honesty and integrity in
personal conduct and devotion to the duty as a manager of public funds. These are normally
overall government objectives and the SAI performance auditor should be conscious of
equity and ethics issues.
Performance audits may examine and report on:
 the quality of information and advice available to government for the formulation
of policy;
 the existence and effectiveness of administrative machinery in place to inform the
government whether or not program objectives and targets have been determined
with a view to fulfilling policy objectives;
 if, and to what extent, stated program objectives have been met;
 the economy, efficiency, effectiveness and ethics of the means used to implement
a program/activity;
 if laws and organization procedures have been complied with;
 if internal accounting controls have been effective and transactions have been
reported accurately; and
 the intended and unintended direct and indirect impacts of programs/activities;
for example, the environmental impact of government activity.
Using the vehicle hire example, a performance audit may find any of the following:
 the files have gone missing or have not been transferred to the correct place;
  management may have hired a much bigger vehicle than necessary to move the
files;
 management may have hired the correct sized vehicle but may not have hired the
cheapest vehicle they could have;
 management may have allowed staff to ignore required occupational health and
safety procedures in moving the files;
 a staff member has hired his brother’s vehicle to his personal benefit rather than
following required procedure;
 internal control procedures may have broken down allowing for duplicate payment
or overpayment of the supplier; and/or
 the financial transaction may have been misclassified resulting in the activity being
seen to be more economical than it really was.
Auditors should not confine the audit to ‘what has been done’ but should also examine
‘what has not been done’ to endeavor to meet the policy objectives. Given the size,
complexity and diversity of many entities, it is generally impracticable to attempt to assess
the overall performance of departments or agencies. Consequently, performance audits
are usually directed towards specific functions, activities, programs or operations of the
agency. It is also common that an SAI may conduct an audit of a topic or theme across
several agencies or even on a whole-of-government basis.

The INTOSAI Performance Audit Guidelines, ISSAIs 3000, 1.2 states that ‘performance
auditing is not overly subject to specific requirements and expectations. While financial
auditing tends to apply relatively fixed standards, performance auditing is more flexible in
its choice of subjects, audit objects, methods, and opinions. Performance auditing is not a
regular audit with formalized opinions, and it does not have its roots in private auditing. It is
an independent examination made on a non-recurring basis. It is by nature wide- ranging
and open to interpretations. It must have at its disposal a wide selection of investigative
and evaluative methods and operate from a quite different knowledge base to that of
traditional auditing. It is not a checklist-based form of auditing.’

1.2. Features and Standards of performance Audit

The special feature of performance auditing is due to the variety and complexity of
questions relating to its work. Within its legal mandate, performance auditing must be free
to examine all government activities from different perspectives.
In conducting a performance audit the auditor should follow applicable professional
standards. Some SAIs in the Pacific will follow the INTOSAI Auditing Standards and others
will rely on Generally Accepted Government Auditing Standards (GAGAS) set out in the
Government Auditing Standards (the Yellow Book) issued by the US Government
Accountability Office.1 The principle, relevant INTOSAI Auditing Standards are set out below.
GAGAS are not codified in the same manner as the INTOSAI Auditing Standards although the
main principles are similar to those in the INTOSAI standards. It is strongly recommended
that any performance auditor working in a jurisdiction that applies GAGAS, familiarize
themselves with the standards as discussed in the Yellow Book.

The INTOSAI General Auditing Standards 200, 2.1 common to SAIs which subscribe to
INTOSAI standards are: ‘(a) the auditor and the SAI must be independent; (b) the auditor
and the SAI must possess the required competence; and (c) the auditor and the SAI must
exercise due care and concern in complying with the INTOSAI Auditing Standards. This
embraces due care in planning, specifying, gathering and evaluating evidence, and in
reporting findings, conclusions and recommendations’.

The performance auditor and SAI must be, and be seen to be, independent and objective
in the conduct of audits.

The INTOSAI General Auditing Standards 200, 1.2 note that SAIs should: ‘(a) recruit
personnel with suitable qualifications; (b) develop and train SAI employees to enable them
to perform their tasks effectively, and to define the basis for the advancement of auditors
and other staff; (c) prepare manuals and other written guidance and
instructions concerning the conduct of audits; (d) support the skills and experience
available within the SAI and identify the skills which are absent; provide a good
distribution of skills to auditing tasks and assign a sufficient number of persons for the
audit; and have proper planning and supervision to achieve its goals at the required level
of due care and concern; and (e) review the efficiency and effectiveness of the SAI’s
internal standards and procedures.’

1.2.1. Work Performed by Staff

INTOSAI Field Auditing Standards 300, 0.3 (b) requires: ‘the work of the audit staff
at each level and audit phase should be properly supervised during the audit,
and documented work should be reviewed by a senior member of the audit staff’.

When work is delegated to members of the audit team, the auditor in charge should
carefully direct, supervise and review this work. When multi-disciplinary audit teams are
used, adequate direction, supervision and review are necessary to ensure that the different
perspectives of team members, their experience and specialties are appropriately used in
the audit. All team members should understand the objectives of the audit, the terms of
reference of work assigned to them and the nature of obligations imposed on them by
applicable auditing standards. Adequate direction, supervision and review will assist in such
an understanding.
1.2.2. Using the Work of an Expert

INTOSAI General Auditing Standards 200, 2.43 state: ‘if the SAI employs external experts
as consultants it must exercise due care to assure itself of the consultants' competence and
aptitude for the particular tasks involved; and
Obtaining advice from an external expert does not relieve the SAI of responsibility for the
opinions formed or conclusions reached on the audit task’.

An expert is a person or firm possessing special skill, knowledge and experience in a

particular field other than auditing. Because of the diverse range of activities subject to
performance auditing, the auditor may need to obtain audit evidence in the form of reports,
opinions, valuations and statements of an expert. Although the auditor may use the work
of an expert as audit evidence, the auditor retains full responsibility for the conclusions in
the audit report. When using the work performed by an expert, the auditor should obtain
sufficient appropriate audit evidence to ensure that such work is adequate for the purposes
of the audit.
1.2.3. Professional Behaviour

The INTOSAI Code of Ethics and Auditing Standards, Chapter 5 requires that: ‘auditors
have a duty to conduct themselves in a professional manner at all times and to apply high
professional standards in carrying out their work to enable them to perform their duties
competently and with impartiality.’

The auditor should comply with ethical principles and codes of conduct governing the
auditor’s professional behaviour and responsibilities, which include:
 integrity;
 objectivity and fairness;
 confidentiality; and
 technical standards.
The auditor should comply with any code of conduct or ethical standards issued by the
government or SAI for which they work. Although this manual sets out a coherent basis
for conducting a performance audit, professional judgement remains an important
ingredient in performance audit work. As with any audit, the performance auditor should
adopt an attitude of professional skepticism throughout the audit, recognizing that
circumstances may exist that could cause the information relating to performance to be
irrelevant, misleading or incorrect.
 1.2.4. Reasonable Assurance
A performance audit conducted in accordance with applicable auditing standards provides
reasonable assurance that the conclusions drawn in the audit report may be relied upon.
Reasonable assurance relates to the accumulation of audit evidence necessary for the
auditor to respond to the audit objective. What is ‘reasonable’ is dependent on the facts
of the situation including what evidence could reasonably be gathered and what
conclusions could reasonably be drawn in the situation.

1.2.5. Terms of the Audit

The INTOSAI General Auditing Standards 200, 2.26 provides that ‘In contrast to private sector
audit, where the auditor's agreed task is specified in an engagement letter, the audited entity
is not in a client relationship with the SAI. The SAI has to discharge its mandate freely and
impartially, taking management views into consideration in forming audit opinions,
conclusions and recommendations, but owing no responsibility to the management of the
audited entity for the scope or nature of the audits undertaken.’

The SAI should formally notify the auditee of the details of the audit, preferably before the
commencement of the audit, to help avoid any misunderstandings. With some audits, the
objective and scope of the audit and the auditor’s obligations are established by law. Even
in those situations identifying the features of the audit may be useful to the agency and
assist the auditor develop a productive working relationship with the agency.

Often during the course of an audit, findings will lead the auditor to broaden the scope of
the audit or eliminate areas that were initially advised as being included. When this
occurs the auditor should advise the auditee management of this change in scope well
before the auditor issues the draft report. If an auditor reports on matters that the
auditee was not even aware were included in the audit scope, the auditor may lose
credibility and the value of the audit report can be compromised. If this happens useful
recommendations may be rejected without due consideration. It is a good idea to discuss
with senior management in the SAI any proposed changes to audit scope and seek their
approval.

1.2.6. Quality Assurance

INTOSAI General Auditing Standards 200, 1.27 states that : ‘The SAI should establish
systems and procedures to: a) confirm that integral quality assurance processes have
operated satisfactorily; b) ensure the quality of the audit report; and c) secure
improvements and avoid repetition of weaknesses.’

Each SAI should have its own quality assurance procedures to meet this standard. The
auditor should be familiar with these processes and conduct and document their audit
with a view to meeting the quality assurance requirements.
 1.3. Performance Auditing Mandate and Objective

ISSAI 3000, 2.1 states: ‘the mandate of performance auditing should cover the state
budget and all corresponding government programs. The auditor must be free to select
audit areas within its mandate. Political decisions and goals established by the legislature
are the basic frame of reference. A performance audit may, as a result of its findings,
question the merits of existing policies. Performance audits are in general ex post audits
that deal with current issues. High levels of quality in the work must be promoted and
secured.’

The mandate for SAIs to undertake performance audits varies from country to country.
Some countries have a performance audit role specifically written into the legislation that
creates the SAI. In other countries it may not be so specific. The head of the SAI could, for
example, have the ability to report to the national parliament on performance issues he or
she may come across in the course of their regularity work. Other SAIs may only be able to
conduct performance audits at the behest of the government or parliament. Even others
may have no wider performance audit mandate than is reflected in their accounts, records
and reports on financial audits; that is, the performance aspects of financial regularity.
The interpretation of the breadth of the performance audit mandate is generally made by
the head of the SAI. Performance auditors should plan and conduct their audits in line with
this interpretation. Put simply, the objective of performance audit in the public sector is to
improve public administration. At one level this is achieved by making sound
recommendations based on sufficient evidence obtained in an audit planned and conducted
to a professional standard.
The actual conduct of an audit can also lead an auditee to improve systems and procedures,
particularly through audit interaction with management at an operational level. The
performance audit process may provide the impetus for management to consider the
activities they manage in a different perspective. They may make changes irrespective of the
findings and recommendations of the audit. This is a very good outcome in terms of
improving public administration.
It is also important that the performance auditor not pre-empt management’s right to
manage. Management may develop their own plans to resolve the findings reported by the
auditor. That is their prerogative and this too will improve public administration, even if the
audit recommendations are not accepted, as long as the findings are based on solid
evidence. It is often the case that audit recommendations are rejected but the entity fixes
the problems in another way, which is more suitable to their operations.
Performance audits also have another role. They provide stakeholders such as senior
management, members of the government and, if made publically available, the public
with information and assurance about the quality of management of public
resources. The emphasis placed on the assurance role of performance auditing as against
the role of public sector improvement will vary between SAIs and is also decided by the
head of the SAI.
1.4. Performance Audit Process
The first stage in performance auditing is strategic planning, which requires the
development and maintenance of information on the agency that will assist in identifying
potential areas for audit. Potential topics can then be analyzed and ranked to form annual
audit strategy documents for each agency. Chapter 2 of this module deals further with
strategic planning. Once a topic has been selected for performance audit, an audit program
is developed and this is gradually refined into a detailed audit plan. It sets out the audit
procedures that will be applied in the conduct of the audit. A preliminary study may be
undertaken to gather information on the topic and to identify significant issues for audit.
Chapter 3 of this module deals with planning the performance audit. The execution stage of
a performance audit is dealt with in particular in Chapter 4 of this module and involves:

 development and execution of an audit program of procedures;

 collection and documentation of sufficient relevant and reliable evidence,
including quantitative and qualitative analysis;
 formulation of audit findings, conclusions and recommendations; and
 development of discussion papers and confirmation of audit findings at an exit
conference.
It should be noted that the audit program and detailed audit plan need to be reviewed
continually throughout the audit to ensure that it remains relevant and sufficient. A
report on the audit will then be drafted for consideration by the government, the
legislature, the agency and the public. Throughout each stage of the performance audit
the emphasis should be on the production of a final report that is balanced and has
value-added impact. The report- writing process should be viewed as a continuous one of
formulating, testing and revising conclusions, if necessary, about the audit topic.
Therefore, issues such as the expected impact and value of the audit, the likely
improvements and savings resulting from it and methods of communicating its
conclusions should be considered throughout the audit. Chapter 5 of this module deals
further with reporting.
Follow-up activities by the SAI provide impetus for the implementation of accepted
recommendations. Management are more likely to focus on implementing
recommendations if they are aware that the SAI could revisit the area. Follow-up
procedures should identify and document the impact of the audit and the progress of the
agency in implementing audit recommendations. The conduct of follow-up activity is also
vital to provide feedback to the SAI, the legislature and the government on
performance audit effectiveness in producing improvements in public sector
management. Follow-up processes are dealt with in Chapter 6.
 Chapter 2
STRATEGIC PLANNING
2.1. Introduction
This chapter discusses the strategic planning process. A performance auditor new to the
profession, the main audience for this manual, will presumably not be heavily involved in
this process. However, all performance auditors should understand the process adopted
by their SAI to undertake strategic planning and should be watchful for information they
may accumulate during the course of their work that may inform or contribute to that
process. Performance auditors should make themselves fully aware of their SAI’s strategic
planning process and their role in that process.
The scope for undertaking a performance audit is almost limitless. Even if a business
activity is currently performing well, there is generally the potential for some
improvement. If performance audit resources were unlimited then audits would be
undertaken on any activity where it was expected that the savings to be made would
exceed the cost of the audit.
In reality, SAIs can only conduct a relatively small number of performance audits and the
resources that can be devoted to them are very limited. Strategic planning, from a
performance audit perspective, revolves around deciding which of the very large number
of performance audit topics will yield the largest long-term gain, either in terms of savings
or the improved achievement of the key objectives of programs and activities.

ISSAI 3000, 3.2 states: ‘strategic planning is the basis for the selection of audit topics. Linked to
a SAI’s annual planning system, it may be a useful tool in setting priorities and selecting audits.
It may serve as a mechanism for selecting future audit themes, and a basis for detailed
planning. Finally, it may serve as an instrument for strategic policy decisions on the future
direction of the audit. Planning might be carried out in the following steps: determining
potential audit areas; establishing the selection criteria to be used; and identifying the main
sources of information for the potential audits. The strategic planning exercise would normally
result in a coherent and cogent audit program for the SAI and serve as a basis for operational
planning and resource allocation.’

In order to determine which audits are to be undertaken the SAI needs to rank audit topics.
The audits which are identified as having sufficient priority to warrant having resources
allocated to them are documented in what is generally called a strategic audit plan (SAP).
Different SAIs may call this by different names but for the purposes of this manual, a
strategic audit plan is any plan to conduct a number of specific audits over a period. Three
years is the most common planning period but some plans range over two years and others
up to five years depending on the SAI. For ease of discussion a longer- term program of
audits will be referred to in this module as a SAP.
2.2. Objectives of Strategic Audit Planning
The objectives of strategic audit planning are to:
 provide a firm basis for the SAI management to give strategic direction for future
audit coverage;
 identify and select audits with the potential to improve public sector accountability
and administration;
 provide a platform for communication with agencies and the legislature on SAI
audit strategies;
 produce a work program that can be achieved with expected available resources;
 understand agency risks and take them into account in audit selection; and
 provide a basis for SAI accountability.
In order to create a SAP, first the SAI must define the population of the audit topics that
exist. Having done that, there are a number of variables to consider in developing a ranking
of audit topics.
2.3. Identifying Potential Audit Topics
Identifying the complete population of potential performance audit topics in even a small
public sector would be a very time consuming and resource-intensive task. Most SAIs make
the assumption that information about the most significant programs and activities is
already available to them. The sources of this information include:
 the auditee, and concerns information such as:
o public documents including annual reports and media releases;
o business plans;
o financial statements and internal financial management reports;
o performance reports;
o internet sites.

 external stakeholders:
o media reports;
o academic papers;
o publications from professional bodies;
o papers issued by NGOs and donor agencies.
 other legislature information:
o previous SAI reports;
 o committee reports and evidence;
o transcripts of sittings;
o other reports to the legislature; and
 from other audit or inspection bodies:
o work done by internal audit teams;
o reports from other internal reviews;
o material from financial auditors;
o other Supreme Audit Institutions’ reports on similar activities.
In reviewing all of these sources the SAI is seeking to determine whether a performance
audit will add value in some way. Rather than assembling a detailed map of all the
performance audits that could be undertaken, it is more common for SAIs to rely on the
accumulated knowledge of their auditors and the available information to identify potential
audit topics. The range of audit topics identified in this way will be far greater than the
available audit resources could realistically cover and the list must therefore be prioritized.
The SAI should also consider undertaking audits across more than one agency. This sort of
performance audit, called perhaps an across-the-board or whole of government audit, or a
cross-portfolio audit, examines a single activity in different agencies. It is particularly useful
where a central department or ministry provides guidance, policy, procedures or rules to
other agencies. A cross agency audit can assess how well the requirements are being
followed and if they are effective at achieving what the central agency expects.
Cross agency audits can also examine activities which should be done in an efficient and
effective manner in different agencies. For example, it is possible to review something like
physical security, energy use or motor vehicle management to assess whether existing rules
are being followed or, where there is no central guidance, whether some rules should be
introduced.
2.4. Prioritizing Audit Topics
In determining those audit topics which are likely to provide the best value for the audit
dollar, there are a number of parameters the SAI may consider. These include the:
 issues that present key risks to public administration;
 significance of the activity in terms of materiality, sensitivity and impact;
 audit resources required to achieve the expected audit result and the availability of
those resources.

In some countries, the legislature may pass legislation which requires the SAI to
undertake particular audits. In these instances, the SAI is obliged to conduct the audit and
it is allocated priority. Committees of the legislature may also make similar requests and
this is taken into account when prioritizing identified audits.
2.5. Issues that Present Key Risks to Public Administration
Some activities are inherently more risky than others, which means there is more chance
of something going wrong. The riskier the activity, the greater the likelihood that a
performance audit will be able to identify risks that could be managed better. For
example, a project to develop a major computer system is far riskier than a project to
customize a similar commercially available system that is itself riskier than acquiring a
commercial system that needs no modification.
Risk will also be determined by the extent of past reviews of the activity. Any area that has
recently been subject to an independent review will generally be less risky than one that
has not. Initiating a performance audit in such an area may be a poor use of audit
resources, particularly if all it does is duplicate the findings of the earlier review.
A performance auditor needs to have an excellent understanding of risk analysis and risk
management techniques to be able to assess the risks associated with a potential audit
topic and also to form a preliminary opinion on whether they are being managed or not.

These risk analysis skills are also very valuable during the conduct of an audit.
Performance auditors responsible for strategic planning who don’t already have training in
risk analysis and management should access training in these techniques as a matter of
priority. Key aspects to consider in determining the significance of an audit include;
materiality, sensitivity and impact.

2.5.1. The Significance of the Activity – Materiality

This parameter is generally the most straightforward to assess. It has two components,
how much does the activity cost over time and how much is invested in it. It should be
recognized that the overall size of an activity is not an absolute determinant of audit
priority. A mature program that has been running for some years and has had various
reviews done of it may be a poor candidate for audit, even though it expends hundreds of
millions of dollars a year. This is because the likelihood of identifying any significant
recommendations is limited. Alternatively, controls may have weakened over time and it
is also possible that the audit could yield significant benefits. Clearly the auditor needs to
know more about a program or activity than just the cost of it in order to decide whether
it is a worthwhile subject for a performance audit.

2.5.2. The Significance of the Activity - Sensitivity

The objectives of any government activity will have a certain profile with the public and
with the legislature. The more important an activity is in the eyes of its collective
stakeholders, the more suitable it is as a performance audit subject. This is because the
stakeholders are, by definition, more concerned about the achievement of objectives and
they benefit from being reassured about this achievement. This means that a small but
sensitive program may be more suitable as a performance audit topic than a much larger
routine one.
Government advertising is one such topic where various stakeholders have a keen interest
in the nature, cost and propriety of the activity, even though in overall terms it is not
generally a major public expenditure.
Programs or activities that are particularly important from a national interest point of view
also rank highly. Health and education programs, for example, are often allocated
significant performance audit resources. In deciding on the priority for an audit topic, the
auditor should consider carefully the importance of the activity to the various key
stakeholders.

2.5.3. The Significance of the Activity – Impact

The more complex an activity is, the greater the likelihood that some key management
controls will not have been considered in the development of management systems. This
means that there is more potential for a performance audit to add value, by identifying
controls that may be put in place to improve the outcomes of the activity. For example, a
distributed accounts processing system operating across a number of geographic sites is
much more difficult to manage effectively than a system at a single site processing the
same volume of transactions.
2.6. The Audit Resources Available
The number of auditors available to undertake performance audit work is one factor that
will limit the number of audits that can be included in a strategic plan. Preparation of the
SAP will require knowledge of the total number of auditors expected to be available to
conduct audits over the period.
Another aspect of audit resources that needs to be considered is what the individual staff
members, at their current stage of development, are capable of doing and what their
development needs in terms of experience are. If some of the available auditors are
inexperienced then there must be sufficient audit activity in the program that they are
capable of doing.
It would be inappropriate to assign an audit, which would involve sensitive dealings with
very senior management, to newly qualified auditors. Likewise it would not be an efficient
use of resources to use highly experienced staff to undertake an audit of a simple line
management activity.
Many larger SAIs have more than one performance audit team and assign areas of public
sector activity to specific teams. A team may have responsibility for one or more discrete
areas such as defense, social security or health. The audit resources assigned to these
teams will be determined by how much priority the SAI assigns to that group of topic
areas. If a SAI decides that it needs a team of 5 people concentrating on audits of say,
education, this is a strategic planning decision and the audit manager must then allocate
priorities within that topic area. The assignment of staff to audit teams with specific
responsibilities is therefore another consideration in the strategic planning process.
 In preparing a strategic plan for performance audit, the SAI needs to consider the total
audit resources available, the breakdown of those resources into functional areas and the
skills and experience of the individuals involved. In practice, this generally becomes a
collective assessment by all of the responsible audit managers who provide input to the
strategic planning process.
Performance audits are often unable to commence at the expected time, mostly for
auditee related reasons. So the plan should include enough audits to provide further
topics if the highest priority ones are not able to be done for some reason. A prudent
audit manager could have as much as 30% more audit hours planned than are actually
available, to be sure that the audit team always has audits that can be commenced if a
planned audit fails to start as expected.

2.7. Developing the Strategic Audit Plan (SAP)

Assembling a SAP from the available topics generally involves using the topic knowledge
and management experience of the audit managers. They should each have a good
understanding of their assigned areas and their staff, as well as of the stated priorities of
the SAI. This is important because the senior management of the SAI may have taken a
strategic decision to focus on a particular theme, such as procurement, or human resource
management. This means that audit managers must take this component of the topic area
into account in defining a possible audit and in assessing its importance.
A well-structured strategic planning process, based on a sound rationale, is necessary to
ensure that the resources of the SAI are used in the most efficient and effective manner.

2.8. Documenting the SAP

The key deliverable of the strategic planning process is a document which is prepared for the
SAI management to enable it to critically assess the proposed planning strategy for overall
consistency with the SAI’s corporate objectives. The basic SAP document will be a list of
audit topics in the various government agencies along with expected resource requirements
for each audit and the proposed start and finish dates for the audit. The document will also
assist management to make appropriate resource allocations and allow an assessment of
the strategic planning process.
The strategic plan should be supported by working papers that may include:
 a description of the agency and its environment;
 an assessment of the risks to good performance of the agency’s programs and
activities.
 a summary of the long-term strategic view of SAI performance audit directions in
each agency; and
 recent SAI audits, recent and proposed enquiries by the legislature, and agency
evaluations and internal audits.
The SAP will provide a list of audits proposed for the next two to five years. This list should
reflect resource availability and should also distinguish between new performance audit
topics, cross-agency audits and follow-up audits.
2.9. Approving and Publishing the SAP
The SAI should have procedures in place that allow the senior management of the SAI to sign
off on the overall plan. This is important because it enables the head of the SAI to ensure the
allocation of resources is appropriate across the whole. It may be that audits that rank as the
most important in one branch or group may not be as important overall as some audits that
have not been selected because the branch or group responsible for them has too many
high priority audits. The senior management of the SAI may then need to reconsider the
allocation of audits and staff to the various audit groups.
Once the SAP is approved it may be worthwhile to provide it to the relevant parliamentary
committee or other government body charged with overseeing the work of the SAI. To
provide transparency it could also be made available as a public document on the SAI’s
website or through some other means. The SAP should also be made available to staff to
provide some context and understanding of the performance audit work of the SAI.
Chapter 3-Planning the Performance Audit
Introduction
This chapter outlines the steps involved with initiating the performance audit: the
purpose of a preliminary study; the approach to be considered in planning individual
audits and the development of an audit program.

3.1. Standards for Planning Individual Audits

ISSAI 3000, 3.4 states that: ‘the auditor should plan the audit in a manner which ensures
that an audit of high quality is carried out in an economic, efficient and effective way and
in a timely manner’2 and also identifies the following steps that are ordinarily included
in planning an audit:
 ‘collect information about the audited agency and its organization in order to assess
risk and to determine materiality;
 define the objective and scope of the audit;
 undertake preliminary analysis to determine the approach to be adopted and
the nature and extent of enquiries to be made later;
 highlight special problems foreseen when planning the audit;
 prepare a budget and a schedule for the audit;
 identify staff requirements and a team for the audit; and
 familiarize the audited entity about the scope, objectives and the assessment
criteria of the audit and discuss with them as necessary.’3
 Planning consists of developing a general strategy and a detailed approach for the
expected nature, timing and extent of the audit. The audit plan is a key document for
controlling and monitoring audits in the SAI. Before executing the performance audit, it is
consequently important to define the audit objectives, the scope and the methodology to
achieve the objectives. The preparation of an audit plan is the necessary first step in
audit planning as it is the document that the head of SAI will authorize so that the audit
can proceed. It is important to remember that planning is not a single audit phase but that
it will be necessary to revisit initial plans over the course of the audit. This will be to
ensure that the audit team is on track, or as a result of unforeseen circumstances, it might
be necessary to revisit and revise the audit plan and its underlying approach.

A preliminary study is an important tool to ensure that the audit team is well acquainted
with the activity to be audited and can develop a persuasive audit proposal for senior
management ‘sign off’.

3.2. The Role of a Preliminary Study/Survey

Once a topic has been selected for audit, the SAI may often conduct a preliminary study so
that it is better able to understand the context of the audit and judge the audit resources
that are required. The preliminary study aims to explain why the audit should be carried
out and provide sufficient justification to proceed with executing the performance audit.

INTOSAI Performance Audit Guidelines 3000, 3.3 provide that: ‘if conditions do exist to
continue on to the execution stage of the audit (main study), an audit proposal should
be prepared. If a performance audit is not recommended, a preliminary study report
should summarize the preliminary study conclusions. To finalize the preliminary study,
an exit conference may be held with the agency management.’

If the head of a SAI decides to move from a preliminary study to a full audit, an audit
proposal for management consideration and endorsement is the next step.
3.3. Developing and Approving the Audit Proposal
The steps involved in developing an audit proposal require that the auditors:

 understand the business of an auditee;

 determine audit objective;
 determine audit scope;
 determine audit criteria;
 identify key audit foci;
 determine the audit approach;
 estimate resources and timing; and
 approve the audit proposal.
 These steps may be grouped together differently or sometimes done in different order,
but in any case, these steps will allow for greater efficiency when developing a detailed
work plan (also referred to as ‘audit program or test program’). It is through a well
thought out plan that the auditor will be able to carry out the audit in an economic,
efficient and effective way and in a timely manner and provide SAI management with the
necessary assurance to approve the audit proposal.

3.4. Understanding the Business of an Auditee

It is important that the auditor develop a sound understanding of the auditee’s business
to allow the development of more specific audit objectives and relevant audit
criteria. This knowledge would include an understanding of:
 the mandate of the agency and the areas being audited within the agency;
 objectives of the agency and of programs related to the audit activity;
 programs and performance goals of the agency;
 organizational and accountability relationships within the agency;
 the internal and external environment of the agency and the stakeholders;
 external constraints affecting program delivery;
 agency management processes and operations; and
 the resources of the agency.
Sources of information to assist in understanding the agency include:
 enabling legislation and legislative speeches;
 ministerial statements, government submissions and decisions;
 recent audit reports, reviews, evaluations and enquiries into the agency;
 agency strategic and corporate plans, mission statement and annual report;
 budget statements;
 agency policy files, management committee and executive board
minutes;
 agency organization charts, internal guidelines and operating manuals;
 agency program evaluation and internal audit plans and reports;
 conference reports and minutes;
 discussions with agency management and key stakeholders; and
 management information systems.
Past audits and other reviews can provide an extremely useful source of information. If
the SAI has maintained a permanent audit file, for either financial or performance
purposes, then this should be a major source of information. They can help avoid
unnecessary work in examining areas that have been under recent scrutiny and
highlight deficiencies that have not yet been remedied. There is, however, no substitute
for discussions with senior agency management to gain an overall program
perspective against the background of the above information.
Performance monitoring, accountability and evaluation processes within the public
sector are generally agency-based. However, there could also be a range of
information, which crosses agencies and which may also assist in the information-
gathering stage, such as:
 studies by industry, professional or special interest groups;
 enquiries or previous reviews by the legislature;
 information held by coordinating agencies or by interdepartmental
committees;
 research by academics;
 work undertaken by other governments overseas; and
 media coverage.
The purpose of understanding the business of the auditee is to allow the auditor to
develop the objective and scope of the audit and to assist with the next planning
component, which is the development of audit criteria. It is not a substitute for field-work
in the agency and should be kept to the minimum required to develop the initial objective,
scope and audit criteria. The appendix to this chapter discusses some key features of
government agencies that may be taken into account in developing an understanding of
the business of the auditee.
3.5. Determine the Audit Objectives
Audit objectives relate to the reasons for conducting the audit and should be established
early in the audit process to assist in identifying the matters to be audited and reported.
The audit objective should address concerns of accountability and good governance and
may focus on the economy, efficiency and/or effectiveness of program management as
well as financial control where relevant. Compliance with legislation and financial
regularity should also be considered where appropriate. For example, an audit objective
may be ‘to determine whether capital procurement managed by the ministry is being
managed to provide best value for money and in accordance with industry best practice.’

The audit objectives should be designed to maximize the benefits and impacts from the
audit. It must be remembered that at this stage of the process, audit knowledge is
relatively limited and all planning must be considered fluid depending upon what is found
as further work is undertaken. In setting the audit objective the audit team should
take into account the mandate of the SAI, and be careful not to exceed it, as well as
the reasons for the audit as defined in the strategic audit plan.

It is good management practice for audit objectives and scope to be discussed

with the agency management. In holding such discussions auditors should be careful
not to compromise their independence.
3.5.1 About Audit Objectives
The purpose of the audit is to initiate improvements in public administration and/or to
provide assurance to management that an activity is being managed effectively,
economically and efficiently and in accordance with government policy and legislation.
A performance audit is an expensive activity so it is rare that an audit will be undertaken
just to meet the assurance objective. In most cases such audits will arise from information
coming to light that an activity is being badly managed in some way and the government
has asked the SAI to review the program as a result. Even in these cases the auditor
should be looking to add value in terms of initiating improvements in those things that
may be able to be done better.
The objective of initiating improvements in administration is often unwritten but follows
implicitly from the act of reviewing the activity. For example, the objective of the audit
was to assess whether the procedures established to manage the program would ensure
that government policy was followed. What is not stated is that if the auditor finds that
these procedures do not ensure that government policy is followed then they will
recommend changes to the procedures to achieve this end.

3.5.2. Writing the Audit Objective

The audit objective consists of two main parts, (a) what is being reviewed and (b) what is
being used as a benchmark. In the example above ‘the program’ is being reviewed and
 ‘government policy’ is the performance benchmark.
The structure of a performance audit objective is therefore: To assess/
determine/discover/report whether a specified activity/program/entity is being managed
to provide the best result to the government in terms of economy/
effectiveness/efficiency/probity/performance reporting/security over assets or
information etc.. For example, in an audit of capital procurement, ‘the objective of this
audit is to determine whether capital procurement managed by the ministry is being
managed to provide best value for money and in accordance with industry best practice.’
When developing an audit objective, the auditor should seek to frame both parts of the
objective in the clearest possible terms and as simply as possible. The objective of the
audit may be written as a question. It is also possible to extend the objective into a series
of associated questions which may be answered in the audit process. The above objective
could be redrafted as:
 ‘the objective of the audit is to answer the question: does the ministry manage capital
procurement so that it provides best value for money and is in accordance with industry best
practice?’ and associated questions may be:
 ‘If not, what is the impact? If the impact is significant, what are the causes? What
may be done to address these causes?’

3.5.3. Importance of Defining the Audit Objective(s)

It is very important to establish audit objectives clearly. It provides a sense of direction to
the auditor and also justifies the purpose of the audit. Each and every audit conclusion
reported must be made against the corresponding audit objective. Suppose the audit
objective was to ensure whether the procurement of material was done at the lowest cost
(economy) with due regard to appropriate quality. In that case, the auditor may come to
anyone of the following conclusions:
 yes, the procurement of material was done at the lowest cost with due regard to
appropriate quality; or
 no, the procurement of material was not done at the lowest cost with due regard to
appropriate quality; or
 yes, the procurement of material was done at the lowest cost, however,
appropriateness of quality was not considered at the time of the procurement
decision; or
 yes, the procurement of material was done at the lowest cost with due regard to
appropriate quality, however, excessive volume of material was procured in
comparison to actual requirements.
After the audit work is complete, the auditor develops conclusions that respond to the audit
objectives. So, if there is no audit objective, there can be no appropriate audit conclusion.
3.6. Determine Audit Scope
The scope of the audit defines the boundaries of the activity being reviewed and tends to
narrow the ambit of the audit from the broad audit objective. In the example, the scope of
the audit as it stands includes all of capital procurement in the ministry, but there may be
some value in refining this in a separate scope statement.
For example:
‘The scope of the audit will include all procurement on construction projects, all other
capital purchases exceeding $5 million and will consider issues of value for money and
compliance with documented procedures. This narrows the scope from the original
objective.’
In some cases, the scope statement outlines the elements excluded from the scope as
defined by the audit objective. For example, following the capital procurement objective
above there could be a scope statement which reads: no procurement under $5 million will
be included in the audit unless it is part of a building construction project.
The decision on the scope of the audit is made taking into account the areas of the activity
where most benefit could be gained from the audit while considering the amount that can
be covered in the time and resources available for audit field work.
Scope may be geographic, excluding some physical areas, where an activity is widely
distributed. Scope is also time-related and the auditor should specify a time-period to be
covered. The scope statement may also narrow the focus of the audit by specifying, for
example, that the audit will only consider issues of effectiveness or compliance.
In some cases, where a performance audit has been requested, the scope may be
extremely narrow. If, for example, there have been credible allegations about corrupt
procurement practices in an agency, the SAI may be asked to conduct an audit to provide
assurance that the allegations are groundless or action needs to be taken. The scope of
such an audit would be restricted to procurement practices and may be further restricted
to compliance with required procedures and the accuracy of financial reporting.

3.6.1. Changing the Scope

When the objectives and scope of an audit are defined, the auditor does not know as
much about the activity being audited as he or she will after several weeks or months of
field work. It may be that at some point in the audit the audit manager realizes that there
is significant value to be gained in changing or widening the scope to include areas that
were not encompassed in the original audit plan.

The INTOSAI Performance Audit Guidelines, ISSAI 3000, 4.4 stipulates that ‘a performance
audit may run for a long time, and there may be changes in knowledge and reality from the
point in time when it started. In performance auditing it is often difficult to make a choice
between the directions set out in the work plan and the description of the audit’s structure on
the one hand, and the interest in studying questions that arise at a later date on the other. To
avoid getting caught up in details and a flood of data, detailed assessments of the need for
information must be made both before and during the audit. Based on experience, this makes
it easier to eliminate extraneous detail and irrelevant approaches, and to sort or structure
the information gathered. Again, however, the auditors must not be rigid and avoid all
unplanned data gathering.’
Senior audit management should be involved in any decision to widen the scope of the
audit. They may decide that this extra work should be identified as part of a future audit,
rather than an extension of the current one. It is essential that the auditee is advised in
writing that the scope of the audit is to be widened. Senior audit management may be
involved in negotiations with the auditee. Where it is prudent to do so, the reasons for
widening the scope should also be explained. The audit objectives and scope are closely
interrelated, and since changes in one usually affect the other, they need to be considered
together.

3.7. Lines of Enquiry

After selecting a suitable project for performance audit and establishing the audit
objectives and scope, it is important to identify significant lines of enquiry or identifying
issues of significance to pursue in the audit.

3.7.1. Factors for Identifying Key Areas

A rational approach to identifying key areas of a selected project involves analysis of entity
information on the basis of valid factors. While there could be a variety of factors that
may be considered by audit teams in different SAIs. Some of the more widely used factors
are:
 significance;
 risk to management;
 likely impact of audit;
 audit-ability.
I. Significance
The significance of an audit area should have regard to the magnitude of its impact on the
project as a whole. It will depend on whether the activity is comparatively minor or
whether shortcomings in the area concerned could flow on to other activities within the
project.
Financial materiality is an aspect of significance. This factor is based on an assessment of
the total value of government assets, annual expenditure and/or annual revenue of the
auditable area. The more material an area is, the higher is its priority for selection as a line
of enquiry. It is analogous to financial materiality in financial audit. However, significance
in performance audit is a wider concept than financial materiality.
Significance will rate highly where the activity is considered to be of particular importance
to the success of the project and where improvement would have a significant impact on
the operations of the project. A low ranking in relation to ‘significance’ would be expected
where the activity is of a routine nature and the impact of poor performance would be
restricted to a small area or be likely to have minimal impact.
 Visibility is another aspect of significance. It is more the external impact of the activity. It
is related to the social, economic and environmental aspects of the activity and the
importance of its operations to the government and the public. In considering this factor,
some weight would be attached to the impact of an error or irregularity in this area of
activity on the management’s accountability to the public. It would also have regard to
the degree of interest shown by the legislature and public in that area of activity.

II. Risk to Management

The auditor should assess the risk that the management of the area to be audited is
deficient in economy, efficiency and effectiveness.
Evidence of risk to good management includes:
 management inaction in response to identified weaknesses;
 adverse comment by the legislature or media;
 non-achievement of stated objectives;
 high staff turnover;
 significant under spending or overspending;
 sudden program expansion;
 overlapping or confused responsibility relationships.
A project’s activity that is complex to manage and operates in an uncertain environment is
more likely to increase risk to management. Some possible indicators of high complexity
and uncertainty are:
 highly decentralized operations with a multiplicity of interested parties;
 use of rapidly changing and sophisticated technology;
 a dynamic and competitive environment; and
 controversial social and political debate surrounding the issue.
III. Likely Impact of Audit
Of major importance in identifying the lines of enquiry is the added value expected from
the audit. A preliminary estimate of the likely benefits from a particular line of enquiry
should be made at the planning stage itself. If detailed audit enquiry into a particular area
of the selected project is not likely to have any significant impact, then audit
recommendations are unlikely to generate appropriate action. While considering this
factor, the question that the audit team needs to answer is ‘is the audit likely to make a
difference?’ If detailed audit in the concerned area of the project is not likely to make a
difference, then there would perhaps be no justification in applying limited audit
resources in that area at the expense of other areas of the project also demanding audit
investigation.
The following list of possible impacts classifies benefits by reference to economy,
 efficiency, effectiveness, planning, control and management and accountability:
Economy
 introduction of charges where none were previously imposed, or revision of
charges;
 reduction in costs through better contracting, bulk buying, etc.;
 reduction in costs through economies on usage of personnel or other resources;
 rationalization of facilities.
Efficiency
 greater outputs from same inputs;
 remedying duplication of effort or lack of coordination.
Effectiveness
 better identification/justification of need;
 clarifying objectives and policies;
 introducing better sub-objectives and targets;
 better achievement of objectives by changing nature of outputs or improved
targeting.
Improved Planning, Control and Management
 introduction of or improvements to management planning;
 clearer definitions of priorities and better-defined targets;
 better-targeted incentives;
 better control and management of human resources, assets, projects and
resources;
 tighter controls against fraud;
 improved financial accounting systems;
 better financial management information;
 better computer security.
Improved Accountability

 improved external control and monitoring by the department/ministry;

 better and/or more accurate performance indicators;
 better comparison between similar organizations;
 clearer and more informative presentation of information.
The greater the opportunities for audit impact, higher the priority to be given to that area
of the project.
IV. Audit-ability
Audit-ability relates to the audit team’s ability to carry out the audit in accordance with
professional standards. A variety of situations may arise that may cause auditors to decide
not to audit a particular area of a selected project even though it is significant. In reaching
such a decision, auditors may consider:
 the nature of the activity is inappropriate; for example, it may not be practical to
attempt to audit the technical considerations of a research facility;
 it does not have or cannot acquire the required expertise;
 the area is undergoing significant and fundamental change;
 suitable criteria are not available to assess performance.
The significance, risk, likely impact of audit and audit-ability of an area activity will
influence prioritizing in identifying lines of enquiry. If an area of the project is ranked
highly on all or most of these elements, it would be identified as a key area for detailed
audit enquiry.
The factors that we have described above are the basis for a systematic approach to
assisting the auditor in applying judgement in selecting audit projects. Using these factors
when supported by valid information and data will help auditors in allocating scarce
resources for the audit projects. Having done this it is useful, to set out the approach the
audit team will use.

3.8. Determine the Audit Criteria

Audit criteria are a key component of a performance audit. Audit criteria are assertions
that the activity being audited is being managed in a way that will achieve the various
performance objectives. The specific objectives to be tested are determined by the scope
of the audit but may include the explicit objectives of the program and also those of
efficiency, economy, safeguarding of assets, security, probity, compliance and financial
regularity. If the audit criteria are not set, there will be no basis for comparison or
measurement and consequently no basis for arriving at audit findings, conclusions and
recommendations.
The word ‘criteria’ is the plural of ‘criterion’ which means a rule, a standard or a test by
which something can be judged. Audit criteria, therefore, are a set of reasonable and
attainable standards of performance against which the achievement of program objectives
can be assessed. In other words, audit criteria reflect a normative or desirable model for
the subject matter under review. They represent good practice, a reasonable and
informed person’s expectation of ‘what should be’.
Examples of simple criteria are ‘government assets should be used only for authorized
purposes’ or ‘ministry financial instructions should be complied with.’ Specifying the audit
criteria will enable the auditor to construct an audit test program that will allow the
auditor to find out if reasonable expectations are met.
The use of audit criteria is similar to the use of hypothesis testing in financial statement
auditing. In performance auditing the hypothesis being tested is ‘that this criterion is being
 achieved by the activity’.

Relevant criteria arising from laws or regulations carry the force of law. Policies and
procedures of the agency are, by definition, accepted by the agency. For example, in an
audit of teacher absenteeism, a criterion might be ‘teachers should attend school in
accordance with their conditions of employment’.
The selection or development of audit criteria becomes more complex where there are no
appropriate readily available criteria. Other externally sourced criteria such as professional
standards, industry best-practice, expert opinion or available bench-marks may require
the auditor to demonstrate that they are applicable to the activity being audited.

Sometimes criteria are not as relevant as they seem to be. If, for example, compliance
with existing financial instructions will not ensure value for money, then testing to
determine whether instructions have been complied with is neither reliable nor sufficient
to form a conclusion that value for money is achieved. Likewise if the government rules
regarding what use of government assets is authorized are vague or non-existent, then
the auditor must analyze the activity to develop a more relevant criterion.

In developing audit criteria, it is also important that they are organized in a logical manner
so that the audit examination can be conducted as rationally and efficiently as possible. In
framing audit objectives as questions, the audit criteria are included in the question. For
example ‘are teachers attending school in accordance with their conditions of
employment?’ Audit procedures should then be developed to allow the auditor to answer
this question with confidence.
3.8.1. The Characteristics of Functional Audit Criteria
The auditor should be satisfied that suitable criteria have been identified to enable the
auditor to assess the activities subject to audit and to achieve the audit objectives. Some
characteristics of suitable criteria are explained below. Ultimately, the relative importance of
the characteristics of the audit criteria is a matter of professional judgement.

ISSAI 3000, Appendix 2 identifies the following characteristics of suitable criteria:

 Reliability Comparability
 Objectivity Completeness
 Usefulness Acceptability
The auditor may also consider the following characteristics for determining the suitability
of criteria:
 Relevance Unambiguous/Understandable
1. Reliability
Reliable criteria result in consistent conclusions when used by another auditor in the
same circumstances. The criterion ‘goods and services are only procured after an objective
competitive evaluation of the most likely possible suppliers’ is reliable if another auditor can
determine that the statement is the benchmark against which the auditee should procure its
goods and services.

2. Objectivity
Objective criteria are free from any bias on the part of the auditor or management. The
auditor should examine every audit criteria with a critical eye to make a judgement as to
whether or not they have been developed independently. The criterion ‘goods and
services are only procured after an objective competitive evaluation of the most likely
possible suppliers’ is objective because it is based on a widely accepted business principle
that applies to any purchases where a competitive marketplace exists.

3. Usefulness
Useful criteria result in findings and conclusions that meet users’ information needs.
The criterion ‘goods and services are only procured after an objective competitive
evaluation of the most likely possible suppliers’ would not be useful if an audit was
requested of the effectiveness of the auditee’s contract administration process and not its
procurement process.

4. Comparability
Comparable criteria are consistent with those used in performance audits of other similar
agencies or activities and with those used in previous performance audits within the
agency.
The criterion ‘goods and services are only procured after an objective competitive
evaluation of the most likely possible suppliers’ is promulgated by laws governing public
entities. Therefore, it is considered to meet the comparability characteristic.

5. Completeness
Completeness refers to the development of all significant criteria appropriate to assessing
performance in the circumstances. To ensure that criteria have been established for all
significant aspects of performance within the scope of the audit, the auditor should go
through each significant aspect of the activity and map it against each performance
element in the audit scope.

6. Acceptability
Acceptable criteria are those to which the audited agency, legislature, media and general
public are generally agreeable. The most acceptable criteria are those which the agency
has already endorsed. It is difficult for an agency to argue that its own internal policies and
procedures do not have to be complied with.
Making a case for accepting audit criteria often starts with ‘basic truths’ and logically
constructing criteria from there. For example:
Basic truth: government purchases should provide the best value-for-money to the
government. While this basic truth cannot be argued, some auditees will interpret ‘value-
for-money’ as ‘cheapest’ and the difference between these two terms may need to be
explained to the auditee. The statement is not suitable as a criterion, however, because
‘best value for money’ is too vague and difficult to determine and cannot be used as a
benchmark in this form.
In theory, the way to get best value is to collect bids or prices from all suppliers who can
provide suitable goods or services. Contacting all possible suppliers is generally neither
possible nor economic so this is not suitable as a criterion. An audit criterion needs to be
practical and this theory should be modified so it provides an acceptable benchmark. The
entity should canvass a sufficient number of appropriate suppliers to be reasonably sure
that there are no suppliers who are going to be able to provide the goods or services at a
greatly reduced cost.
A more useful audit criterion then becomes, ‘goods and services are only procured after
an objective, competitive evaluation of the most likely possible suppliers’.
7. Relevance
Relevant criteria link the audit objectives with the activity objectives within the scope of
the audit. If the auditor is developing a criterion to assess the effectiveness of an activity
then it would not be relevant to include aspects of cost or economy, which would have
their own criteria. In an audit of electoral education, for example, a criterion that would
consider effectiveness would be ‘all potential voters have ready access to material which
will allow them to know their rights and responsibilities in the electoral process.’ It may
be that for reasons of cost, many voters do not have this access, but inserting the phrase
‘where it is cost-effective to do so’ would change the nature of the criterion and make it
less effective. It is far better to have a criterion relevant to effectiveness and another
relevant to cost.
The criterion ‘goods and services are only procured after an objective competitive
evaluation of the most likely possible suppliers’ is relevant because it indicates one thing
that should be done to ensure that procurement achieves value for money or economy.

8. Unambiguous/Understandable
Unambiguous criteria are clearly stated and are not subject to significantly different
interpretations. If a criterion can be interpreted in more than one way, then it is possible
that management and audit will see the same criterion differently. It is also possible that
the audit team member making findings against the criterion will see things differently to
the audit manager, so the manager will not get the audit he or she expected.
After drafting, every criterion should be closely reviewed to ensure that it means one
thing and one thing only. The criterion ‘goods and services are only procured after an
objective competitive evaluation of the most likely possible suppliers’ is relevant because
it is not subject to interpretation. The phrases ‘an objective competitive evaluation’ and
‘possible suppliers’ can only mean one thing. The use of the phrase ‘most likely’ is less
specific and is subject to a reasonableness test, ‘who could reasonably supply these goods
or services?’ but an independent person would mostly come up with the same answer to
this question. These characteristics should be considered together when identifying
criteria and deciding on their suitability.
Figure 1. A Model of Government Performance Expectations

Government performance expectations

Mission performance goals
Input Economy Process Outcome Output
Efficiency Effectiveness Effectiveness
Financial Amount, Quantity Mission &
timing Physical Productivity Quality: Outcome Goal
Quantity, quality Unit Cost products, Achievement
Timing, price Operating delivery Financial Viability
Ratios Timelines Cost-Benefit
Price/cost Cost-Effectiveness
Crosscutting Performance Goals
Compliance with Laws and Regulations
Resources – Safeguarding - Infrastructure
Continuous Improvement Reliability,
Validity, Availability of Information
Underlying Values
Customer and Stakeholder Satisfaction
3.8.2. Sources of Audit Criteria
The audit team will need to identify or develop audit criteria that are valid for the
nature of the activity under review. These may include quantitative and/or qualitative
measures. To avoid the necessity o f creating criteria from first principles for each
audit, the audit team should investigate the following sources:
 laws and regulations;
 legislative or executive decisions;
 historical comparisons;
 professional standards;
 key performance indicators of the auditee;
 independent expert advice;
 new or established scientific knowledge ;
 criteria used previously in similar audits;
 agencies carrying out similar activities; and
 performance standards used by the agency, or previous enquiries by the
legislature;
 general management and subject matter literature;
 the internal policies and procedures of the organization;
 government-wide policies and procedures;
 professional organizations and standard-setting bodies;
 criteria published by other SAIs.
These sources provide a basis for the development of suitable criteria for the audit, but
may require interpretation and modification to ensure their relevance to the agency One
source of criteria can be internationally accepted good practice processes.
Criteria must be realistic and take into account the context of the agency. However, if the
agency has established its own performance criteria, the auditor should examine these
critically to assess whether they are suitable for audit use.

3.10. Determine the High-Level Approach to Audit

In determining the audit approach, it is important to identify probable types and sources of
evidence and how the audit team will gather this information for the previously determined
audit criteria. These can be identified before the more detailed work plan is developed. See
Chapter 4 on how this is done. At this stage, the audit approach is a basic skeleton to inform
SAI decision makers of the team’s proposed approach to the audit. Figure 2 below, provides
an Example of an Audit Approach.
Figure 2. Example of an Audit Approach

Line of Enquiry: Evaluation of Options

Audit Objective: To determine if the need for the project was properly established to
ensure that desired results were achieved at an appropriate cost
Type(s) of
Probable sources of Suggested audit
Criterion evidence
evidence technique(s)
required
1.A formal project
Project appraisal report;
needs definition
Documentary; project manager, members
should be carried out Document review,
testimonial; of the committee on public
prior to interviewing; photography
Physical works, samples of
commencement of the
rehabilitated roads
project
Project need definition
2. The project needs
report; proceedings of any
definition should be
committee set-up for
carried out and Documentary; Document review,
defining project needs;
reviewed by persons testimonial interviewing
project manager; members
technically qualified
of the committee on public
for the task.
works
3.The project needs
should be stated in
Documentary Project appraisal Document review
specific measurable
terms
4.Adequate funds
should be allocated Documentary;
Project budget, project Document review, data
for this stage of the testimonial;
manager analysis, interviewing
project management analytical
project
5. Decision making in
the needs definition Project approval file
process should follow Documentary recommendations of the Document review
an objective, rational committee on public works
approach.

3.11. Estimate Resources and Timing

The estimation of resources and timing of the audit will be a task for the audit manager,
who should prepare a budget and schedule for the audit. The proposed audit timing and
budget should be approved by the relevant senior management, which in many cases will
be the head of the SAI. Once the audit proposal (with a work plan) has been prepared, the
audit manager will review these estimates to ensure that sufficient time and resources are
allocated to the audit to address the issues identified and respond to each of the audit
criteria.
 Staffing the Audit
Audit management will decide which particular auditors will be assigned to an audit and
will take into account such things as:
 capability and experience of individual staff members;
 special expertise requirements;
 development needs of staff; and
 staffing combinations e.g. placing experienced and inexperienced staff together.
If the experience or expertise requirements of the audit cannot be met by available SAI staff,
consideration should be given to engaging experts to work with the audit team.
3.12. Approve the Audit Proposal
Senior SAI management should be kept informed of and involved in the development of the
audit proposal. However, it is also necessary for the audit team to seek head of SAI approval
to proceed with the audit. This formal approval is necessary not just for internal SAI quality
assurance but also as a basis for the head of SAI to inform an agency that they will be the
subject of an audit and provide them with the high-level specifics of the audit.
Chapter 4
EXECUTING THE PERFORMANCE AUDIT
IV.1. Introduction
The main purpose of the audit is to execute the audit program by conducting the audit in
an efficient and effective way in order to achieve the audit objectives and criteria and, at
the end, producing a high-quality report.
Once a decision has been made to conduct a full audit, the auditor should develop a
detailed audit program/plan to specify how the audit criteria should be tested and identify
possible means of collecting appropriate evidence against each audit criterion. This
includes developing sub-criteria or control objectives. The auditor then develops tests to
confirm whether or not these sub-criteria or control objectives have been met. The
auditor must ensure that for every sub-criteria or control objective there is least one audit
test.
The audit objectives and criteria in the detailed audit plan will normally be tested by
appropriate audit methods that include:
 observing, interviewing and questioning;
 documenting, testing and checking; and
 analyzing data.
During the implementation of the audit the processes evolve gradually through interaction
with the auditee and through developing an improved understanding of the way the
activity is managed.
 I. Auditee Liaison
Establishing a professional working relationship and careful management of communication
with the auditee while maintaining audit independence, is an essential facet of an effective
audit. To enhance the good relationship, this includes not only the correspondences and
consultations between the auditee and the auditors but also maintaining their cooperation
throughout the process is important. The correspondence includes the following.
II. Formal Advice of the Audit
It is in the interest of both the auditor and the auditee that the auditor sends a letter or memo to
the auditee before the start of the audit field work. This documentation is important to avoid
misunderstandings about the proposed audit. This letter is similar in some respects to the
engagement letter in a financial audit and some SAIs use this terminology. Other SAIs do not use
the term ‘engagement letter’ but often provide a ‘terms of reference’ or just a letter or memo
setting out what the auditee needs to know.
The letter should explain the mandate under which the SAI is performing the audit and the
obligation of the auditee under the relevant legislation to cooperate with the SAI and
provide access to records and staff and the objective and scope of the audit. This letter
should also advise the auditee of what will happen at the end of the audit and the consultation
process for the discussion papers and the draft report. It is also useful to emphasize the positive
nature of the audit and that the SAI will be seeking to develop recommendations which will assist
auditee management.

IV.2. Entrance Meeting/Entry Conference

Upon approval of the audit program the audit manager arranges with the auditee for a
formal meeting prior to the official commencement. This is to communicate relevant
information contained in the program to management. The objectives of the entry
meeting/entrance conference are to:
 enable the audit team to meet the key auditee staff;
 discuss the audit program i.e. identifying the audit objective, audit scope and the
audit work to be performed (a copy of the audit program should be supplied for
the auditee);
 establish suitable liaison arrangements both at the management and working
levels including arrangements for continuous cooperation between the parties;
 ensure the auditee clearly understands the audit processes, including a description
of access powers and the steps taken to safeguard the confidentiality of the
information obtained and the audit issues before issuing the audit opinion;
 outline the auditee’s responsibilities, clarify any queries or misunderstandings the
auditee may have and inform the auditee of the expected timeframes of the audit.
Depending on the nature of the audit, careful consideration needs to be given to the level of
the SAI and agency representatives at the entry meeting/entrance conference. The entry
meeting/entrance conference must be documented.
Figure 3. Example of Formal Advice of an Audit

IV.3. Consultation during Audit

Throughout the audit, a continuous constructive process of interaction between the parties should
be properly maintained. This is to ensure the success of achieving the audit objective. It is good
practice that the auditee is provided with regular briefings on the progress on the audit. This
allows for a number of benefits which include:
  enabling management or the responsible officers to comment on or clarify any
arising incongruity during the audit;
 providing the management with the opportunity to place a different perspective
on audit findings;
 drawing on management’s experience in assessing recommendations for
improvement;
 advising management of issues requiring early corrective action.
Allowing an audit to be carried out in such a cooperative manner means there is a much
greater possibility of suggestions for improvement being seriously considered by auditee
management. It is important that the audit team members maintain an objective,
independent and professional attitude. They should also exercise professional judgement.
IV.4. Developing the Audit Test Program
Audit test programs (referred to here as test programs) are guidelines for action during the
execution phase of the audit. Test programs set out the detailed audit procedures for cost
effective collection of evidence.
Developing test programs for carrying out audits provides assurance that there is a key link
between the audit objectives and the conduct of an audit. In this respect audit programs
serve as:
 a guide for gathering competent, relevant, sufficient evidence during the execution
phase of the audit in a cost effective way;
 a framework for assigning work amongst the members of the audit team;
 a means of transferring knowledge to junior staff; and
 a basis for documenting the work done and the exercise of due care.
Test programs need to be developed after obtaining an understanding of the entity,
defining audit objectives and scope, establishing audit criteria, and deciding on the audit
approach because these constitute the key inputs of an audit program. At the same time,
the test program should be developed before proceeding to gather evidence so that the
actual evidence gathering process is efficient and effective. However, it must be
remembered that in real life all these stages are iterative. In other words, we may have to
revisit and partially modify our audit objectives, scope and audit criteria while developing
the test program in order to make our test program more focused. We may also have to
revisit our test program after commencing the execution phase if, for example, it is found
that certain sources of evidence suggested in the test program turn out to be
inappropriate.

IV.5. Audit Techniques

This refers to methods/techniques used by the auditor to gather evidence. Examples
include (among others) documentation review, interviews, questionnaires, data analysis
and physical observation.
 IV.6. Audit Procedures
Audit procedures refer to the action steps to be taken to execute an audit technique,
systematically and reasonably. Figure 6, is a case example of the development of audit
procedures that may be carried out for the audit technique ‘document review’ against the
criterion ‘a formal project needs-definition should be carried out prior to commencement
of the project’:
Figure 4. Example of an Audit Procedure

Name of the project: Road rehabilitation and maintenance project

Line of enquiry: Evaluation of options
Audit objective: To determine if the need for the project was properly
established to ensure that the desired results were achieved
at an appropriate cost.
A formal project needs-definition should be carried out prior
Criterion: to commencement of the project.
Documentation review

Audit procedures: 1. Review the project needs-definition segment of the project

appraisal report.
2. Obtain information to verify whether survey of users was
conducted, traffic volume statistics were collected and
anticipated growth in traffic determined.
3. If verification at Step 2 above is positive, determine whether
the data was gathered and reported in a systematic manner
supported by appropriate methods.
4. Conclude whether the criterion has been met
5. Note any exceptions.

3.6.1. Detailed Test Program

The detailed audit program should:
 establish a clear relationship between audit objectives, audit methodology, and the
anticipated field work to be carried out;
 identify and document the procedures to be p erformed;
 provide a basis for more detailed audit planning once the audit team has started to
collect and analyse information; and
 facilitate supervision and review.
As a minimum, the detailed audit plan should set out and specify:
 objectives and sub-objectives of the activity being audited; these should include the
explicit objectives of the activity but also the universal implicit ones of economy,
compliance, financial regularity etc unless they have been excluded from the scope of
the audit.
 audit criteria to be applied to test whether the objectives are being met;
 the timeframe of the activity to be covered;
 procedures/techniques for collecting the evidence;
 the allocation and timing of tasks to be performed by audit team members; and
 other specific requirements as necessary.
Figure 5. Example of Detailed Test Program
Line of Enquiry: Project options should be evaluated against project objectives
Criterion: A formal project needs-definition carried out Done By Date
prior to commencement of the project
Audit Procedures:
▪ Review the project needs definition segment of
the project appraisal report.
▪ Obtain information to verify whether a survey of
users was conducted, traffic volume statistics
were collected and anticipated growth in traffic
determined.
▪ If verification at step 2 above is positive,
determine whether the data was gathered and
reported in a systematic manner supported by
appropriate methods
▪ Conclude whether the criterion has been met
▪ Note any exceptions

As the processes evolve gradually throughout the audit, the auditor must perform further
revision of the detailed audit program to ensure the validity (methods should measure what
they are intended to measure) and reliability (findings should remain consistent if studies
are made repeatedly in the same environment) of methods to be used to collect and
analyse data.
The level of detail will depend on a number of factors:
 the number of criteria to be tested;
 the complexity of the audit issues to be tested;
 the extent of the audit. For example, for audit carried out in different geographical
locations, a detailed plan will be required to ensure consistency; and
  the level of the staff carrying out the audit. Where junior staff have responsibility for
carrying out fieldwork, a more detailed plan would normally be appropriate.
In carrying out planned audit procedures, additional information not explicitly covered by
the audit plan may come to light. In this case, the auditor should identify and highlight the
issues discovered and consult with audit management to assess whether the audit
program should be modified.
It is also important that auditors weigh the cost of obtaining information and the
additional value of the information to the audit. The auditor is intent on gaining enough
evidence to support an argument that the control objective is being met, or that it is not.
He or she should be careful not to gather more information than is needed to support this
argument because this would be a waste of expensive audit resources.

In the Figure 5 example, the test program may be:

‘Select a sample of ten purchases over the threshold amount from the general ledger
for the twelve months to 30 June.
Obtain the documentation for each purchase from the accounts payable section.
Review the tender board minutes for each purchase to ensure that each one has been
through the tendering process.’
Once an audit is underway, new issues may arise that require further reconsideration and
revision of the initial audit plan and specified criteria. However, before adding new issues,
the likely impact on the audit budget and timetable must be considered. As noted
previously, if these additions involve an increase in the scope of the audit, then the auditee
should be formally advised.
IV.7. Fieldwork
The main steps in the fieldwork stage of the audit are:
 gathering and verifying audit evidence;
 analyzing audit evidence; and
 preparing audit working papers.
IV.8. Audit Evidence
The INTOSAI Auditing Standards Glossary defines Audit Evidence as ‘Information that
forms the foundation which supports the auditor's or SAI's opinions, conclusions or
reports.’
I. Standard of Evidence
The standards for performance audits require that the auditor retain a record of the work in
working papers, have procedures for their preparation, and support the findings and conclusions
in the report through evidence.
 Audit procedures should be aimed at fulfilling the INTOSAI Field Standards 300, 0.3(d), which
states: ‘Competent, relevant and reasonable evidence should be obtained to support the
auditor's judgement and conclusions regarding the organization, program, activity or function
under audit.’

Evidence should also be sufficient for the auditor to form an opinion

II. Competency of Evidence
Evidence is competent (valid and reliable) if it actually represents what it purports to
represent. The reliability of evidence can be ensured and assessed by considering that
corroboration of evidence is a powerful technique for increasing reliability. This involves
the auditor looking for different types of evidence from different sources:
 evidence sourced from outside the agency is normally viewed as more reliable for
audit purposes than information generated within the agency;
 documentary evidence is usually considered to be more reliable than oral
evidence;
 evidence generated through direct auditor observation or analysis is more reliable
than indirectly obtained evidence;
 the reliability of agency-generated information will be a function of the reliability
of the agency’s internal control systems and its reliability will need to be assessed
if audit teams use it as a source of evidence;
 oral evidence that is corroborated in writing is more reliable than oral evidence
alone; and
 original documents are more reliable than photocopies (if originals are reviewed
the auditor should note the source of the original and the date copied).
Photocopies of evidence regarded as being of significant importance to an audit
should, whenever possible, be certified by relevant authorities, unless the auditor
has sighted the original document.

III. Relevance of Evidence

Relevance requires that the evidence bear a clear and logical relationship to the audit
objectives and to the criteria. As noted in the previous chapter, one approach to planning
for data collection is to list, for each sub-criterion, the nature and location of evidence that
is needed, as well as the audit procedure that is to be implemented.

IV. Reasonable and Sufficient Evidence

Reasonable evidence is economical in that the cost of gathering it is commensurate with
the result which the auditor or the SAI is trying to achieve. Sufficient evidence would be
that amount of information needed for a knowledgeable person to understand and accept
the auditor’s conclusions. Auditors should prepare and maintain audit documentation in
the form of working papers. Working papers which record the work done in planning,
conducting, and reporting on the audit should contain enough information to enable an
experienced auditor, who has had no previous connection with the audit, to understand
the evidence that supports the auditors’ judgments and conclusions. Audit managers
should ensure that working papers provide persuasive support for findings, conclusions,
and recommendations before the report on the audit is issued.
Auditing standards require that the auditor should obtain sufficient appropriate audit
evidence to be able to draw reasonable conclusions on which to base the audit report.
Auditors need to continually ask themselves two questions. ‘Does this evidence relate to
audit criterion being assessed?’ and ’Do I have enough evidence to form a conclusion?’
The first of these questions is simple to answer and is a matter of logic and judgement.
The second is more complex and requires the auditor to consider whether the evidence
would persuade a reasonable person that the finding should be accepted. In deciding how
much weight the ‘reasonable person’ would place on the evidence, the auditor must take
account of the status of the document. For example, first hand evidence is better than
hearsay evidence, documentary evidence is more convincing than oral evidence, final
documents are more conclusive than draft documents. The other factor to consider is
how much evidence will be needed to persuade stakeholders of the validity of the finding.
The factors that dictate the strength of evidence required to support an observation in
performance auditing include:
 level of materiality or significance of the observation;
 degree of risk associated with coming to an incorrect conclusion;
 experience gained in previous audit examinations on the degree of reliability of the
agency’s records and representations;
 known agency sensitivity to an issue; and
 cost of obtaining the evidence relative to the benefits in terms of supporting the
observation.
Evidence gathered during a performance audit may be predominantly qualitative in
nature and requires extensive use of professional judgement. Accordingly, the auditor
would ordinarily seek corroborating evidence from different sources or of a different
nature in making assessments and forming conclusions.
When planning the audit, the auditor would identify the probable nature, sources and
availability of audit evidence required. The auditor should consider such factors as the
availability of other audit reports or studies and the cost of obtaining the audit evidence.

Characteristics of Performance Audit Evidence

In performance audit terms, audit evidence concerns the facts or information used:
 to conclude whether an agency’s management and employees have carried out
appropriate actions to conform to operational principles, policies or standards for
using resources effectively, efficiently and economically; and
 to demonstrate to a third party that the auditor’s findings are credible and
defensible.
Auditors need to be aware of potential problems or weaknesses with performance audit
evidence. Potential problems include:
 evidence is based on a single source (this may impact on reliability, validity and
sufficiency);
 oral evidence is not supportable by documentation or observation (reliability);
 evidence is time-sensitive, i.e. outdated and does not reflect changes (relevance);
 evidence is too expensive to obtain relative to benefits (relevance and sufficiency);
 the source of evidence has a vested interest in outcome (reliability);
 samples collected are not representative (relevance, validity, sufficiency);
 evidence may be related to an isolated occurrence (validity, sufficiency);
 evidence is incomplete, i.e. does not demonstrate a cause or effect (reliability,
sufficiency); and
 evidence is conflicting (reliability).