GUID 3910

Central Concepts for

Performance Auditing

INTOSAI Guidances are issued

by the International
Organisation of Supreme Audit
Institutions, INTOSAI, as part of
the INTOSAI Framework of
Professional Pronouncements.
For more information visit
INTOSAI www.issai.org

Pre-IFPP document - this document was developed

before the creation of the INTOSAI Framework of
Professional Pronouncements (IFPP) in 2016. It
may therefore differ in formal purpose from more
recent INTOSAI Auditing Guidelines.
 INTOSAI

INTOSAI, 2019
1) Former ISSAI 3100 - Performance Audit Guidelines: Key Principles
endorsed in 2010
2) Content reformulated and endorsed as ISSAI 3100 - Guidelines on
Central Concepts for Performance Auditing in 2016
3) With the establishment of the Intosai Framework of Professional
Pronouncements (IFPP), relabeled as GUID 3910 - Central Concepts for
Performance Auditing with editorial changes in 2019

GUID 3910 is available in all INTOSAI official languages: Arabic, English,

French, German and Spanish
TABLE OF CONTENTS

1. INTRODUCTION 4

2. DEFINITION OF PERFORMANCE AUDITING 6

Economy, efficiency and effectiveness 6

3. CENTRAL CONCEPTS FOR PERFORMANCE AUDITING 8

Independence and ethics 8
Intended users and responsible parties 12
Subject matter 13
Confidence and assurance in performance auditing 14
Forms of providing assurance 15
Audit objective(s) 16
Economy 18
Efficiency 18
Effectiveness 19
Cost-effectiveness 21
Audit approach 22
Audit criteria 25
Audit risk 27
Communication 28
Audited entity 29
Other stakeholders 30
Skills 31
Use of external experts 33
Supervision 34
Professional judgment and scepticism 36
Quality control 42
Quality control (while conducting the audit) 42
Quality assurance (after the completion of the audit) 44
Materiality 45
Documentation 47
1 INTRODUCTION

1) Professional standards and guidelines are essential for the credibility, quality
and professionalism of public-sector auditing. ISSAI 100 Fundamental
Principles of Public-Sector Auditing defines the purpose and authority of
ISSAIs and the framework for public- sector auditing, amongst other things.
ISSAI 300 Performance Audit Principles builds on and further develops
the fundamental principles of ISSAI 100 to suit the specific context of
performance auditing.

2) ISSAI 3000 is the Performance Audit Standard and should be read and
understood in conjunction with ISSAI 100 and ISSAI 300. It provides the
requirements for the professional practice of performance auditing followed
by explanations in order to enhance the clarity and readability of the
standard. ISSAI 3000 is the authoritative standard for performance auditing
and consequently each requirement must be complied with if an SAI chooses
to adopt it.

3) For each requirement set out in ISSAI 3000, supporting non-mandatory

guidance are provided in GUID 3910 Central Concepts for Performance
Auditing, and GUID 3920 the Performance Auditing Process.

4) GUID 3910 is intended to help the auditor interpret central concepts for
performance auditing used in ISSAI 3000. Thus, the guidance provided
in this document should make it easier to understand and implement the
requirements in the performance audit standard.

5) GUID 3910 has two sections. The first one defines performance auditing

4
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

and elaborates on the meaning of economy, efficiency and effectiveness. The

second section deals with central concepts. Some of these concepts are audit-
oriented, such as intended users and responsible parties, subject matter,
confidence and assurance, audit objective(s), audit approach, audit criteria,
audit risk, quality control, and materiality. Some concepts are focused on the
auditor, such as the concept of independence and ethics, skills, supervision,
professional judgment and scepticism. Other concepts are related to key
tasks, such as communication and documentation.

6) GUID 3910 and GUID 3920 should be read together to get a deeper
understanding of how the central concepts are considered throughout the
audit process.

5
2 DEFINITION OF
PERFORMANCE AUDITING

ECONOMY, EFFICIENCY AND EFFECTIVENESS

7) Performance auditing carried out by SAIs is an independent, objective

and reliable examination of whether government undertakings, systems,
operations, programmes, activities or organisations are operating in
accordance with the principles of economy, efficiency and effectiveness and
whether there is room for improvement.

8) The principles of economy, efficiency and effectiveness can be defined as

follows:

a) The principle of economy means minimising the costs of resources.

The resources used have to be available in due time, of appropriate
quantity and quality, and at the best price.

b) The principle of efficiency means getting the most from the available
resources. It is concerned with the relationship between resources
employed and outputs delivered in terms of quantity, quality and
timing.

c) The principle of effectiveness concerns meeting the objectives set and

achieving the intended results.

9) Performance auditing promotes accountability by assisting those with

governance and oversight responsibilities to improve performance. It does
so by examining whether decisions by the legislature or the executive are

6
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

economically, efficiently and effectively prepared and implemented, and

whether taxpayers or citizens have received value for money. Performance
auditing promotes transparency by affording the legislature, the executive,
taxpayers and other sources of finance, and those targeted by government
policies and the media, an insight into the management and outcomes of
different government activities. (For further guidelines on the three E’s, see
Audit Objective)

7
3 CENTRAL CONCEPTS FOR
PERFORMANCE AUDITING

INDEPENDENCE AND ETHICS

Requirement according to ISSAI 3000:

The auditor shall comply with the SAI’s procedures for independence and ethics,
which in turn shall comply with the related ISSAIs on independence and ethics.
(ISSAI 3000/21)

GUIDANCE

10) Ethics means the moral principles of an individual that include

independence, integrity, objectivity, professional competence and due
care, confidentiality and professional behaviour. To be independent, and
be seen as such, the auditor needs to be free from situations which could
impair the auditor’s objectivity. Independence comprises:

a) Independence in fact - allows the auditor to perform activities without

being affected by influences that compromise professional judgment;
to act with integrity and exercise objectivity and professional
scepticism.

b) Independence in appearance - the absence of circumstances that would

cause a reasonable and informed stakeholder, having knowledge of

8
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

relevant information, to reasonably doubt the integrity, objectivity,

or professional scepticism of the auditor, or conclude that they have
been compromised.

11) Independence is important in the context of a performance audit because of

key decisions made by the auditor, such as:

a) identifying and deciding on an audit topic;

b) establishing the audit objective;

c) identifying the applicable criteria;

d) determining the methodological approach to the audit;

e) assessing audit evidence and forming conclusions;

f) discussing audit criteria and findings with the audited entity;

g) assessing the positions of various stakeholders; and

h) writing a fair and balanced report.

12) The auditor should be cognisant of any issues or situations that might threaten
the independence of the SAI and/or the members of the audit team.

Requirement according to ISSAI 3000:

The auditor shall take care to remain independent so that the audit findings and
conclusions are impartial and will be seen as such by the intended users.
(ISSAI 3000/23)

GUIDANCE

13) In addition to confirming his/her independence, throughout the audit

process, the auditor need to establish open and good communication with
the responsible party of the audit about its understanding of the auditor’s

9
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

independence. (For further guidelines on this topic, see Intended users and
Communication).

14) Threats to compliance with the relevant independence requirements must be

considered before and during an audit. Threats may fall into one or more of
the following categories:

a) Self-interest. This threat occurs when the auditor could benefit directly
or indirectly from an interest or relationship with the responsible party.

b) Advocacy. This threat occurs when an auditor promotes a position or

opinion to the point that neutrality and/or objectivity may be, or may
be perceived to be, impaired.

c) Familiarity. This threat occurs when, by virtue of a close relationship

with the responsible party, the auditor becomes too sympathetic to its
interests.

d) Intimidation. This threat occurs when the auditor may be deterred from
acting objectively and exercising professional scepticism by threats,
actual or perceived, from the responsible party.

e) Self-review. This threat occurs when any product or judgment from a

previous engagement needs to be evaluated to reach conclusions in
the current engagement.

f) Management participation. This threat occurs when the audited entity

becomes unduly involved in the audit, for example as a team member.

15) The nature of the threats and the applicable control mechanisms necessary to
eliminate them, or reduce them to an acceptable level, will differ depending
on the particulars of the audit.

16) In considering the significance of any particular matter, qualitative factors

ought to be taken into account. A matter may be considered insignificant only
if it is both trivial and inconsequential.

17) If the threat is other than insignificant, available control mechanisms ought to
be identified and, where applicable, applied to eliminate the threat or reduce

10
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

it to an acceptable level. In the event that an auditor does not comply with
independence requirements, the SAI must consider if disciplinary action is
needed, up to and including termination of employment.

18) Applying control mechanisms eliminates or reduces threats to an acceptable

level. Safeguards are necessary when threats identified are at a level where a
reasonable observer would likely conclude that compliance with the relevant
ethical or independence requirements may be compromised.

19) Examples of audit-specific control mechanisms could include the following:

a) involving another person to review the work done or advise as necessary,

without compromising the auditor’s independence. This person could
be someone from outside the SAI, or someone from inside it who was
not otherwise associated with the audit team. The person needs to be
independent of the responsible party and will not, by reason of the
review performed or advice given, be considered part of the audit team;

b) consulting a third party, such as a committee of independent directors,

a professional regulatory body, or a professional colleague;

c) rotating personnel to performance audits of different entities after a

few years to counter the familiarity threat;

d) all individuals working on an audit having to confirm their independence

before commencing work on the audit and consider throughout the
audit; and

e) removing a person from the audit team when that person’s financial
interests, relationships, or activities create a threat to independence.

11
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

INTENDED USERS AND RESPONSIBLE PARTIES

Requirement according to ISSAI 3000:

The auditor shall explicitly identify the intended users and the responsible
parties of the audit and throughout the audit consider the implication of these
roles in order to conduct the audit accordingly.
(ISSAI 3000/25)

GUIDANCE

20) The intended users are the persons for whom the auditor prepares the
performance audit. The legislature, the executive, government agencies,
third parties concerned by the audit report, and the public can all be intended
users. A responsible party may also be an intended user, but it will rarely
be the only one. In many cases the legislature or the executive will be the
primary intended user of an audit report. However, other intended users may
exist, both inside as well as outside the executive. Citizens can use the results
to make better-informed choices and thus could be intended users. Interest
groups, organisations involved in implementing policies, the academic
community and last but not least the media, can all also be intended users in
a specific context. It is advisable to find out who the relevant intended users
are in the very early stages of the audit.

21) A special group of intended users are the experts in a specific audit field.
Authoritative reports benefit from their support. On the other hand, reports
are in danger of losing authority when experts challenge conclusions and
recommendations.

22) The responsible parties are primarily the ones that are supposed to act
upon the conclusions and recommendations in the audit report. The role
of the responsible party may be shared by a range of individuals, each with
responsibility for a different aspect of the subject matter. The responsible

12
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

party may include those responsible for the subject matter being audited in
an operative and/or supervisory role. At the end of the accountability chain
there will always be a responsible party (for example, a minister) that will be
held to account by the legislature for spending and performance in a certain
area.

23) The performance auditor frequently has considerable discretion in the

selection of the subject matter and identification of criteria, which in turn
influences who the relevant responsible parties and intended users are. The
auditor needs to consider the roles of the intended users while designing and
conducting the audit and at the same time maintain their independence.

SUBJECT MATTER

Requirement according to ISSAI 3000:

The auditor shall identify the subject matter of a performance audit.
(ISSAI 3000/29)

GUIDANCE

24) Subject matter refers to what is audited. The subject matter of a performance
audit need not be limited to specific programmes, entities or funds but can
include activities or existing situations (including causes and consequences).
Examples might be service delivery by the responsible parties or the effects
of government policy and regulations on administration, stakeholders,
businesses, citizens and society. The subject matter is determined by the
objective and formulated in the audit questions.

25) In many SAIs, the mandate of performance auditing will stop short of reviewing
the policy bases of government programmes. In these cases, performance
auditing does not question the merit of policy objectives but can rather involve
examinations of actions taken to design, implement, or evaluate the results of

13
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

these policies, and may imply an examination of the adequacy of information

leading to policy decisions.

26) The subject matter needs to reflect the risk and materiality within the audit
area. This is important in order to add value and ensure that the audit is
relevant. Identification of the subject matter will often be done on the basis
of a risk analysis. Whereas the subject matter refers to what is audited, the
scope defines the boundaries of the subject matter.

CONFIDENCE AND ASSURANCE IN PERFORMANCE AUDITING

Requirement according to ISSAI 3000:

The auditor shall communicate assurance about the outcome of the audit of the
subject matter against criteria in a transparent way.
(ISSAI 3000/32)

GUIDANCE

27) All audit work is a type of assurance service, meaning that the auditor provides
reliable and valid information to an intended user (typically the legislature
or the executive) about the activities of a responsible party (typically a
government agency or the executive).

“Reliable and valid information” in this context requires that the conclusions
on the subject matter are logically linked to the audit objective(s) and criteria,
and are supported by sufficient and appropriate audit evidence. To achieve
this, the conclusion(s) must be clearly linked to the audit objective(s) and
audit criteria, and written in a way that enhances the degree of confidence
of the intended users about the evaluation of the underlying subject matter
against criteria.

28) Thus, assurance reports are intended to provide confidence for intended

14
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

users that the audit conclusions accurately reflect the state of the underlying
subject matter. Put simply, intended users of assurance reports should be able
to be reasonably sure that the conclusion(s) are reliable and valid.

29) To achieve this, the conclusion(s) must be clearly linked to the audit objective(s)
and audit criteria, and written in a way that enhances the degree of confidence
of the intended users about the evaluation of the underlying subject matter
against criteria.

30) Reaching an audit conclusion is an inferential exercise involving considerable

judgement. Because of this it is very important that all aspects of the
conclusion(s) are supported by evidence-based findings relative to the
audit criteria. On the other hand, all findings should also be considered when
expressing the conclusion. The logic is this: “Given findings A, B, C and D
compared to the applicable audit criteria X, the natural conclusion is as follows.”

31) Second, in order to reach a conclusion, it is very important that the findings
are based on sufficient and appropriate evidence. If the evidence is flawed in
any way, the findings and conclusion will also be flawed.

32) Providing assurance in this sense requires that conclusions are based on
solid findings compared to the audit criteria, and that findings are based
on solid evidence. However, it is also important to make these links clear to
the intended users. This is done by being clear on how findings, criteria and
conclusions were developed in a balanced and reasonable manner, and why
the combinations of findings and criteria result in a certain overall conclusion
or set of conclusions (ISSAI 100/32). If this is done properly, the intended users
can be confident about the validity of the conclusions. The auditor has then
provided assurance.

FORMS OF PROVIDING ASSURANCE

33) Assurance can be conveyed in different ways. Some examples of how this can
be done include, but are not limited to, the following:

a) through an overall view on aspects of economy, efficiency and

15
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

effectiveness, where the audit objective, the subject matter, the

evidence obtained and the findings reached allow for such a conclusion;
or

b) by providing specific information on a range of points including the

audit objective, the questions asked, the evidence obtained, the audit
criteria used, the findings reached and the specific conclusions.

34) It is good practice to make an informed choice, depending on the

circumstances, regarding how assurance is communicated. It is also
important not to confuse alternative

a) with a formal attestation opinion which explicitly conveys the level of

assurance (ISSAI 300/21). A formal opinion, comparable to the opinion
on financial statements, is not possible in performance audit. Thus, how
assurance is conveyed will fall under the heading of “In other forms” as
described in ISSAI 100/32.

AUDIT OBJECTIVE(S)

Requirement according to ISSAI 3000:

The auditor shall set a clearly-defined audit objective(s) that relates to the
principles of economy, efficiency and/or effectiveness.
(ISSAI 3000/35)

GUIDANCE

35) The audit objective(s) state(s) the purpose of the audit and what the auditor
seeks to achieve by conducting the audit. The audit objective(s) need(s)
to be formulated in a way that makes it possible to conclude whether the
objective(s) has/have been reached after the audit is finished.

16
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

36) It is good practice to establish the audit objective(s) early in the planning
process to assist in identifying the matters to be audited and reported on. The
audit objective(s) determine(s) the subject matter, which is formulated in the
audit questions, and provides the answer for why the audit is taking place.
The audit objective(s) and scope are interrelated and need to be considered
together. Further information on defining the audit objective(s) is provided in
GUID 3920.

37) In the definition of performance auditing the three E’s, economy, effectiveness
and efficiency have a central place. The relation between these three
principles is explained further in figure 1 below. In this figure input refers to the
financial, human, and material resources used for a government intervention
(government undertaking, policy, system, operation, programme, activity
or organisation). Output refers to the products, capital goods and services
which result from a government intervention. Outcome refers to the likely or
achieved effects of an intervention’s outputs. These can be short-term, mid-
term or even long-term (long term effects can also be referred to as ‘impacts’).

Figure 1.

Cost effectiveness

Efficiency

Objectives Inputs Processes Outputs Outcome

Economy

Effectiveness

17
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

»» ECONOMY

38) Auditing economy focuses the audit on how the audited entity succeeded in
minimising the cost of resources (input) taking into account the appropriate
quality of these resources. This type of audit focuses only on the input. The
main question is: “Are the resources used available in due time, of appropriate
quantity and quality, and at the best price?” Quality is an important concept
on the input side (both in economy as well as in efficiency).

»» EFFICIENCY

39) Auditing efficiency focuses the audit on whether the resources used have
been put to optimal or satisfactory use, or whether the same or similar results
in terms of quantity, quality and turn-around time could have been achieved
with fewer resources. Efficiency assesses the relationship between inputs and
outputs. The key questions are: “Are we getting the most output – in terms
of quantity and quality – from our inputs?” or “Could the same output have
been achieved with less input?”

40) Audits of efficiency can be aimed at technical efficiency (for example, can
processes be streamlined to improve performance?), allocative efficiency
(for example, can efficiency be improved by allocating resources differently,
for instance by moving them to instruments that contribute the most to the
output?), or scale or synergy efficiency (for example, can the same output
be realized with less input by sharing means or processes, or even merging
organisations?).

41) Efficiency is a relative concept. A process, instrument or programme is either

more or less efficient than another. This means that for an audit on efficiency,
some kind of comparison is needed. Examples are: comparing similar activities
in comparable entities, comparing one process (in one entity) with the same
process at an earlier point in time, comparing a process before and after
adaptation of policy or procedure, comparing the efficiency of an organisation
with an accepted set of characteristics of efficient organisations. Efficiency-
oriented audits can also examine the processes leading from input to output

18
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

to expose shortcomings in these processes or in their implementation. This can

lead to a better understanding of why processes are efficient, even without
measuring the efficiency itself.

»» EFFECTIVENESS

42) Effectiveness is about the extent to which policy objectives have been met in
terms of the generated output. It is concerned with the relationship between
goals or objectives on the one hand, and outcome on the other. As shown in
the figure the question of effectiveness consists of two parts: first, to what
extent are the objectives met and second, can this be attributed to the output
of the policy pursued. Both parts are discussed below.

Quality

In the definition of the three e’s the quality of output is an important factor.
If the costs of products or services decline while at the same time their
quality has decreased, one can question whether economy, efficiency of
effectiveness are being achieved. It is therefore important to assess if and how
quality is affected when striving for economy, efficiency and/or effectiveness.
Sometimes criteria for quality are clearly defined for specific products or
services and can be used for these assessments. One of other possible ways
of assessing the quality of products or services is by measuring customer
satisfaction.

43) The first question: “to what extent are the objectives of a programme or
policy met?” can be answered by a result-oriented audit. In such audits the
main focus is on the objectives (are they specific and measurable?) and on the
comparison between the actual output and/or outcome and these objectives
(see also the result-oriented approach in paragraph 61).

44) Measurement and comparison of outputs or outcomes against objectives

in a result-orientated approach may give an indication of effectiveness but
does not establish a causal relationship between the outcome and the
programme being audited. Nor does it provide information on the reasons for
performance. The second part of the effectiveness question, however, may

19
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

address the contribution the programme makes to achieving objectives. When

auditing effectiveness, one should try to identify the relationship between
the achievement of objectives and the implemented programme. A useful
instrument for this is the ‘theory of change’.

Theory of change

Describes how and why an initiative or programme is expected to work.

It is more than a schematic description of expected input, output and
results linked with arrows (figure 1). It describes the assumptions behind
the arrows on how the input will lead to the desired output and how this
output will lead to the desired outcome. In other words, how change will
occur. Identifying the theory behind a programme or a policy can be done by
analysing key documents in which the policy is described or by interviewing
those responsible for the policy or programme. It can help the auditor (and
sometimes also those responsible) to obtain a clear description and better
understanding of the assumptions on the causal relationship between the
output and the intended outcome (objectives) of a policy or programme.

45) Effectiveness can be measured by various methods. The most sophisticated

methods compare the situation being addressed before and after the
introduction of the policy or programme and involve measuring the behaviour
of a control group, which has not been subject to the policy or programme
(the counterfactual). This can be done by a randomized trial or as a quasi-
experiment. However this type of method is not always feasible. Sometimes
more qualitative methods are better suited to gain insight into causal relations
between policy or programme and effect, especially for finding answers to
questions such as ‘what works for whom under which circumstances?’. When
drawing conclusions on the causal relation between policy or programme and
effects it is important to be transparent on the strengths and limitations of the
methods used. There are various (evaluation) handbooks providing guidance
in choosing the right methods.

46) Another approach that is often used in performance auditing is not measuring
effectiveness itself but focusing on the conditions that are (thought to be)
necessary to ensure effectiveness. These conditions may include good
management practices and procedures to ensure the correct and timely

20
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

delivery of services (see output in the figure in paragraph 8). Other conditions
might be the extent to which target groups have been reached, or the level of
performance. These types of audits often have the characteristics of a process
based audit or a programme evaluation. Data from performance management
systems can be very useful for these kinds of audits; however assessing the
quality of these data is of great importance.

47) An audit will often focus on only one of the three E’s, since auditing effectiveness
of efficiency often is time consuming and can require specific expertise.
However it is advisable not to examine aspects of economy or efficiency or
effectiveness of activities in total isolation. For instance, looking at economy
without also considering, at least briefly, the outcome of a policy might lead
to cheap but ineffective interventions. Conversely, in an audit of effectiveness,
the auditor may also wish to consider aspects of economy and efficiency: the
outcomes of an audited entity, activity, programme or operation may have
had the desired outcome, but were the resources very costly?

48) An audit does not necessarily have to focus only on the intended effects of
a policy. Unintended effects (positive or negative) can also be relevant for
the auditor to consider. Unintended effects can for instance be revealed
by interviewing the target group of an intervention, critics of the audited
programme or other relevant stakeholders. Addressing unintended effects
might be especially relevant if the auditee seems unaware of these effects or
if the effects are not included in the theory of change as described in the key
documents on the audited policy or programme.

COST-EFFECTIVENESS

49) Cost-effectiveness combines both elements of efficiency and effectiveness

(see figure 1) by analysing the relationship between the outcome of an
instrument, a project or programme and the input, in terms of money and
human capital. The result of this type of analysis can be expressed as a ratio:
cost per unit of outcome or output per unit of cost. An example is the cost
of reducing CO2 emissions. This can be described as X euro/kg avoided CO2
emissions or the other way around as X kg avoided CO2/euro spent.

21
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

Equity and performance auditing

Some SAIs distinguish equity as a fourth e. This describes the principle that
everyone should be able to exercise their civil rights (e.g. Freedom of expression,
access to information, freedom to associate, freedom to vote, gender equality),
and their political and social rights (e.g. Health, education, housing, and
security). Public policies of protection and social development play a key role
in building equity. Equity issues can also be treated as an additional topic in
performance audits or as an effectiveness issue where it is an explicit policy goal
or programme objective. The examination of equity may involve, for example,
equality of access to services, distributional impacts, and impact on regional
disparities.

AUDIT APPROACH

Requirement according to ISSAI 3000:

The auditor shall choose a result-, problem or system-oriented audit approach,
or a combination thereof.
(ISSAI 3000/40)

GUIDANCE

50) The audit approach determines the nature of the examination to be made.
The audit approach is an important link between the audit objective(s), audit
criteria and the work done to collect evidence. Performance auditing generally
follows one of three approaches or a combination thereof:

a) a result-oriented approach, which assesses whether outcome or output

objectives have been achieved as intended or programmes and services
are operating as intended;

b) a problem-oriented approach, which examines, verifies and analyses

the causes of particular problems or deviations from audit criteria;

22
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

c) a system-oriented approach, which examines the proper functioning of

management systems

51) A result-oriented approach deals mainly with questions such as:

‘What is the performance or what results have been achieved, and have the
requirements or the objectives been met?’ In this approach, the auditor
studies the actual performance, results and outcomes and relates those to
the (criteria based on the) policy goals/objectives. The findings will therefore
often be in the form of a deviation from the criteria. Recommendations, if
presented, are often aimed at eliminating such deviations. The perspective is
in that sense basically normative.

52) A problem-oriented approach deals primarily with problem verification and

problem analysis. It has its starting point in a problem or a “known” deviation
from what should or could be. The audit criteria have a less significant role
than in the result-oriented approach. They are mainly used for identifying the
problem(s) as a starting point for the audit. A major task in the audit is to verify
the existence of stated problems and to analyse their causes from different
perspectives (problems related to economy, efficiency, and effectiveness of
government undertakings or programmes). The problem-oriented approach
deals with questions such as: ‘What is the problem? What are the causes of
the problem? To what extent can the government solve the problem? What,
if any, is the programme of the government to solve the problem?

53) The perspective is analytical and instrumental; the aim is to deliver updated
information on the stated problems and how to deal with them. The auditor
can use various methods and is not restricted in his/her analyses. The
approach may either apply the technique of providing answers to audit
questions or focus on testing stated hypotheses. All possible material causes
are considered (only general goals are taken for granted). So proposals to
amend laws, regulations, and structural design of government undertakings
are not excluded, if it is shown that the existing structure gives rise to severe
and verified problems.

54) The system oriented approach is an approach that does not focus primarily

23
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

on the policy or the goals, but on well-functioning management systems as

a condition for effective and efficient policies. Examples of these systems are
financial management systems, evaluation systems, control systems or ICT
systems. This type of audit can use descriptive questions such as:

a) What is the objective of the system?

b) Who are the responsible actors within the system?

c) What are the responsibilities of each actor?

d) Which rules, regulations and procedures are relevant?

e) What are the relevant information flows?

And can be complemented by more evaluative questions such as:

a) To what extent is there a sound plan?

b) Is there a good quality monitoring system?

c) Is the comparison between the information of the monitoring system

and the plan leading to adjustments, if necessary?

d) Is planning, monitoring and adjusting recorded in a systematic way,

ensuring accountability to a higher administrative level?

e) Are processes evaluated periodically in a proper way?

24
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

AUDIT CRITERIA

Requirements according to ISSAI 3000:

The auditor shall establish suitable audit criteria, which correspond to the audit
objective(s) and audit questions and are related to the principles of economy,
efficiency and/or effectiveness.

(ISSAI 3000/45)
The auditor shall, as part of planning and/or conducting the audit, discuss the
audit criteria with the audited entity.
(ISSAI 3000/49)

GUIDANCE

55) The audit criteria represent the standards against which the audit evidence
is judged. Performance audit criteria are reasonable and attainable, audit-
specific standards of performance against which the economy, efficiency,
and effectiveness can be assessed and evaluated to determine whether
performance falls short of, meets or exceeds expectations. The audit criteria
are intended to give direction to the assessment (helping the auditor to
answer questions such as ‘On what grounds is it possible to assess actual
performance?’ ‘What is required or expected?’ ‘What results are to be
achieved – and how?’).

56) In defining audit criteria, the auditor needs to consider that the criteria are
relevant, understandable, complete, reliable, and objective. These attributes
can be described as follows:

a) Relevant audit criteria contribute to conclusions that assist decision-

making by intended users and to conclusions that answer on the audit
questions.

b) Understandable audit criteria are those that are clearly stated,

25
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

contribute to clear conclusions and are comprehensible to the intended

users. They are not subject to wide variations in interpretation.

c) Complete audit criteria are those that are sufficient for the audit
purpose and do not omit relevant factors. They are meaningful and
make it possible to provide the intended users with a practical overview
for their information and decision-making needs.

d) Reliable audit criteria result in reasonably consistent conclusions when

used by another auditor in the same circumstances.

e) Objective audit criteria are free from any bias on the part of the auditor
or the audited entity.

57) The audit criteria can be qualitative or quantitative and may be general or
specific, focusing on what is expected, according to sound principles, scientific
knowledge and best practice; or on what could be (given better conditions)
or on what should be according to laws, regulations or objectives. Diverse
sources, besides legislation, can be used to identify audit criteria, including
regulations, standards, sound principles and best practices, performance
measurement frameworks and organisational policies and procedures.

58) Criteria can perform a series of important roles to assist the conduct of a
performance audit, including:

a) providing a basis on which procedures can be built for the collection of

audit evidence;

b) providing the basis for assessing the evidence, developing audit findings
and reaching conclusions on the audit objectives;

c) helping to add form and structure to observations;

d) forming a common basis for communication within the audit team and
with SAI management concerning the nature of the audit; and

e) forming a basis for communication with the audited entity’s

management.

26
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

59) In performance auditing, the general concepts of economy, efficiency, and

effectiveness need to be interpreted in relation to the subject matter, and
the resulting criteria will usually vary from one audit to another. However,
established criteria may also be useful for other audits of the same audited
entity or for audits of entities with a similar scope.

60) Audit criteria are established by the auditor. However, they must be discussed
with the audited entity (and possibly with other stakeholders) during the
planning phase, or at the latest in the conducting phase of the audit. Discussing
the audit criteria with the audited entity serves to ensure there is a shared and
common understanding of what criteria will be used as benchmarks when
evaluating the audited entity. It is therefore important to clearly define the
criteria that the audited entity will be assessed against.

AUDIT RISK

Requirement according to ISSAI 3000:

The auditor shall actively manage audit risk to avoid the development of
incorrect or incomplete audit findings, conclusions, and recommendations,
providing unbalanced information or failing to add value.
(ISSAI 3000/52)

GUIDANCE

61) Audit risk is the possibility that the auditor’s findings, conclusions and
recommendations may be improper or incomplete, as a result of factors
such as the insufficiency or inappropriateness of evidence, an inadequate
audit process, or intentional omissions or misleading information due to
misrepresentation or fraud.

62) Dealing with audit risk is embedded in the whole process and methodology of
performance audit. To manage audit risk, the auditor needs to:

27
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

a) Identify the risks

b) Assess these risks

c) Develop and implement strategies to prevent and mitigate the risks

d) Monitor audit risk and mitigation strategies throughout the audit and
make adjustments as needed to changing circumstances (i.e. apply a
risk management approach when addressing audit risk).

COMMUNICATION

Requirements according to ISSAI 3000:

The auditor shall plan for and maintain effective and proper communication
of key aspects of the audit with the audited entity and relevant stakeholders
throughout the audit process.
(ISSAI 3000/55)
The auditor shall take care to ensure that communication with stakeholders does
not compromise the independence and impartiality of the SAI.
(ISSAI 3000/59)
The SAI shall clearly communicate the standards that were followed to conduct
the performance audit.
(ISSAI 3000/61)

GUIDANCE

63) The development of good and proper external relations is often a key factor
in achieving effective and efficient audits of government programmes. The
progress and outcome of the audit will be enhanced if the audit team can
obtain good contact and foster confidence by maintaining a fully professional

28
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

approach during the course of the audit. One needs to keep in mind that it is
the SAI that seeks access to sources, data and arguments in an audit. Without
good communication it may prove difficult to create an atmosphere which will
serve that interest.

»» AUDITED ENTITY

64) The communication process between the auditor and audited entity begins at
the planning stage of the audit and continues throughout the audit process,
by a constructive process of interaction, as different findings, arguments and
perspectives are assessed.

65) The auditor needs to inform the audited entity of the audit subject matter,
audit objective(s), audit criteria, audit questions, the time period to be audited,
and the government undertakings, organisations and/or programmes to be
included in the audit, as soon as possible after the decision to start an audit.

66) The communication of these key aspects provides a clear picture of what the
audit is about and why it is undertaken, what the result might be, and how
the audit will affect the audited entity (e.g. time, documentation, resources)
before the audit starts. Furthermore, it creates the basis for exchanging views,
avoiding misunderstandings and facilitating the process. This does not mean
that the audited entity dictates conditions or in any way controls the audit
process, but rather involves establishing a constructive process of interaction.
As a rule, the assistance of individuals from the audited entity is essential to
an effective audit. An active dialogue during the audit with the audited entity,
experts and others makes it easier; for instance, to continuously verify the
auditor’s understanding and preliminary audit findings.

67) The following topics may serve as examples for further discussion between
the auditor and the audited entity:

a) the audit scope, audit criteria, methodology, and the expected audit
process;

b) functions and persons of importance for the data-collection and who

the SAI may liaise with, the expected types and quantities of documents

29
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

that would or could be requested by the SAI;

c) how to keep the audit entity’s management informed about the

progress of the audit and emerging findings;

d) the ability to carry out the audit as planned (e.g. resources, time
schedule, scope).

68) When important audit findings are made during an audit, the SAI needs to
consider communicating the findings with those charged with corporate
governance in a timely manner.

69) Finally, one needs to remember the importance of feedback from the audited
entity on how well the communication process functioned during the audit
and if there is room for improvement. It is also important to follow up on
whether the audited entity found the report to be fair, balanced and useful.

»» OTHER STAKEHOLDERS

70) Good external relations are important not only in the short-term perspective
of getting access to information and achieving a good understanding of the
subject matter; it is equally important in the long-term perspective for an SAI
to gain trust, respect and credibility with stakeholders.

71) Some of the key stakeholder groups are:

a) The legislature

b) The executive (other than the audited entity)

c) The citizens

d) The media

e) The academic community

f) Non-governmental organisations

72) During communications with stakeholders the auditor needs to be, and must

30
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

be seen to be free of influences that would impair the objectivity of the SAI or
the auditor. Accordingly, the auditor must be independent. More information
on independence is provided in the section on “Independence and ethics”.

73) Further information regarding the communication process can be found in

GUID 3920.

SKILLS

Requirement according to ISSAI 3000:

The SAI shall ensure that the audit team has collectively the necessary
professional competence to perform the audit.
(ISSAI 3000/63)

GUIDANCE

74) Performance auditing is a knowledge-based, complex investigatory activity

with professional values occupying a central position. These values include the
importance of the auditor being given the opportunity to develop skills and
attain good quality results in audits. This includes creating an environment
that is stimulating and that furthers quality improvements.

75) To become a performance auditor, a performance audit team-leader or a

performance audit manager, there is a need for a wide range of skills and
disciplines, including:

a) research design,

b) social sciences,

c) scientific investigation/evaluation methods and the experience needed

to apply such knowledge,

d) good knowledge of organisational management,

31
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

e) knowledge of government organisations, programmes, and functions,

f) personal qualities including integrity, creativity, judgment, analytical

skills, team work,

g) ability to communicate clearly and effectively, orally and in writing; and

h) special skills depending on the nature of the specific audit (e.g. statistics,
information technology (IT), engineering) or expert knowledge of the
subject matter concerned.

76) Special knowledge of the different functional areas to be audited might also
prove essential, but advanced skills in accounting and financial auditing are
normally not needed in performance auditing. It is important to ensure that
competence is built step by step, and to stimulate knowledge sharing and
learning in the organisation. On-the-job learning and training can help the
auditor develop the professional knowledge and skills needed for performance
auditing.

77) Often SAIs organise their performance auditing separately from financial
and compliance auditing, with personnel selected for performance auditing
having different backgrounds and skills from those selected for the other
audit streams. To meet the quality requirements, it is a good practice for the
SAI to have a training and staff development programme to ensure that its
staff maintains professional proficiency through continuous education and
training. A key factor in the development process is learning through practical
auditing work.

78) Continuous education and training may include topics such as current
developments in performance audit methodology, research design,
management or supervision, qualitative investigation methods, case study
analysis, statistical sampling, quantitative data-gathering techniques,
evaluation design, data analysis, and reader-based writing amongst others.
It may also include subjects related to the auditor’s fieldwork, such as public
administration, public policy and structure, government administration policy,
economics, social sciences, or Information Technology. It is good practice to
require auditors to maintain their skills by obtaining a certain number of

32
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

continuous professional education hours every year or within a 2-year period.

79) Since performance auditing is a team effort, it is not a good practice to

conclude a performance audit alone, since the issues involved are complex.
Consequently, not all members need to possess every skill mentioned above.
Furthermore, it may not always be possible for an SAI to recruit people who
meet all the requirements. The required skills may therefore be developed
once a person is in service, as long as candidates for appointment have clearly
demonstrated the potential mindset and aptitude for the kind of work that
performance auditing entails.

80) The auditor needs to possess adequate professional proficiency to perform

his/her tasks. The SAIs need to recruit personnel with suitable qualifications,
adopt policies and procedures to develop and train SAI employees to perform
their tasks effectively, prepare written guidance concerning the conduct of
audits, support the skills and experience available with the SAI and review the
internal procedures

»» USE OF EXTERNAL EXPERTS

81) Experts are often used in performance auditing to complement the skills
set of the audit team and to improve the quality of the audit. An expert,
if needed, is a person or firm possessing special skills, knowledge, and
experience in a particular field other than auditing. Before using experts,
the auditor needs to ensure that the expert has the necessary competence
required for the purposes of the audit, and that he/she is informed about
the conditions and ethics required. The expert must be well informed about
rules of confidentiality. Any external experts engaged with the audit also need
to be independent from situations and relationships that could impair the
external experts’ objectivity. Although the performance auditor may use the
work of an expert as evidence, the auditor retains full responsibility for the
conclusions in the audit report.

33
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

SUPERVISION

Requirement according to ISSAI 3000:

The SAI shall ensure that the work of the audit staff at each level and audit
phase is properly supervised during the audit process.
(ISSAI 3000/66)

GUIDANCE

82) Supervision is essential to ensure that audit objectives are met and the quality
of the audit work is maintained. Proper supervision and control is therefore
necessary in all cases, regardless of the competence of the individual auditor.
Audit supervision involves providing sufficient guidance and direction to staff
assigned to the audit, to address the audit objectives and to follow applicable
methodology, while staying informed about significant problems encountered,
and reviewing the work performed. More specifically, supervision includes:

a) Ensuring that all team members fully understand the audit objectives;

b) Ensuring that audit procedures are adequate and properly carried out;

c) Ensuring that the audit evidence is relevant, reliable, sufficient and

documented;

d) Ensuring international and national auditing standards are followed;

e) Tracking the progress of the engagement to ensure that budgets,

timetables and schedules are met;

f) Considering the competence and capabilities of individual members of

the engagement team, whether they have sufficient time to carry out
their work, whether they understand their instructions and whether
the work is being carried out in accordance with the planned approach
to the engagement;

34
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

g) Addressing significant matters arising during the engagement,

considering their significance and modifying the planned approach
appropriately;

h) Supporting the auditor as and when needed to overcome challenges in

the audit;

i) Providing hands-on support in solving issues that arise;

j) Identifying matters for consultation or consideration by more

experienced engagement team members during the engagement;

k) Reviewing the audit work.

83) The nature and extent of staff supervision, the review of audit work, and
evidence of it, varies depending on a number of factors, such as the size of
the audit organisation, the significance and complexity of the work, and the
experience of the staff.

84) All audit work needs to be reviewed by a senior member of the audit team
as the audit progresses and particularly before the audit reports are finalised.
Review brings more than one level of experience and judgment to the audit
task and needs to ensure that:

a) The audit work has been performed according to the audit plan;

b) The nature, timing, and extent of the procedures performed are

consistent with the audit programmes;

c) The results of the audit procedures and evidence obtained are clearly
reflected in the audit documentation and the conclusions reached are
consistent with the results of the work performed;

d) Consultations have taken place, where appropriate, and the resulting

advice documented and implemented;

e) The evidence obtained is sufficient and appropriate to support the

observations, conclusions, and recommendations in the report.

35
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

PROFESSIONAL JUDGMENT AND SCEPTICISM

Requirement according to ISSAI 3000:

The auditor shall exercise professional judgment and scepticism and consider
issues from different perspectives, maintaining an open and objective attitude
to various views and arguments.
(ISSAI 3000/68)

GUIDANCE

85) Professional judgment is the application of relevant training, knowledge and

experience in making informed decisions about the courses of action that are
appropriate in the circumstances of the audit engagement. In performance
auditing, the audit team gathers a large amount of audit-specific information
and exercises a high degree of professional judgment and discretion concerning
the relevant issues.

86) In addition to personnel directly involved in the audit, professional judgment

may involve collaboration with other stakeholders, external specialists, and
management in the audit organisation.

87) Professional judgment is essential to the proper conduct of an audit. The

following are examples of how professional judgment is required in the
context of performance audits:

a) Identifying and evaluating any threats to independence, including

threats to the appearance of independence;

b) Deciding what to audit;

c) Determining the required level of understanding of the audit subject

matter and related circumstances;

d) Determining the objective(s), questions and scope of the audit;

36
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

e) Determining the criteria;

f) Determining the nature, timing, and extent of audit procedures;

g) Determining the review and consultation procedures required for the

audit, and how advice will be addressed;

h) Determining which findings are significant enough to report;

i) Evaluating whether sufficient and appropriate audit evidence has been

obtained, and whether more needs to be done to answer the audit
questions and to conclude against the objective(s);

j) Drawing conclusions based on the audit evidence obtained compared

with the criteria and audit objective(s); and

k) Determining the recommendations to be made.

88) Professional scepticism means maintaining professional distance from the

audited entity and an alert and questioning attitude when assessing the
sufficiency and appropriateness of the audit evidence obtained throughout
the audit. It is vital that the auditor exercises professional scepticism and
adopts a critical approach, makes rational assessments and discounts
personal preferences and those of others. The following are examples of
how professional scepticism is particularly important in the context of a
performance audit:

a) Considering the integrity of management;

b) Questioning responses to inquiries and other information obtained

from management and those charged with governance;

c) Revising risk assessment as a result of identified material or significant

inconsistent information;

d) Planning for sufficient procedures to evaluate the reliability of data to

be used during the audit; and

e) Being alert to audit evidence that contradicts other audit evidence

obtained.

37
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

89) Professional scepticism is often demonstrated in the various discussions held

with the audit team, management, and those charged with governance.
Examples of how professional scepticism can be applied and assessed at each
stage of the audit include, but are not limited to, the following:

a) The auditor assessing the logic of the audit argument, alternative

perspectives and views presented, and amending where necessary
his/her understanding during the course of the audit, rather than just
relying on evidence supporting the final conclusion.

b) The auditor challenging management views, assumptions, not just

accepting them.

c) The auditor assessing the reliability of the source of the documents.

90) The auditor needs to be receptive to views and arguments and to consider
issues from different perspectives. This is necessary in order to avoid errors
of judgment or cognitive bias. Therefore, it is important that the auditor
exercises professional scepticism and adopts a critical approach, makes
rational assessments and discounts personal preferences and those of others.

Requirement according to ISSAI 3000:

The auditor shall assess the risk of fraud when planning the audit and be alert
to the possibility of fraud throughout the audit process.
(ISSAI 3000/73)

GUIDANCE

91) Fraud is defined as an intentional act by one or more individuals among

employees, management, those charged with governance, or third parties
involving the use of deception to obtain an unjust or illegal advantage such as:

a) breach of trust,

b) collusive awarding of grants and contributions,

38
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

c) collusive bidding or awarding on contracts,

d) deceit,

e) dishonest acts,

f) false representation,

g) fraudulent concealment,

h) illegal acts,

i) intentional misstatements,

j) irregularities,

k) kickbacks,

l) secret commissions, and

m) theft.

92) Unlike error, fraud is intentional and usually involves deliberate concealment
of the facts. It may involve one or more members from the audited entity or
third parties. The primary responsibility for the prevention and detection of
fraud rests both with those charged with governance of the audited entity
and with management of the audited entity. The auditor’s responsibilities are
to identify and evaluate the risk of fraud where the risk is significant. The
auditor also needs to determine audit procedures in response to those risks.

93) The auditor needs to maintain professional scepticism during the planning
phase and during the entire audit because, typically, management and
employees engaged in fraud will take steps to conceal the fraud from the
auditor and others inside and outside the audited entity. When conducting
audits, the auditor needs to maintain an awareness of the possibility of
fraud related to the subject matter (for example contracting or grants and
contributions). If the auditor suspects or encounters fraud, he/she has to bring
the matter to the attention of the supervisor and the relevant authorities for
further action.

39
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

Requirement according to ISSAI 3000:

The auditor shall maintain a high standard of professional behaviour.
(ISSAI 3000/75)

GUIDANCE

94) High expectations for the auditing profession include compliance with all
relevant legal, regulatory and professional obligations, and avoidance of any
conduct that might bring discredit to the auditor’s work, including actions
that would cause an objective third party with knowledge of the relevant
information to conclude that the auditor’s work was professionally deficient.
The auditor is expected to apply a systematic audit approach and due care
in all phases of the audit process. Due care generally refers to the care that
a person of normal prudence would have exercised in performing a given
work. This includes adequate care in audit planning, gathering and evaluating
evidence and in reporting findings, conclusions and recommendations. The
audit team and the SAI must exercise due care and concern in complying with
the auditing standards.

95) Legislatures and citizens expect the SAI and its auditors to maintain a high
level of competence. This underscores the need to maintain individual
professional skill and competence by keeping abreast of, and complying
with, developments in professional standards and pertinent legislation. The
expectation to operate with due care requires the auditor to act diligently and
according to applicable technical and professional standards when performing
performance audits. Diligence includes the responsibility to act with care, in
respect of an engagement.

96) A high standard of professional behaviour needs to be maintained throughout

the audit process, from topic selection and audit planning, to reporting. It is
important for the auditor to work systematically, with due care and objectivity.
A good practice is the use of audit programmes which are detailed work plans
to guide the execution of the work.

40
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

Requirement according to ISSAI 3000:

The auditor shall be willing to innovate throughout the audit process.
(ISSAI 3000/77)

GUIDANCE

97) By being creative, flexible, and resourceful, the auditor will be in a better
position to identify opportunities to develop innovative audit approaches for
collecting, interpreting, and analysing information. It is important to recognize
that different stages of the audit process provide different levels of innovation
opportunities. During the planning stage, the auditor may have the greatest
opportunity to innovate while still in the process of determining the best audit
approaches and techniques applicable to the audit.

98) Within audit and the fields of evaluation and social science, methodologies
will evolve and develop, and new techniques and technologies for evidence
gathering and analysis might be established which enhance the quality of
the audit and audit report. As SAIs adopt new techniques and technologies
as result of this, the auditor should be perceptive and willing to try new
techniques and methodologies. Tools like data analytics and data mining
can be used, for example, for identifying trends, patterns and knowledge
from large amounts of data. Using enhanced data analytics can lead to more
focused risk assessments, more efficient execution of the audit, and more
effective reporting. Other examples of innovation include the broadened use
of electronic working papers and knowledge management systems for sharing
information that could be useful to more than one audit team, and the use of
drones for photographic purposes (for example in agricultural performance
audits).

99) An SAI needs to foster an innovative culture, and auditors need to learn from
each other and open their minds to doing things differently. In other words,
an SAI could:

a) Stimulate innovative, low-cost, sustainable and web-based ways for

SAIs to exchange views, documents and experiences;

41
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

b) Encourage collaborative audits of relevant topics and foster

experimentation with new approaches, techniques and reporting;

c) Lead by example in its governance and modus operandi;

d) Seek an independent evaluation of its own governance and modus

operandi;

e) Facilitate activities to develop its capacity to “deliver the message” in

an effective way; and

f) Keep informed about new evaluation methodologies.

QUALITY CONTROL

Requirement according to ISSAI 3000:

The SAI shall establish and maintain a system to safeguard quality, which the
auditor shall comply with to ensure that all requirements are met, and place
emphasis on appropriate, balanced, and fair reports that add value and answer
the audit questions.
(ISSAI 3000/79)

GUIDANCE

»» QUALITY CONTROL (WHILE CONDUCTING THE AUDIT)

100) A quality control system includes policies and procedures designed to provide
the SAI with reasonable assurance that it, and its personnel, comply with
professional standards and applicable legal and regulatory requirements. The
objective is to ensure that audits are conducted at a consistently high level.
Quality control procedures cover matters such as direction, supervision and
review of the audit process and the need for consultation in order to reach
decisions on difficult or contentious matters.

42
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

101) The system of quality control (QC) needs to be designed so it is appropriate

to the SAI’s mandate and circumstances and able to respond to their risks to
quality. For the system of quality control to be effective, it needs to be part
of the SAI’s strategy, culture, and policies and procedures. This way, quality
is built into the performance of the audit and the production of the report,
rather than being an additional process once the report is produced (see
quality assurance below). Maintaining a system of quality control requires
ongoing monitoring and a commitment to continuous improvement.

102) QC procedures need to be an integral part of the conduct of each performance

audit to minimise the risks of error and drive consistency in conduct. Those
procedures need to be documented and include, for example, the various
steps in the audit process, checks to be undertaken (such as management
review, peer review of draft work and editorial review of final reports). It
may be helpful for the SAI to first clearly define the characteristics of what
constitutes a high-quality audit report.

103) A key aspect of any performance audit is the formal and informal consultation
that takes place within audit teams, between audit teams, and with internal
or external specialists. Consultation during the course of an assurance
engagement is important, as it helps to promote quality and improves the
application of professional judgment, as well as reducing the risk of error.
Consultation is advantageous for reaching sound conclusions, for ensuring
that the report is appropriate, fair and balanced and that it adds value. It is a
good practice to document the key consultations that take place, the nature
of the advice received, and the manner in which the audit team deals with
the advice.

104) A key component of QC is an engagement quality control reviewer (EQCR).

An EQCR is an individual, independent from the audit team, who conducts
an objective evaluation of significant matters, including risks identified and
significant judgments made by the audit team, and the team’s conclusions
reached in formulating the audit report. It is a good practice to appoint an
EQCR to high-risk audits, as defined by the SAI.

105) It is difficult for an SAI to develop effective quality control procedures on an

individual basis that can guarantee high-quality performance audit reports

43
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

across the organisation. It is therefore important to develop such procedures

at an institutional level. It is equally important for the auditor to be – and
remain – competent and motivated as well as open to feedback from quality
control. Control procedures need therefore to be complemented by support,
such as on-the-job training and guidance for the audit team.

106) See ISSAI 140 - Quality Control for SAIs for additional guidance on quality
control.

»» QUALITY ASSURANCE (AFTER THE COMPLETION OF THE AUDIT)

107) A quality assurance (QA) process allows audits to be independently assessed

after their completion on a consistent basis against specific criteria. The main
purpose of a QA process is to monitor the SAI’s quality control system as
designed and assess if the appropriate controls are in place and are working
appropriately. Undertaking a QC process outlined above would be step one
that the QA process would review and the SAI can develop its own criteria,
based on its particular circumstances, with examples of criteria-based
questions including:

a) To what extent does the report clearly describe the context within
which the area examined is carried out?

b) To what extent is the report well-structured and well written, and does
it include an effective executive summary?

c) To what extent is the rationale for the scope clearly set out?

d) Is the audit methodology clearly set out?

e) To what extent were the report’s findings, conclusions and

recommendations balanced, logical, consistent and supported by the
evidence quoted?

f) To what extent has the audit been successful in concluding against its
objectives and providing useful information to help improve public
services?

g) To what extent is there sufficient documentation on team

competencies, audit procedures carried out, evidence to support

44
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

findings, consultations done and treatment of comments received,

and supervision?

108) Those carrying out the independent QA could be senior members of the
performance auditing unit (with no involvement in the conduct of the audit)
or external. A peer review, carried out by members of other national SAIs,
might also be considered for this purpose. The benefit of a peer review is
that the members clearly understand the role and responsibilities of the SAI,
while at the same time they have the distance from the SAI to allow their
assessment to be independent. Using external QA provides an opportunity for
the SAI to demonstrate its accountability to stakeholders, understanding that
the main reason for QA is to improve audits, the audit process and the system
of quality control. The SAI can use the results of QA reviews by circulating good
examples of performance audit reports within the SAI for the benefit of all
auditors. Where performance audit reports are found to need strengthening,
senior staff members should assess the QC system to identify which controls
might need strengthening to produce performance audit reports that meet
the standards of the SAI, and work with the audit teams to identify lessons
learned and possibilities for training, mentoring and coaching in specific areas.

MATERIALITY

Requirement according to ISSAI 3000:

The auditor shall consider materiality at all stages of the audit process, including
the financial, social and political aspects of the subject matter with the goal of
delivering as much added value as possible.
(ISSAI 3000/83)

45
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

GUIDANCE

109) Materiality is the relative importance of a matter, in the context in which it is

being considered, that can change or influence the decisions of users of the
report such as legislatures or executive. Materiality can be considered in the
context of quantitative and qualitative factors, such as relative magnitude,
the nature and effect on the subject matter and the interests expressed
by intended users or recipients. In addition to monetary value, materiality
includes issues of social and political significance, compliance, transparency,
governance and accountability. Materiality can vary over time and can depend
on the perspective of the intended users and responsible parties.

110) Qualitative factors may include such things as:

a) Whether a finding is the result of an intentional act (fraud) or is

unintentional.

b) Whether a particular aspect of the programme or entity is significant

with regard to the nature, visibility and sensitivity of the programme or
audited entity.

c) Whether the health or safety of citizens is affected.

d) Whether a finding relates to transparency or accountability.

111) The consideration of materiality is relevant in all aspects of performance

audits. Therefore the auditor needs to consider materiality when selecting
the audit topics, determining the audit objective(s), questions and scope,
defining the criteria, evaluating the evidence, documenting the findings and
developing the conclusions and recommendations.

112) Findings are considered to be material if they, individually or in the aggregate,

could reasonably be expected to influence relevant decisions taken by
intended users on the basis of the auditor’s report. The auditor’s consideration
of materiality is a matter of professional judgement, and is affected by the
auditor’s perception of the common information needs of the intended users.

46
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

113) Quantitative factors relate to the magnitude of the findings that are expressed
numerically. The auditor needs to consider the aggregate effect of individually
insignificant findings.

DOCUMENTATION

Requirement according to ISSAI 3000:

The auditor shall document the audit in a sufficiently complete and detailed
manner.
(ISSAI 3000/86)

GUIDANCE

114) The auditor needs to keep all relevant documents collected and generated
during a performance audit. Examples of the types of records that are
generally expected to be documented for most performance audits include
(a) details of the audit plan and methodology, (b) results of fieldwork and
analysis, (c) communications and feedback with the audited entity, and (d)
supervisory reviews and other quality control safeguards. However, the
particular circumstances of the performance audit, will determine the specific
purpose and context of the audit documentation. This includes substantive
e-mail communications sent to, or received from, an official in an audited
entity or an outside party that are relevant to the audit and are related to the
report. The documentation records who performed the audit work and the
date such work was completed. Documentation of the audit work has to be
sufficient to enable an experienced auditor with no previous connection to
the audit to understand:

a) The nature, timing and extent of the work conducted;

b) The findings of the audit work, and the audit evidence obtained; and

c) Significant matters arising during the audit (for example changes in the

47
GUID 3910 - CENTRAL CONCEPTS FOR PERFORMANCE AUDITING

scope or approach of the audit, decisions regarding a new risk factor

identified during the course of the audit, actions taken as a result of
disagreement between the audit entity and the team), the conclusions
reached thereon, and significant professional judgments made in
reaching those conclusions.

115) If, in the context of a performance audit, the auditor collects personal data or
information, he/she must ensure that it is adequately safeguarded. The nature
and sensitivity of the information are factors in determining what security is
adequate.

116) In determining the nature and extent of the documentation for a particular
audit area or procedure step, the auditor generally needs more audit
documentation when:

a) the risk is high (the risk associated with conducting the audit or when
the finding is significant, sensitive or contentious);

b) more judgment is needed in performing the work or evaluating the

results; and

c) the evidence is more significant (i.e., the evidence is critical to conclude

on the objectives of the audit).

117) It is advisable for the documentation to include a system that cross- references
the audit report to the working papers.

118) The auditor needs to adopt appropriate procedures to maintain the

confidentiality and safe custody of the working papers. The auditor also
needs to retain the working papers for a period sufficient to meet the needs
of the legal, regulatory, administrative and professional record retention
requirements and to conduct audit follow-up activities.

48