12/19/2017

AUDITING AGILE IN AGILE TIME

GEMMA BEVILACQUA CGEIT, PMP, PMI-ACP, CSM, CSP
PROGRAM MANAGER, SYSTEM ARCHITECT, UNDISCLOSED

19 December 2017

© 2017 ISACA. All Rights Reserved

WELCOME

• Audio is streamed over your computer Use the Papers tab to find the following:

• Dial in numbers and codes are on the left • PDF Copy of today’s presentation

To receive your CPE credit: • CPE job aid

1. Complete 3 checkpoints • Have a question for the speaker? Access

the Q&A tab
- or -
• Technical issues? Access the Help tab
2. Watch the recorded version from the
beginning to the very end • Questions or suggestions?
Visit https://support.isaca.org
• Don’t forget to take the survey!

© 2017 ISACA. All Rights Reserved

1
 12/19/2017

TODAY’S SPEAKER

GEMMA BEVILACQUA CGEIT, PMP, PMI-ACP, CSM, CSP

PROGRAM MANAGER, INFRASTRUCTURE OPERATIONS MANAGER, SYSTEM ARCHITECT
UNDISCLOSED

© 2017 ISACA. All Rights Reserved

AGENDA

The business case for Agile/Agile Foundation

How Agile is Different/Agile Methodologies

Agile Teams/How is Agile Success Measured?

Common Barriers/Leading Causes of Failed Agile Deliverables

Things to account for in an Agile Audit

Auditing Agile by integrating COBIT 5

WRAP UP

© 2017 ISACA. All Rights Reserved

2
 12/19/2017

THE BUSINESS CASE FOR AGILE

• Software is the principal driver of growth, innovation, efficiency, and

productivity in companies.

• How software is delivered is the key for how responsive companies

can be to increasing market demands.

• Agile software development has grown increasingly popular over the

last decade.

© 2017 ISACA. All Rights Reserved

THE BUSINESS CASE FOR AGILE

• The top reasons agile drives enterprise successes are: accelerating

product and service delivery, building high quality products that
customer’s value, reducing risk, eliminating waste, enhancing
collaboration, increases our ability to manage changing priorities, and
improves project visibility, which is not a surprise as organizations are
responding to increasing customer expectations.

• Agile is built on the foundation of continuous improvement: You need

to inspect, learn from, and adapt your performance to keep improving. 1

© 2017 ISACA. All Rights Reserved

3
 12/19/2017

AGILE FOUNDATION

Agile was codified in 2001 with the Agile Manifesto, which is a set of
values for managing software development that include: individuals and
interactions over processes and tools, working software over
comprehensive documentation, customer collaboration over contract
negotiation, and responding to change over following a plan. 2

© 2017 ISACA. All Rights Reserved

AGILE FOUNDATION

• In 2005 the Agile Project Leadership Network created the Declaration of

Interdependence.

• Which specifies that Agile and adaptive approaches for linking people,
projects, and value.

• It states that we are a community of project leaders that are highly

successful at delivering results.

• To achieve these results: “we increase the return in investment by making

continuous flow of value our focus.

© 2017 ISACA. All Rights Reserved

4
 12/19/2017

AGILE FOUNDATION

• We deliver reliable results by engaging customers in frequent interactions

and shared ownership. We expect uncertainty and manage for it through
iterations, anticipation, and adaptation.

• We unleash creativity and innovation by recognizing that individuals are

ultimate source of value, and creating an environment where they can make a
difference.

• We boost performance through group accountability for results and shared

responsibility for team effectiveness.

• We improve effectiveness and reliability through situationally specific

strategies, processes and practices.” 3

© 2017 ISACA. All Rights Reserved

HOW AGILE IS DIFFERENT

• Agile assumes that resources and time are fixed and scope is variable;
therefore it is value driven.

• In contrast, the traditional waterfall approach assumes that scope is

fixed, resources, and time are variable; therefore it is plan driven. 4

• Agile is the approach most favor in order to deliver in a fast changing

environment.

10

© 2017 ISACA. All Rights Reserved

5
 12/19/2017

AGILE METHODOLOGIES

• Agile is an overarching term for various work management approaches

that share these common principles.

• Some of the more popular methodologies include: SCRUM – it is

simple and useful in several scenarios, Lean – focuses on waste
elimination, Kanban – focuses on waste elimination and continuous
improvement, and Extreme Programming (XP) favors rapid,
incremental development.

• Scrum is the most widely used so that is the best example for COBIT5.

• Other Agile methodologies would use a variation of the SCRUM

example provided below.

11

© 2017 ISACA. All Rights Reserved

AGILE METHODOLOGIES

• SCRUM is the most widely practiced methodology.

• Its focus is based on three pillars: transparency, inspection, and

adaptation. It works based on self-organizing teams delivering working
software in a sprint (set period of time).

• Team members commit to finishing a set number of features (items

from the backlog) in each sprint.

• The team gathers daily for a stand up meeting (15 min meeting) where
they answer three questions: what have you done yesterday, what will
you do today, and do you have any obstacles.

12

© 2017 ISACA. All Rights Reserved

6
 12/19/2017

AGILE METHODOLOGIES

• Sprint planning is a collaborative effort involving a ScrumMaster, who

facilitates the meeting, a Product Owner, who clarifies the details of the
product backlog items and their respective acceptance criteria, and the
entire Agile Team, who define the work and effort necessary to meet
their sprint commitment.

• The sprint retrospective is a meeting facilitated by the ScrumMaster at

which the team discusses the just-concluded sprint and determines
what could be changed that might make the next sprint more
productive.

13

© 2017 ISACA. All Rights Reserved

AGILE METHODOLOGIES

• An information radiator is a large graphical representation of project

information kept plainly in sight within an agile development team’s
shared workspace.

• The term is generic rather than specific: information radiators can

include most types of charts used in agile development.

• Burn down charts, task boards, planning boards and storyboards are
among the possibilities.

• An information radiator is usually hand-drawn or printed but can also

include computer-generated charts and electronic displays.

14

© 2017 ISACA. All Rights Reserved

7
 12/19/2017

AGILE METHODOLOGIES

• The purpose of information radiators is to help keep the team focused on

what really needs their attention and to promote transparency.5

• An information radiator is usually hand-drawn or printed but can also

include computer-generated charts and electronic displays.

• The purpose of information radiators is to help keep the team focused on

what really needs their attention and to promote transparency.5

• Agile teams are optimally sized with seven plus or minus two team
members.

15

© 2017 ISACA. All Rights Reserved

AGILE TEAMS
• ScrumMasters are servant leaders that help remove daily obstacles.

• They use collaboration and facilitation techniques to help teams execute.

• Product Owners represent the needs of the customer.

• They decide what features go on the backlog and prioritize them.

• They own the vision of the product and represent the customer’s
interests.

• Team members such as developers, testers, UX, or other roles collaborate

to create a finished product.

16

© 2017 ISACA. All Rights Reserved

8
 12/19/2017

HOW IS AGILE SUCCESS MEASURED?

• Success is measured by on time delivery of projects, product quality,

and consumer satisfaction.6

• On a daily basis success is measured by Information radiators such as

burn up charts, or burn down charts to measure velocity and work in
progress (WIP).

17

© 2017 ISACA. All Rights Reserved

COMMON BARRIERS

• Common barriers to Agile adoption are culture, for example the ability
to change, resistance to change, and management support.7

• Agile can be best scaled by using consistent process and practices,

implementation of common toolsets across teams.

• Agile approaches can help by obtaining user feedback faster but they
can still miss important requirements.

18

© 2017 ISACA. All Rights Reserved

9
 12/19/2017

LEADING CAUSES OF FAILED AGILE DELIVERABLES

• Misunderstood and missed requirements are the leading reasons for

delays, rework, and quality issues.

• To avoid missed requirements invest more time in checking for their

understanding before they begin work.8

• Accounting for all requirements will produce benefits including; alignment

between the business and IT, higher customer satisfaction due to faster
delivery, higher quality and lower cost.

19

© 2017 ISACA. All Rights Reserved

LEADING CAUSES OF FAILED AGILE DELIVERABLES

• As organizations are striving to deliver faster they are sacrificing

quality and missing requirements.

• This will cause significant rework, through defects which increase

costs, and cycle times.

20

© 2017 ISACA. All Rights Reserved

10
 12/19/2017

THINGS TO ACCOUNT FOR IN AN AGILE AUDIT

• Due to the sprints used in Agile/Scrum, it is important to calendar

the audit to the start date of the activities for the next sprint.

• This will provide ample time to validate the audit purpose with the
audit committee and stakeholders.

• Write an audit plan based on a risk control matrix and the dedicated
test plans.

21

© 2017 ISACA. All Rights Reserved

THINGS TO ACCOUNT FOR IN AN AGILE AUDIT

• Plan to participate in the meetings as they are key information drivers regarding the
effectiveness of the program.

• Sprint Review has two parts: the demo when the developers show the work done to
the customers, end-users and management, and the Stakeholders Inspect-and-Adapt
process when you got the Sprint outcome approval and functional changes add-ons.

• Retrospective meeting gives you an overview on how the team is self-managed, how
people are engaged, lessons learned or process improvements.

22

© 2017 ISACA. All Rights Reserved

11
 12/19/2017

THINGS TO ACCOUNT FOR IN AN AGILE AUDIT

• Sprint Planning meeting gives a realistic picture on the alignment

between business and development.

• Daily scrum meeting is a 15 min touch base meeting.

23

© 2017 ISACA. All Rights Reserved

THINGS TO ACCOUNT FOR IN AN AGILE AUDIT

• Other focus areas are: Roles and responsibilities, artifacts, quality

insurance, risk management, communication and governance,
acceptance criteria: definition of done (DOD), Level of Done (LOD)
Definition of Ready (DoR), Lessons learned, Team dynamics, integrated
testing, management and operations involvement.

24

© 2017 ISACA. All Rights Reserved

12
 12/19/2017

THINGS TO ACCOUNT FOR IN AN AGILE AUDIT

• Focus on WIP. WIP consumes investment capital and delivers no return on

investment until it is converted into an accepted product.

• It hides bottlenecks in processes that slow overall workflow and mask

efficiency issues.

• It represents risk in the form of potential rework, since there may still be
changes to items until those items have been accepted.

• Review processes around the product backlog

25

© 2017 ISACA. All Rights Reserved

THINGS TO ACCOUNT FOR IN AN AGILE AUDIT

• WIP on the burn down chart

• Findings logs from the sprint retrospectives

• Review outcomes: percentage of tests covered, number of failed

builds, number of failed unit tests, and number of failed acceptance
tests.

26

© 2017 ISACA. All Rights Reserved

13
 12/19/2017

AUDITING AGILE BY INTEGRATING COBIT 5

1. Evaluate, Direct and Monitor

EDM 02: ensure business benefits delivery.9

• This is Product Owner’s main function.

• The Product Owner tracks return-of-investment and time-

to-market.

• The Product Owner also provides information to the

customer or management regarding project completeness

27

© 2017 ISACA. All Rights Reserved

AUDITING AGILE BY INTEGRATING COBIT 5

1. Evaluate, Direct and Monitor

EDM 03: Ensure Risk Optimization.10

• Risk should be assessed all along the Scrum process.

• Specifically , during the Daily Scrum, through the Scrum Board, at Sprint
Planning meetings for scope, functional and business risks, at
Retrospectives for Process, Capacity, Non-Functional and Security Risks,
at Sprint Planning where risk supports Product Backlog prioritization.

• The Scrum Team (Scrum Master, Product Owner, Developers) are all
accountable.

28

© 2017 ISACA. All Rights Reserved

14
 12/19/2017

AUDITING AGILE BY INTEGRATING COBIT 5

1. Evaluate, Direct and Monitor

EDM 05: Ensure Stakeholder Transparency.11

• Transparency is the core of Scrum.

• Lack of transparency is a Scrum issue.

• Stakeholders are involved in the Scrum process at last at Sprint Review

and Sprint Planning.

• The Product Owner is the main relationship manager to customers

(business) during the process.

29

© 2017 ISACA. All Rights Reserved

AUDITING AGILE BY INTEGRATING COBIT 5

1. Evaluate, Direct and Monitor

EDM 05: Ensure Stakeholder Transparency.11

• The ScrumMaster is connected with management.

• They ensure that everything is highly visible to all stakeholders.

• Sprint validation is done by the Product Owner before Sprint Review only
at Scrum Team level.

• Risks regarding information radiators missing, or poor visual management.

These are risks that reduce team dynamics, hide bottleneck and emerging
issues, or over-commitment.

30

© 2017 ISACA. All Rights Reserved

15
 12/19/2017

AUDITING AGILE BY INTEGRATING COBIT 5

2. Align, Plan, Organize

AP006. Manage Budget and Costs. 12

• The Product Owner is accountable for budget and cost tracking.

• Budgets should be monitored/managed regularly during the

Sprint.

• The final review should be part of the Sprint Review.

31

© 2017 ISACA. All Rights Reserved

AUDITING AGILE BY INTEGRATING COBIT 5

2. Align, Plan, Organize

AP007. Manage Human Resources.13

• Scrum Master monitors the development team and reports

any issues to their management so they can be addressed
if Scrum methods are failing.

• Scrum Team members should be dedicated at 100% to the

project.

• Deviations are documented as Risks.

32

© 2017 ISACA. All Rights Reserved

16
 12/19/2017

AUDITING AGILE BY INTEGRATING COBIT 5

2. Align, Plan, Organize

AP008. Manage Relationships.14

• The alignment of IT and Business strategy is managed by

Product Owner.

• The also monitor that the delivery of IT services are in line

with Business requirements at Sprint Review.

33

© 2017 ISACA. All Rights Reserved

AUDITING AGILE BY INTEGRATING COBIT 5

2. Align, Plan, Organize

AP009. Manage Service Agreements.15

• The ScrumMaster checks incidents to be sure they are in line with the
Scrum process.

• The Product Owner checks stakeholder satisfaction, reports quality of

delivered services with Partner Management.

• The Product Owner also validates status for stakeholders.

34

© 2017 ISACA. All Rights Reserved

17
 12/19/2017

AUDITING AGILE BY INTEGRATING COBIT 5

2. Align, Plan, Organize

AP010. Manage Suppliers. 16

• Risk management is managed by both ScrumMaster and Development

Team on Daily.

• The IT services delivery is in line with business requirement through

Sprint inspection by Product Owner.

35

© 2017 ISACA. All Rights Reserved

AUDITING AGILE BY INTEGRATING COBIT 5

2. Align, Plan, Organize

AP011. Manage Quality.17

• Realized benefits are tracked at Sprint ends by the Product Owner.

Cost performance index is updated at portfolio level. Schedule
performance index is part of the Sprint end reporting. Release
Burndown Chart is updated and velocity is determined. DoD policies
are applied: Developers are using LoD: code correspond to standards,
code is clean, re-factored, unit tested, checked in, built, has a number
of unit tests applied. They have a source code library, code standards,
automated builds, and unit test environments. There is a DoD for each
output: work item, Sprint, Release, Integration, Production.

36

© 2017 ISACA. All Rights Reserved

18
 12/19/2017

IN CONCLUSION

The use of the COBIT5 framework in the audit process will help to identify risks and
ensure that key items in the process have the proper control and are not missed.

37

© 2017 ISACA. All Rights Reserved

Questions?

38

© 2017 ISACA. All Rights Reserved

19
 12/19/2017

THIS TRAINING CONTENT (“CONTENT”) IS PROVIDED TO YOU WITHOUT WARRANTY, “AS IS” AND “WITH ALL
FAULTS.” ISACA MAKES NO REPRESENTATIONS OR WARRANTIES EXPRESS OR IMPLIED, INCLUDING
THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR PERFORMANCE, AND NON-
INFRINGEMENT, ALL OF WHICH ARE HEREBY EXPRESSLY DISCLAIMED.
YOU ASSUME THE ENTIRE RISK FOR USE OF THE CONTENT AND ACKNOWLEDGE THAT: ISACA HAS
DESIGNED THE CONTENT PRIMARILY AS AN EDUCATIONAL RESOURCE FOR IT PROFESSIONALS AND
THEREFORE THE CONTENT SHOULD NOT BE DEEMED EITHER TO SET FORTH ALL APPROPRIATE
PROCEDURES, TESTS, OR CONTROLS OR TO SUGGEST THAT OTHER PROCEDURES, TESTS, OR
CONTROLS THAT ARE NOT INCLUDED MAY NOT BE APPROPRIATE; ISACA DOES NOT CLAIM THAT USE OF
THE CONTENT WILL ASSURE A SUCCESSFUL OUTCOME AND YOU ARE RESPONSIBLE FOR APPLYING
PROFESSIONAL JUDGMENT TO THE SPECIFIC CIRCUMSTANCES PRESENTED TO DETERMINING THE
APPROPRIATE PROCEDURES, TESTS, OR CONTROLS.

Copyright © 2017 by the Information Systems Audit and Control Association, Inc. (ISACA). All rights reserved. This
webinar may not be used, copied, reproduced, modified, distributed, displayed, stored in a retrieval system, or
transmitted in any form by any means (electronic, mechanical, photocopying, recording or otherwise).
39

© 2017 ISACA. All Rights Reserved

THANK YOU
FOR ATTENDING THIS
WEBINAR

© 2017 ISACA. All Rights Reserved

20