Software Technology Support Center (STSC)

Helping Government Organizations Buy and Build Software Better

Verification &
Validation of
Requirements
David A. Cook, Ph.D.
STSC/Shim Enterprises

As Of: 5/8/2002 1
 STSC

Here is Edward Bear, coming

downstairs now, bump, bump,
bump on the back of his head,
behind Christopher Robin. It
is, as far as he knows, the only
way of coming downstairs, but
sometimes he feels that there
really is another way, if only he
could stop bumping for a
moment and think if it. And
then he feels that perhaps there
isn’t.

From Software Project Survival Guide, Steve McConnell, 1998

As Of: 5/8/2002 David A. Cook V&V 2

 STSC

Improving the Process:

Verification and Validation

Verification

Are we building the product

right?

Validation

Are we building the right

product?

As Of: 5/8/2002 David A. Cook V&V 3

 STSC

And the Difference Is

Verification Validation

• Means that each “stage” • Means that the overall

of development follows product and process
processes and meets user needs. User
standards. Quality is the satisfaction is the goal.
goal.

As Of: 5/8/2002 David A. Cook V&V 4

 STSC

There Are Two Prerequisites for V&V

• Know who your customers are.

• Understand their different agendas.
• You want customers who can help you in identifying,
verifying and validating requirements.
• Most likely – you need end users and “funders”. Each
helps you in certain key areas.

• Know how your organization develops

software.
• Know your process.

As Of: 5/8/2002 David A. Cook V&V 5

 STSC

Have a Process
• A process gives But we DO
have a
structure to time and process.
We save
effort. every sheet
of paper we
• This permits V&V can find!!
activities to occur at
planned times.
• Consult your friendly
local “pusher” of
CMM, CMMI, ISO
9000.

As Of: 5/8/2002 David A. Cook V&V 6

 STSC

What I Mean Is… You mean – I

should tell the
TRUTH about how I
develop
• Have a process written down
software??
• Follow the process
• Update the process as necessary
• Make sure the process actually works

• Make sure you use a lifecycle

• Follow the lifecycle
• Update and use alternative lifecycles as needed

• In short – know what you are ACTUALLY doing

As Of: 5/8/2002 David A. Cook V&V 7

 STSC

Verification of Requirements

As Of: 5/8/2002 David A. Cook V&V 8

 STSC

Verification of Requirements

• As mentioned earlier – you need to know who

your customers are – because THEY have to
be involved in the verification

• Three components of verification

• Inspections
• Metrics
• Configuration management

As Of: 5/8/2002 David A. Cook V&V 9

 STSC

Verification – Part 1 Inspections

• I would love to cover inspections and

reviews – but time does not permit. If
you are interested, email me and I will
send you a presentation on reviews
and inspections that I co-presented
last year.

• If you can only do ONE inspection on

a project, you get the biggest “bang
for the buck” (ROI) performing
requirements inspections.

As Of: 5/8/2002 David A. Cook V&V 10

 STSC

Inspection

• The problem is that people interpret

requirements in different ways.

• To prevent this, you need to address two

different issues.
• Do we all interpret the requirement the same way?
• Are the requirements complete?

As Of: 5/8/2002 David A. Cook V&V 11

 STSC

Interpretation - Exercise
Count the number of occurrences below of the
letter e . No questions – just count!!

“Any activity you can perform to reduce

testing errors is cost-efficient. Inspections
are very effective, and requirements
inspections provide the the biggest ROI of
any inspection effort.”

ANSWER: the letter e occurs ___ times.

As Of: 5/8/2002 David A. Cook V&V 12

 STSC

Are the Requirements Complete?

• The best way to determine this is to use

checklists to ensure that you are asking the
right questions at inspection time.

• In addition, such things as a trained and

prepared inspection team plus adequate
preparation help.

Ada
95

As Of: 5/8/2002 David A. Cook V&V 13

 STSC

What to Inspect:

• The software requirements specification (or

however you list the requirements).
• The sources or preliminaries for the SRS
(concept of operations) or any documents that
preceded the SRS.
If it is used as a
requirements
reference, then
inspect it!

As Of: 5/8/2002 David A. Cook V&V 14

 STSC

Sample Checklists

• Following are some sample checklists.

• These checklists grew out of research with

several customers.

Let’s see – I have to

pay the grocer,
pay the electricity bill,
pay the mortgage,
and make a car payment.

This is an expensive check list!

As Of: 5/8/2002 David A. Cook V&V 15

 STSC

Requirements Review Checklist

• Is problem partitioning complete?
• Are external and internal interfaces
properly defined?
• Can each requirement be tested?
• Can each requirement be numbered and
easily identified? (Is the requirement
tracable?)

As Of: 5/8/2002 David A. Cook V&V 16

 STSC

Requirements Review Checklist

(Cont.)
• Has necessary prototyping been conducted
for users?
• Have implied requirements (such as speed,
errors, response time) been stated?
• Is performance achievable within constraints
of other system elements?

As Of: 5/8/2002 David A. Cook V&V 17

 STSC

Requirements Review Checklist (Cont.)

• Are requirements consistent with schedule,

resources and budget?
• Is information to be displayed to the user
listed in the requirements?
• Have future issues (upgrades, planned
migration, long-term use) been addressed in
requirements?

As Of: 5/8/2002 David A. Cook V&V 18

 STSC

What You Are Looking For…

Even I can’t remember
• Are requirements that are all of these at once –
so I inspect
• Unambiguous requirements when I
am finished writing
• Complete them!

• Verifiable
• Consistent
• Modifiable
• Traceable
• Usable
• Prioritized (optional)

As Of: 5/8/2002 David A. Cook V&V 19

 STSC

Metrics for
Requirements
• During the
requirements phase,
there are few metrics
that are very useful.

• One simple metric is

simply the % of
requirements that
have been inspected.

As Of: 5/8/2002 David A. Cook V&V 20

 STSC

Metrics for Requirements

• Another useful metric

# Of requirements that reviewers

interpreted the same

Total # of requirements reviewed

As Of: 5/8/2002 David A. Cook V&V 21

 STSC

Configuration Management

“The most frustrating software problems are

often caused by poor configuration
management. The problems are frustrating
because they take time to fix. They often
happen at the worst time, and they are totally
unnecessary”.

-Watts Humphrey,
“Managing The Software Process.”

As Of: 5/8/2002 David A. Cook V&V 22

 STSC

Configuration Management

• Again, time does not permit a complete

discussion of CM. If you want an intro, email
and I’ll send you a presentation.

• Requirements require a
centralized location and
STRICT configuration
management. If you are
“sloppy” here – soon
it all goes downhill.

As Of: 5/8/2002 David A. Cook V&V 23

 STSC

Validation of Requirements

As Of: 5/8/2002 David A. Cook V&V 24

 STSC

Validation Is Difficult for Software

• Based on three concepts

• Testing
• Metrics
• Quality assurance teams

• Testing is the least important

• Difficult to test requirements prior to coding
• Most requirement methodologies (prototyping,
simulation) rely on many assumptions and
simplifications

As Of: 5/8/2002 David A. Cook V&V 25

 STSC

Metrics for Validation

• Metrics can be useful, but

mostly in hindsight.

• If you know how many of

your bugs or how much of
your defect fix time are
requirements-related, you
can adjust inspections and
reviews accordingly.

As Of: 5/8/2002 David A. Cook V&V 26

 STSC

Validation of Requirements

• The best way to validate requirements is to

involve customers in the requirements
inspection process.
• End-users.
• Program office.
• User management.
• End-users are the most effective, but hardest
to include.
• End-users typically only see the “small
picture”, and requirements are written in the
large.

As Of: 5/8/2002 David A. Cook V&V 27

 STSC

Validation Typically Occurs Twice

• During requirements gathering/analysis/

review.
• After coding and during test. This SHOULD
be the function of the QA team.

If Dogs Wrote
Requirements

As Of: 5/8/2002 David A. Cook V&V 28

 STSC

Danger, Danger

• QA and testing are

EXTREMELY EXPENSIVE!

• Anything you can do to shorten the QA/test

phase is useful. If you rely on QA and testing
to find and fix errors – your
software is probably
• Late
• Over budget
• Still full of errors after you deliver it!!

As Of: 5/8/2002 David A. Cook V&V 29

 STSC

Validation Summary

• In summary – validation requires a tie-in

between implementers and users. If you can’t
involve users as much as you would like, then
designate people to (ALAC) Act Like A
customer.

• I have no checklists for validation, but suggest

that you focus on two areas:
• External interfaces to systems.
• Internal interfaces between modules or sub-systems.

As Of: 5/8/2002 David A. Cook V&V 30

 STSC

As Of: 5/8/2002 David A. Cook V&V 31

 STSC

Good Source of Information

• Software Verification and Validation for
Practitioners and Managers, by Steven R.
Rakitin

As Of: 5/8/2002 David A. Cook V&V 32

 STSC

In Short … There Are Solutions!

As Of: 5/8/2002 David A. Cook V&V 33

 STSC

For Additional Information

David A. Cook
STSC/Shim Enterprises
International Sign for
(801) 775-3055 Computer User at Work
DSN 775-3055

david.cook@hill.af.mil

As Of: 5/8/2002 David A. Cook V&V 34