Open Source Project 2008

Software Quality Assurance Plan

Version 1.0

Date: August, 2008

 Document Description
Title ID Version
Software Quality Assurance Plan SQAP 1.0
Prepared by: Date

Daniel Schneider (Quality Assurer), by reusing the SQAP created by 26.08.08

Liliana Guzmán for Open Source Project 2007.

Reviewed by: Date

Johannes Schröder, Project Manager

Approved by: Date

Document Change Record

Rev/Ver Description of changes Approved by Date
Level approved
1.1
Introduction

Purpose
The purpose of this Software Quality Assurance Plan (SQAP) is to establish the goals, processes, and
responsibilities required to implement effective quality assurance functions for the Open Source Project
(OSP) 2008.

The SQAP provides the framework necessary to ensure a consistent approach to software quality
assurance throughout the project life cycle. It defines the approach that will be used by Software Quality
personnel to monitor and assess software development processes and products to provide objective
insight into the quality of the software. The systematic monitoring of OSP products, processes, and
services will be evaluated to ensure they meet requirements and comply with OSP policies, standards,
and procedures.

Scope
This plan covers SQA activities throughout all software development phases of the OSP 2008.

Reference Documents
The following documents were used or referenced in the development of this plan:

 OSP Project Plan

Management
This section describes the management organizational structure, its roles and responsibilities, and the
software quality tasks to be performed.

Management Organization
OSP efforts are supported by Technical University of Kaiserslautern (see internal website for a detailed
organizational chart http://ksi-devlab/mediawiki-osp08). Relevant entities/roles that are of interest and
applicable to this SQAP and the software assurance effort are described at the following subsections.

Project Manager
The OSP 2008 project manager (PM) is responsible for management of project objectives and the
project plan. The project manager is specifically responsible for the success of the OSP project, including
but not limited to cost, schedule, and quality.

Quality Assurer
The Quality Assurer (QA) is responsible for supporting the PM in the coordination of the definition and
implementation of a SQAP for the project.

The QA provides project management with visibility into the processes being used by the software
development teams and the quality of the products being built. The quality maintains a level of
independence from the project and the software developers. Risk and problem escalation begins at the
project level, and extends to the PM and the QA.

Responsibilities include, but are not limited to:

 Develop and maintain the project SQAP

 Generate and maintain a schedule of software quality assurance activities
 Conduct process and product assessments, as described within this plan, using objective criteria
 Identify and document noncompliance and problems for all software assurance related activities
 Communicate results from assessments with relevant stakeholders
 Ensure resolution of noncompliance and escalate any issues that cannot be resolved within the
project
 Identify lessons learned that could improve processes for future products
 Develop and maintain metrics

Software Development Process

In this project an iterative process model is used to fulfill the goals of the project described in the project
plan.

According to the project plan, the development is divided into two major phases. In Fig. 1 the process
model is schematically pictured.

First there is a planning phase, where the requirements are set and categorized by their priority
according to the customer. Then the system's basic architecture is developed.

In the second phase, the development of the concrete system is started. The system is developed in
small cycles. At first only the core features of the system are implemented. In the iterations, more
sophisticated features are added depending on the priorities the customer declared in the requirements
specification.
 Planning phase
1. Define requirements
2. Prioritize requirements 3. Develop system
and plan increments architecture

Development phase

7. Validate System false 4. Develop

(System finished?) increment

6. Integrate 5. Validate
increment increment

Figure 1: Schematic process model for iterative development used in this project

Software Development Process – Quality Assurance Plan

For the Quality Assurance Plan, the planning phase is again separated into two phases to get a more
strict separation between the “paperwork” and the executable system. Fig. 2 shows the new schematic
process model that is used.

In this project we use two categories of quality assurance techniques. At first there are constructive
quality assurance techniques. They are called constructive because their intent is to avoid mistakes
made by the developers. The idea is, if the techniques are used correctly, there should be less severe
defects in the upcoming system, because less conceptual defects are present in the system. The other
category insists of analytic quality assurance techniques. The intent of these techniques is to reveal
already existent defects in a program. There are kinds of defects that can be revealed by both or one of
them so both strategies are important.

There exist several methods for each technique that can be applied in specific phases of the software
development process.
 Planning phase Intermediate phase

1. Define 2. Prioritize requirements 3. Develop system

requirements and plan increments architecture

Development phase

7. Validate System false 4. Develop

(System finished?) increment

6. Integrate 5. Validate
increment increment

Figure 2: Schematic process model for iterative development used for quality assurance in this project

Planning Phase
In the first phase, all requirements are in the focus of the development team. This phase is important to
be able to plan the increments and to be able to develop a flexible and well-suited system architecture.

Constructive quality assurance techniques are a good choice for this phase. Therefore formal inspections
are used to check the requirements.

Intermediate Phase
The development of the system architecture can be seen as an intermediate step between planning and
development, because it logically belongs to both phases. It belongs to the planning phase because all
requirements have to be focused that the system’s architecture is flexible enough to be able to
implement all requirements in an efficient way. It belongs to the development phase because the basic
components of the architecture can already be developed. There could be general components in the
system, e.g. non-functional components that can already be implemented.

In this phase, constructive and analytic quality assurance techniques are used.

Development Phase
In the development phase, the defined iteration cycles are performed. In an iteration cycle, only parts of
the requirements are focused. Every iteration cycle has its own set of requirements that have to be
fulfilled. If an increment is developed, it has to be integrated into the base system.

In this phase, the focus is put on analytic quality assurance techniques.

Quality Assurance Tasks
This section summarizes the quality planning, control and steering tasks to be performed during the
development of software. These tasks are selected based on the project plan, planned deliverables, and
identified reviews. The planned tasks include, but are not limit to:

 Quality planning

o Perform SQAP for the project

The current SQAP has been specified according to OSP project plan and based on the IEEE
STD 730-2002, IEEE Standard for Software Quality Assurance Plans, and the NASA Software
Quality Assurance Plan.
o Review and update SQAP

At the beginning of each phase, QA should review the SQAP and update it according to the
project progress and corrective actions.

 Quality control

o Product assessment:

The product assessment start with the identification of the minimum documentation
required for the current project, approval mechanism and approval criteria (See section
Documentation). It also includes reviews of the major work product (See section Reviews)
and testing (See section Testing).

o Process assessment

The process assessments will be covered project planning, project monitoring and control,
reviews, requirements specification and management, architectural design, coding, test
design and execution, and risk management

 Quality steering

o Software problem reporting and corrective actions

o Lessons learned

Table 1 provides a summary of software quality assurance tasks during the project phases. At first the
SQAP is created and maintained over the whole project. For quality steering, the project progess is
tracked. If problems seem to come up, the PM is informed.

Quality Tasks Planning Phase Intermediate Phase Development Phase

Quality Planning Create SQAP for current Review SQAP for Review SQAP for
and next phase, create current phase, extend current phase
templates and SQAP for next phase
guidelines

Quality Control Assess process Assess process Assess process

adherence and see adherence and see adherence and see
Table 2 Table 3 Table 4

Quality Steering Report problems and Report problems and Report problems and
keep track of the keep track of the keep track of the
project project project

Table 1 Software Quality Assurance Tasks Matrix

Quality Control Tasks Planning Phase

Documentation Assess requirements specification documents

Reviews Perform a formal review of the requirements specification, validate the

database schema against the requirements

Testing ---

Table 2 Software Quality Control Tasks for Planning Phase

Quality Control Tasks Intermediate Phase

Documentation Assess the software architecture

Reviews Perform a structured walkthrough for the use cases that were identified in
the requirements specification

Testing Develop a test environment for the software

Table 3 Software Quality Control Tasks for Intermediate Phase

Quality Control Tasks Development Phase

Documentation See Test documentation

Reviews See Test documentation

Testing See Test documentation

Table 4 Software Quality Control Tasks for Development Phase

Software Assurance Estimated Resources

The staffing resource levels provided in the table below represent what the Quality Assurance has
currently planned.

Intermediate Phase

Development Phase
Planning Phase

Total
(Effort in PD)
Task Support personnel

(Effort in PD)

(Effort in PD)

(Effort in PD)
Quality planning Quality Assurer 5 2 2 9

Quality control Quality Assurer 5 4 10 19

Project Manager 2 2 2 6

Software Architect 1 3 1 5

Development Manager 0 2 3 5

Requirements Engineer 3 2 1 6

Database Expert 2 2 1 5

Quality steering Quality Assurer 3 3 3 9

Table 5 Effort for Quality Assurance

Documentation
This section identifies the minimum documentation governing the requirements, developments,
verification and validation of software that falls within the scope of this SQAP. Each document will be
assessed and approved by the following mechanism:
 Approval mechanism

Input: Any project document

Input criteria: The document is completed and includes no obviously known defects.

Participants/ responsibilities:

Participant Responsibilities
 Project Manager To initiate the process and to provide the document
To review the document
To plan and supervise corrective actions
Quality Assurer To coordinate the process
To review the document
To suggest and monitor corrective actions
To assure the completion of the corrective actions
Others To review the document

Procedure:

1. According to the project plan, the PM submits the document to approval.

2. The PM and the QA and other possible reviewers compare the document according to the
corresponding checklist.

3. If there are defects found in the document, corrective actions have to be planned. The QA
should assure the completion of the corrective actions. A defect can either be major or minor,
depending on the decision the reviewers make.

 If there were major defects in the document they have to be fixed and another review
has to be done with the corrected document. The document cannot be accepted.

 In the case of minor defects the document can be accepted under the condition that
the defects are to be fixed. No additional review of the document is needed.

4. The document is accepted and can be submitted to the persons that are interested in this
document.

Associated documentation:

 Approval resolution template

 Checklist according to the document

Exit criteria: The document is approved by the review team.

Exit: Approval resolution, Approved document.

The following table identifies the minimum documentation required for the project, the approval
criteria and the responsible persons.

Approval criteria Responsible persons

Project manager

Client
Requirements engineer

Development Manager
Software Architect
Correctness

Quality Assurer

Database Expert
Verifiable

Modifiable

Traceable
Prioritized
Completeness

Consistency

Non-ambiguity
Document

Project plan X X X X R A
Software
Quality
X X X X A R
Assurance
Plan
Requirements
X X X X X X X X A A R A
Specification
Database
X X X X X X A A A R
Schema
Architectural
X X X X X X A A R
Design
Component
X X X X X X A A A R
Design
Test Plans X X X X X X X X A R
Test Reports
X X X X X X A R A
and Artifacts
Software
X X X X X A A A R
User’s Guide
R: Responsible; A: Approve;

Standards, practices, conventions and metrics

This section highlights the standards, practices, quality requirements, and metrics to be applied to
ensure a successful software quality program.
Software Quality Program
The specific procedures, work instructions, checklist, and templates used by quality assurance, will be
published at the developer’s site http://ksi-devlab/mediawiki-osp08. They include, but are not limit to:

 Procedures/Guidelines: Inspection procedure, Perspective Based Reading, Architectural design

review guidelines, Component design review guidelines

 Work instructions: Testing planning, Design of test cases

 Checklist: Checklist according to the document

 Templates: Approval resolution template, Inspection reports, Test plan template, Test results
template, Problem reports template, Lessons learned template, Assessment results and
corrective actions status

Standard Metrics
The following standard metrics are the minimum planned metrics that will be collected, reported, and
maintained in the area of software quality assurance:

 Effort expended for all phases(Planned vs. Actual)

 Number of SQ assessments, e.g. reviews, test activities(Planned vs. Actual)
 Number of SQ assessment findings or non compliances (Open vs. Closed)

Additional Project metrics may also be collected, reported, and maintained. Sample metrics include:

 Number of approved documents (Planned vs. Actual)

 Number of open vs. closed software problem reports (reported by customer)

 Number of open vs. closed defects (reported by team)

 Number of open vs. closed change request

 Number of requirement changes (related to change requests, but more severe)

Reviews
During the project several major inspections are planned: software requirement specification inspection
architectural design review and component design review. The process for planned inspections is
described in the following table:
 Inspection process

Input: Project artifact

Input criteria: The PM submits the project artifact to inspection.

Participants/ responsibilities:

Participant Responsibilities
Project manager To initiate the process and to provide the product artifact
To provide the required resources to conduct the process
To participate in the inspection
To plan and supervise corrective actions
Quality assurance To coordinate the process
To review the document
To suggest and monitor corrective actions
To assure the completion of the corrective actions
Inspectors To review the document
To prepare the inspection
To participate in the inspection

Procedure:

1. Organization

a. According to the project plan, the PM submits the artifact to inspect.

b. The QA organizes the inspection and distributes the artifact, the reading guidelines,
checklists and the corresponding inspection report to each inspector according to the
inspection plan.

2. Preparation

a. Each inspector reviews the artifact individually according to the reading guidelines,
checklists and fulfills the inspection report.

b. Each inspector sends the inspection report to the QA.

3. Inspection meeting

a. The inspectors, the PM and the QA discuss the detected defects. The QA creates the
final inspection report.

b. QA includes the defects in Wiki.

 4. Rework

a. The PM and the QA discuss and agree corrective actions.

b. The PM assigns the corrective actions and provides the necessary resources to fix the
defects.

c. The QA assure the completion of the corrective actions.

Associated documentation:

 Reading Guidelines

 Checklists (Suitable for the document)

 Inspection report

Exit criteria: The QA and the PM approve the rework of the artifact

Exit: Reworked Artifact, Inspection report.

Inspection Week for Planning Phase

Week: 08 – 12.09.08

Artifacts to check:

 Requirements specification

 Software Architecture

 Database Schema

Requirement specification
Artifact Requirement specification
Purpose To check the correctness of the requirement specification from the point of
view of different involved roles.

Participants Requirements Engineer

Quality Assurer
Project Manager
Other reviewers
Estimated date/ Preparation: 09.09.08 (ca 2h, afternoon)
duration Inspection: 10.09.08 (ca. 1,5h) (morning, 9 – 11) (only use cases)
 Rework (Author): 10.09.08 (afternoon, 4h)
Inputs Requirement specification
Technique Formal inspection using Focused Checklists
Roles
 Developer(s)
 Customer
 Tester
 DB-Expert
Procedure/Templates Reading guidelines
Focused Checklist for each role
Inspection report
Output Inspection report

Software architecture
Artifact Software architecture
Purpose To check the correctness and the feasibility of the architecture design based on
the requirements specification
Participants Software Architect
Quality Assurer
Project Manager
Estimated date/ Send away: 11.09.08 (morning)
duration Feedback: XX
Rework: XX
Inputs Software requirement specification
Architectural Design
Technique Expert Review
Procedure/Templates Inspection report
Output Inspection report

Database schema
Artifact Database Schema
Purpose To check the correctness of the database scheme according to the
requirements specification
Participants Database Expert
Requirements Engineer
Quality Assurer
Project Manager
Other Reviewers
Estimated date/ Preparation: 10.09.08 (afternoon, for author, 2h)
duration Meeting: 11.09.08 (afternoon,2h)
Rework: 11.09.08 (1,5h)
Inputs Database schema
Requirements specification
Technique Informal review
The author prepares the topic and presents it to the team. They try to reveal
defects in the meeting.
Procedure/Templates Inspection report
Output Inspection report

Testing
QA is responsible to lead and perform the test plan, including unit testing, integration testing
(verification) and acceptance testing (validation). QA will monitor testing efforts to assure that test
schedules are adhered to and maintained to reflect an accurate progression of the testing activities.
They will assure that tests are conducted using approved test procedures and appropriate test tools, and
that test anomalies are identified, documented, addressed, and tracked to closure. In addition, QA will
assure that assumptions, constraints, and test results are accurately recorded to substantiate the
requirements verification/validation status.

QA personnel will review post-test execution related artifacts including test reports, test results,
problem reports, updated requirements verification matrices, etc.

Problem Reporting and Corrective Action

The QA generates tracks, and trends assessment findings and observations, available at the developer’s
site in the section Quality Assurance (http://ksi-devlab/mediawiki-osp08). See section problem reports
and corrective actions.

Each problem report will be sent to the PM directly. The QA and the PM agree and plan the definitive
corrective actions. The QA is responsible for supervising the completion of the corrective actions.

During the project, the QA informs weekly the assessments results and the corrective actions status to
the project manager.