Page 1 of 10

Standardized Protocol for Method Validation/Verification

Standard Operating Procedure
Quality Assurance Unit
Laboratory Services Section - Austin

Table of Contents

I. Purpose ...................................................................................................................................................... 2
II. Scope ......................................................................................................................................................... 2
III. Definitions ................................................................................................................................................ 3
IV. Responsibilities and Authority ................................................................................................................. 4
V. Validation and Verification and Guidelines .............................................................................................. 4
VI. Validation Sample Size ............................................................................................................................. 8
VII. Validation Records .................................................................................................................................. 9
VIII. References and Supporting Guidance Documents (not otherwise listed) ............................................ 9
 Page 2 of 10

I. Purpose
This document establishes guidelines for the minimum requirements to perform a validation or verification
study. Validations must be performed for all non-standard and laboratory-developed methods. Verifications
must be performed for all unmodified standard methods such as EPA and FDA official methods. The DSHS
Laboratory is regulated and/or accredited by/to ISO 17025, TNI, CAP-CLIA, FDA, EPA, and the USDA.
There are many commonalities in the validation and verification requirements of the applicable regulatory
and/or accrediting bodies that the DSHS is subject to; this procedure consolidates the common requirements
as applicable to all validations/verifications and isolates any additional requirements that are specific to a
single standard that may be considered too cumbersome and/or irrelevant to areas not subject to those
specific requirements. The conceptual view of the validation and verification process is shown in Figure 1
below.

Figure 11

II. Scope
This procedure applies to all testing areas of the laboratory regardless of regulatory body. All laboratory
tests must be validated or verified before being placed into routine use for testing and reporting of patient,
animal, environmental, or surveillance samples.

1
Mattocks, Christopher J et al. “A standardized framework for the validation and verification of clinical molecular genetic
tests.” European journal of human genetics: EJHG vol. 18,12 (2010): 1276-88. doi:10.1038/ejhg.2010.101
 Page 3 of 10

III. Definitions
A. Accuracy – Closeness of agreement between a measured value and a true value.

B. Analytic Measurement Range (AMR) - The range of analyte values that a method can directly measure
on the sample without any dilution, concentration, or other pretreatment not part of the usual assay
process.

C. Analytical Sensitivity – The smallest quantity of an analyte that can be reproducibly distinguished
from background levels. Positive agreement as compared to reference method.

D. Analytical Specificity – The ability of a method to detect only the analyte it is designed to detect.
Negative agreement as compared to reference method.

E. AOAC – Association of Official Analytical Chemists

F. ASTM – American Society for Testing and Materials

G. CAP – College of American Pathologists. Deemed to be an accreditation body by CLIA and currently
directs the Laboratory Accreditation Program (LAP), established in 1961.

H. CLIA- Clinical Laboratory Improvement Amendments of 1988. Responsible under the Centers for
Medicare & Medicaid Services (CMS), an agency within the US Department of Health and Human
Services for the regulation of clinical laboratories in the United States.

I. Detection limit – A detection limit is the lowest amount of analyte in a sample which can be detected
but not necessarily quantitated as an exact value. It is often called the limit of detection (LOD) which
is the lowest concentration level that can be determined statistically different from a blank at a
specified level of confidence. It is determined from the analysis of sample blanks. Method detection
limit (MDL) is the minimum concentration of a substance than can be measured and reported with
99% confidence that the analyte concentration is greater than zero. It is determined from analysis of a
sample in a given matrix containing the analyte.

J. Diagnostic Sensitivity – The percentage of subjects with the target condition whose test values are
positive.

K. Diagnostic Specificity – The percentage of subjects without the target condition whose test values are
negative

L. FDA – U.S. Food and Drug Administration

M. Limit of Quantitation (LOQ) – This is the level above which quantitative results may be determined
with acceptable accuracy and precision.

N. Linearity – Linearity is the ability of the method to elicit results that are directly proportional to
analyte concentration within a given range.
 Page 4 of 10

O. Non-standard Method – Refers to a method that is not taken from authoritative and validated sources.
This includes methods from scientific journals and unpublished laboratory-developed methods.

P. Precision – The ability of the laboratory to duplicate results time after time on different days and with
different operators. Measures random error and the precision or imprecision can be expressed in CV%
from the calculated standard deviation SD and mean. Repeat measurements of samples at varying
concentrations, within-run and between run over a period of time should be performed.

Q. Qualitative Method – A method that identifies analyte(s) based on chemical, biological, or physical
properties; method of analysis whose response is either the presence or absence of the analyte detected
either directly or indirectly in a certain amount of sample.

R. Quantitative Method – A method that provides an estimate of the amount of analyte present in the test
sample, expressed as a numerical value in appropriate units, with trueness and precision which are fit
for the purpose.

S. Range – A range is the interval between the upper and lower concentration of analyte in sample for
which it has been demonstrated that the analytical procedure has an acceptable level of accuracy,
precision, and linearity.

T. Reference Range – The set of values that is deemed normal for a sample population.

U. Reportable Range – The range of analyte concentrations that can be measured with acceptable
accuracy and precision to include the Analytic Measurement Range (AMR). This can be determined
by a linearity study for quantitative methods.

V. Ruggedness or robustness – Ruggedness is a measure of an analytical procedure’s capacity to remain

unaffected by small, but deliberate variations in method parameters and provides an indication of its
reliability during normal usage.

W. Validation – A validation is the process of establishing the performance characteristics and limitations
of a method and the identification of the influences which may change these characteristics and to
what extent.

X. Verification – The one-time process performed to determine or to confirm a test’s expected

performance compared to actual results produced by the laboratory.

IV. Responsibilities and Authority

Verification and validation studies must be planned and performed by competent personnel. The
Laboratory Director is responsible for reviewing and approving all validation and verification plans and
final reports. Any modifications to validation or verification plans, such as: changes to the procedure,
acceptance criteria, or the decision to exclude data, must be approved in writing by a Branch Manager or
Lead Technical Manager and the appropriate Quality Assurance Officer prior to making the modification.

V. Validation and Verification and Guidelines

References:
• TNI Standard EL-V1M2-2016-Rev2.1 Section 5.4.5
• TNI Standard EL-V1M4-2017-Rev2.2 Section 1.5 (Chemical Testing)
• TNI Standard EL-V1M5-2016-Rev2.0 Section 1.5 (Microbiological Testing)
 Page 5 of 10

• TNI Standard EL-V1M6-2016-Rev2.0 Section 1.5 (Radiochemical Testing)

• ISO 17025:2017 Section 7.2.2
• CAP Common Checklist items: COM.40250, COM.40350, COM.40475
• 42 CFR § 493.1253 Standard: Establishment and verification of performance specifications.

Standard methods, such as approved procedures from ASTM, FDA, EPA, etc., require only a verification
study prior to use in the laboratory. ISO 17025:2017, TNI 2016, and CAP-CLIA require that procedures
used without modification be subject to independent verification by the laboratory where testing will
occur prior to being placed into routine use. A verification is a provision of objective evidence that a given
item fulfills specified requirements. A verification study demonstrates that the laboratory can meet or
exceed the method specifications of the manufacturer or published standard method.

Non-standard methods, standard methods used outside their intended scope or otherwise modified, and
laboratory-developed methods/tests require validation. Laboratory validations shall be a planned activity
assigned to qualified and authorized personnel and shall be as extensive as is necessary to meet defined
method performance specifications.

Validation and verification plans shall include predefined acceptance criteria. Any change to the plan,
acceptance criteria, or the decision to exclude any data from the final report must be documented,
technically justified, and approved by a QA Officer and Branch Manager or Lead Technical Manager.
When applicable, validation and verification study results should be compared to the reference method or
‘gold standard’ in which the new method is intended to replace or improve. These comparisons may
include statistical analyses such as confidence intervals, linear regression, F-test, and t-test.

Validations shall comply with the requirements included in the references above and should generally
evaluate the following method performance specifications as applicable:

• Accuracy
• Precision
• Reportable Range
• Reference Range
• Sensitivity
• Specificity
• False Positive/Negative Rate
• Interferences
• Carryover
• Detection Limit
• Limit of Quantitation
• Linearity
• Ruggedness
• Sample and reagent stability

Ruggedness is a quality characteristic that contains a potentially unlimited number of sources of

variability that can be tested as part of a validation or verification. Some of the sources of variability that
may be considered for investigation during a study are:

• Instrument to instrument
• Operator to operator
• Changes in sample quantity or concentration
• Reagent lot variability
 Page 6 of 10

• Time of day
• Temperature variations

The size and scope of validation and verification studies vary based on available time, cost, amount of
testing material/template available, future use of method, and whether the method is qualitative or
quantitative. When planning a validation or verification study the regulatory and/or accreditation body
requirements will form the basis of the plan. Often there is a lack of clear guidance in these regulations
and more information is needed to adequately plan a validation or verification. In these cases, it is
necessary to refer to consensus standards such as those available from AOAC, ASTM, and CLSI (among
others) for further guidance. If the regulatory/accreditation body requirements and the consensus standards
fail to adequately address any ‘grey area’ in the validation/verification plan, the final step is to apply
sound technical and statistical practices in consultation with internal and/or external subject matter experts
to address any remaining issues. Records of these consultations shall be retained as part of the final
validation/verification report.

Table 1 summarizes the general guidelines for performance characteristic evaluation of microbiological
test methods from AOAC. Table 2 is adapted from the National Association of Testing Authorities
(NATA) and summarizes the general guidelines for performance characteristic evaluation of chemical test
methods. These tables should be used as guides when planning ISO 17025 and/or TNI validation and
verification studies. Additionally, all TNI validations shall comply with Section 1.5 of the applicable
module listed in the references section above. Table 3 contains validation and verification performance
characteristics that must be determined for clinical testing under CAP/CLIA regulations as applicable.

Table 1: Microbiological Test Method Performance Characteristic Guidelines2

Performance Identification Quantitative Qualitative (P/A) Verification
Characteristic
Accuracy Yes Yes No No
Matrix No Yes Yes Yes
Effects/Interferences
Precision No Yes No Yes
Selectivity No Yes Yes No
Specificity Yes Yes Yes No
Inclusivity Yes Yes Yes No
Exclusivity Yes Yes Yes No
False-Positive Rate No Yes Yes No
False-Negative Rate No Yes Yes No
LOD (sensitivity) No Yes Yes No
LOQ No Yes No No
Ruggedness Yes Yes Yes No
Linearity/Range No Yes No No

2
AOAC International. "How to Meet ISO 17025 Requirements for Method Verification." (2007).
 Page 7 of 10

Table 2: Chemical Test Method Performance Characteristic Guidelines3

Validation Verification
Characteristics to be evaluated Quantitative Qualitative Quantitative Qualitative
Method Method Method Method
Accuracy Yes Yes Yes Yes
Limit of Detection Yes No Yes No
Limit of Quantitation Yes No Yes No
Sensitivity Yes Yes Yes Yes
Selectivity Yes Yes Yes Yes
Range/Linearity Yes No Yes No
Matrix Effects Yes Yes Yes Yes
Precision
Yes Yes Yes Yes
(repeatability/reproducibility)
Ruggedness Yes Yes No No
Measurement Uncertainty Yes No Yes* No
*In cases where a well-recognized test method specifies limits to the values of the major sources of
measurement uncertainty and specifies the form of presentation of the calculated results, the laboratory is
considered to have satisfied the measurement uncertainty requirements by following the test method and
reporting instructions.

Note: Many regulatory/accreditation agencies require ongoing verification of some of the performance
specifications listed above. These ongoing quality control measures should be incorporated into method
SOPs and routine instrument maintenance plans at the appropriate intervals as applicable.

Table 3: Requirements for Clinical Tests (CAP-CLIA)

Performance Characteristic Validation (Laboratory Verification (FDA Approved
Developed Tests and Tests)
Modified FDA Tests)
Accuracy Yes Yes
Precision Yes Yes
Sensitivity Yes No
Specificity Yes No
Reportable Range or AMR* Yes Yes
Reference Range Yes No
Interferences (COM.40500) Yes No
*CLSI EP28-A3c describes the process for determining this performance characteristic

A confusion matrix also should be included for all qualitative clinical tests. An example of a confusion
matrix is show in Figure 2 below.

3
NATA Technical Note 17 “Guidelines for the validation and verification of quantitative and qualitative test methods” (2012)
 Page 8 of 10

Figure 2: Example Confusion Matrix

Predicted Values
Actual Values Positive Negative
Sensitivity
Positive True Positive (TP) False Negative (FN)

Specificity
Negative False Positive (FP) True Negative (TN)

Precision Negative Predictive Accuracy

Value

There are situations where a validation or verification may need to be repeated. Some examples are:
relocating an instrument, placing a new instrument into service, and modifications to methods. Some
general guidelines for determining if a modification to a method is substantial enough to require a new
validation or verification are provided in the references below:

• Appendix 5 and Appendix 6 of the FDA Guidelines for the Validation of Chemical Methods in
Food, Feeds, Cosmetics, and Veterinary Products, (3rd ed. 2019)
• Attachment A of ORA Laboratory Procedure 5.4.5: Methods, Method Verification, and Validation

VI. Validation Sample Size

References:
• CLSI EP5-A2
• CLSI EP10-A3
• CLSI EP12-A2
• CLSI EP15-A2
• CAP Common Checklist COM.40350
• FDA Guidelines for the Validation of Analytical Methods for the Detection of Microbial
Pathogens in Foods and Feeds, (3rd ed. 2019)
• FDA Guidelines for the Validation of Chemical Methods in Food, Feeds, Cosmetics, and
Veterinary Products, (3rd ed. 2019)
• B. Magnusson and U. Örnemark (eds.) Eurachem Guide: The Fitness for Purpose of Analytical
Methods – A Laboratory Guide to Method Validation and Related Topics, (2nd ed. 2014)

The number of samples included in a validation or verification study determines the statistical validity of
the results and thus determines how much confidence can be placed in the results of the validation study.
Definitive guidelines for validation/verification study sample numbers are limited. The references listed
above give general guidelines for the number of samples that should be included in a study with the
exception of the CAP, which sets the minimum at 20 samples for validation studies unless fewer samples
are explicitly authorized by the Laboratory Director. Validations and verifications are a balance between
costs, risks, and technical limitations. Generally, a higher quantity of test samples is desired and produces
increased levels of statistical significance, however, it may not always be possible to have a large sample
 Page 9 of 10

size due to lack of availability or prohibitive costs. Table 4 below demonstrates the increased statistical
validity of a qualitative test as the number of samples increases.

Table 4: Samples required to determine false positive/negative rates (qualitative test)4

Confidence Level
FN or FP Rate 80% 90% 95% 99%
<1% 161 230 299 459
<2% 80 114 149 228
<5% 32 45 59 90
<10% 16 22 29 44

VII. Validation Records

The validation/verification plan template (DSHS Quality Assurance XXX Lab-Wide
Validation/Verification plan template) and the validation/verification report template (DSHS Quality
Assurance XXX Lab-Wide Validation/Verification report template) may be used for validations and
verifications as applicable.

The following records shall be retained for all validations and verifications:

A. Validation/verification procedure/plan;
B. Specification of the requirements, including acceptance criteria;
C. Determination of the performance characteristics of the method;
D. Results obtained (including an explanation and investigation of discrepant/discordant results and any
excluded data); and
E. A statement on the validity of the method, detailing its fitness for the intended use.

Note: All validations and verifications (regardless of accrediting/regulatory body) shall comply with the
following technical records requirement from ISO 17025: ‘Records shall contain sufficient information
to facilitate, if possible, identification of factors affecting the uncertainty and to enable the test to be
repeated under conditions as close as possible to the original.’ This requires traceability to all aspects of
the work performed including, but not limited to the following:

• Identity of personnel performing the work;

• Identity of all instruments, including pipettes;
• Reagent lot numbers and expiration dates; and
• Environmental conditions

Validation and verification records shall be kept for at least the life of the test plus two years.

VIII. References and Supporting Guidance Documents (not otherwise listed)

A. Jones, S., Carley, S., Harrison, M. An introduction to power and sample size estimation. Emerg Med J
20, 453-458 (2003)
B. Molecular Diagnostic Assay Validation (Update to the 2009 AMP Molecular Diagnostic Assay
Validation White Paper), Association for Molecular Pathology, September 2014
C. CLIA-Compliant Analytical Method Validation Plan and Template for LRN-C Laboratories, APHL
(2013)

4
FDA Guidelines for the Validation of Chemical Methods in Food, Feeds, Cosmetics, and Veterinary Products, (3rd ed. 2019)
 Page 10 of 10

D. FSIS Guidance for Test Kit Manufacturers, Laboratories: Evaluating the Performance of Pathogen Test
Kit Methods, USDA (2010)
E. B. Magnusson and U. Örnemark (eds.) Eurachem Guide: The Fitness for Purpose of Analytical Methods
– A Laboratory Guide to Method Validation and Related Topics, (2nd ed. 2014). ISBN 978-91-87461-
59-0. Available from www.eurachem.org
F. V. Barwick (ed.), Planning and Reporting Method Validation Studies – Supplement to Eurachem Guide
on the Fitness for Purpose of Analytical Methods (2019). Available from http://www.eurachem.org"
G. Westgard J. O.: Basic Method Validation, Westgard Quality Corporation
H. Sarewitz S.J.: CAP Accreditation Requirements for Validating Laboratory Tests, 7/9/13
I. Lawrence Jennings, Vivianna M. Van Deerlin, and Margaret L. Gulley (2009) Recommended Principles
and Practices for Validating Clinical Molecular Pathology Tests. Archives of Pathology & Laboratory
Medicine: May 2009, Vol. 133, No. 5, pp. 743-755.