2/12/20

Product Assurance
Space System Design, MAE 342, Princeton University
Robert Stengel
• Assembly, Integration,
and Verification
• Dependability
• Reliability
• Task Planning
• Quality Assurance

Copyright 2016 by Robert Stengel. All rights reserved. For educational use only.
http://www.princeton.edu/~stengel/MAE342.html 1

Failure Analysis of Cygnus CRS Orb-3

Orbital Sciences Antares 130
• Possible causes
– Manufacturing defect in
turbopump Aerojet
Rocketdyne AJ-130
motor
• Refurbished Energomash
NK-33 motor from
stockpile
• Built in 1970s
– Design flaw in hydraulic
balance assembly and
thrust bearings
2

1
 2/12/20

Assemble, Integrate, and Verify

• Assemble • Integrate
– Build spacecraft – Make it function
• Verify
– Demonstrate compliance with goals
• Qualification of design
• Acceptance of hardware
– Methods – System Level
• Test • Spacecraft
• Analysis • Module or sub-system
• Inspection • Unit
• Design Review • Equipment or component

Fortescue, Ch. 17 3

Manage Risk

2
 2/12/20

Classify Risk

5
http://www.riskbusinessamericas.com/Public.IndustryRiskProfiles.aspx

Assess Risk

6
http://www.riskbusinessamericas.com/Public.IndustryRiskProfiles.aspx

3
 2/12/20

Spacecraft Product Assurance

• Origins
– Industrial Revolution
– Formal quality assurance during WWII
• Evolution
– Standards and certification methods borrowed
from USAF, ABMA
– See Lecture 24 Course Materials on Blackboard
• Special problems
– Extremes of operating conditions
– Length of unattended operation
– Inaccessibility for maintenance
Fortescue, Ch. 19 7

Saturn V Second Stage

Integral serial tanks, with
common bulkhead

4
 2/12/20

Principles and Definitions for

Product Assurance
• Quality
• Basis for quality assessment
• Proof of quality

Fortescue, Ch. 19 10

10

5
 2/12/20

Objectives and Project Phases

Fortescue, Ch. 19 11

11

Overlapping Issues

12
Fortescue, Ch. 19

12

6
 2/12/20

Task Planning

Situation awareness
Decomposition and identification of communities
Development of strategy and tactics
Phase
Process Outcome
Objective Tactical Situation Situation
(short-term) Assessment Awareness
Strategic Comprehension Understanding
(long-term)

13

13

Boyd’s “OODA Loop”

for Combat Operations
Derived from air-combat
maneuvering strategy
General application to learning
processes other than military

14

14

7
 2/12/20

Elements of
Situation Awareness

• Perception
• Comprehension
• Projection
Endsley, 1995 15

15

Important Dichotomies
in Planning
Strength, Weakness, Opportunity, and
Threat (SWOT) Analysis “Knok-Knoks” and “Unk-
Unks”

16

16

8
 2/12/20

Program Management:
Gantt Chart
Project schedule
Task breakdown and dependency
Start, interim, and finish elements
Time elapsed, time to go

17

17

Program Evaluation and Review

Technique (PERT) Chart
Milestones
Path descriptors
Activities, precursors, and successors
Timing and coordination
Identification of critical path
Optimization and constraint

18

18

9
 2/12/20

-ilities
• Dependability
– Availability
– Maintainability
– Security
• Reliability
– Qualitative
– Quantitative
– Design or predicted
– Operational
19

19

Parts Procurement
• Vendors’ track record
• Standardization
• Procurement systems
– Organization
– Documentation
• Substitution of less reliable
equivalents
• Out-of-date/specification parts
20

20

10
 2/12/20

Materials and Processes

Fortescue, Ch. 19 21

21

Materials to Avoid

Fortescue, Ch. 19 22

22

11
 2/12/20

Material Problems in Orbit

Fortescue, Ch. 19 23

23

Materials Problems within Parts

Fortescue, Ch. 19
24

24

12
 2/12/20

Product Assurance in
Manufacturing
• Controls and Records
• Training and certification
• Traceability
• Measurement and calibration
• Non-conformance control
• Alerts, handling, … margins
• Audits
25
Fortescue, Ch. 19

25

Non-Conformance Control

Fortescue, Ch. 19 26

26

13
 2/12/20

Technology Readiness Levels

27
Fortescue, Ch. 19

27

Product Assurance and Safety

in Operations

28

28

14
 2/12/20

Reliability of a Component
If failure rate is constant,

R = e− λt
where failure rate is estimated as

⎧⎪ 1 MTBF (repairable system)

λ=⎨
⎪⎩ 1 MTTF (non-repairable system)
MTBF : Mean time between failures
MTTF : Mean time to failure
Also see Lecture 17 slides for reliability assessment 29

29

Failure Rate, λ

Expected number of failures per unit

time
30

30

15
 2/12/20

Reliability Enhancement
• Use of redundancy
• Design diversity
• Limitation of failure effects
• De-rating of parts
• Radiation screening
• Handling/assembly controls
• Inspection/testing

Fortescue, Ch. 19 31

31

Reliability Analysis Techniques

• Failure state probabilities
• Worst-case analysis
– https://en.wikipedia.org/wiki/Worst-case_circuit_analysis
• Failure modes and effects analysis
– https://en.wikipedia.org/wiki/Failure_mode_and_effects_analysis
• Fault tree analysis
– https://en.wikipedia.org/wiki/Fault_tree_analysis
• Contingency analysis
– What to do when failure occurs

Fortescue, Ch. 19 32

32

16
 2/12/20

Probability Distributions

33

33

Relative Frequency of Discrete,

Mutually Exclusive Events
ni
Pr ( xi ) = in [0,1]; i = 1 to I
N
• N = total number of events
• ni = number of events with value xi
• I = number of different values
• xi = ordered set of hypotheses or values

I
1 I
∑ Pr ( xi ) = N ∑ ni = 1
i=1 i=1

34

34

17
 2/12/20

Cumulative Probability, Pr(x ≥/≤ a),

and Discrete Measurements of a
Continuous Variable
1
0.9
0.8
0.7
0.6
Pr(x)
0.5
Cum Pr(x) ≥ a
0.4 Cum Pr(x) ≤ a
0.3
0.2
0.1
0
1 2 3 4 5

Suppose x represents a continuum of colors

xi is the center of a band in x
Pr ( xi ± Δx / 2 ) = ni / N
I

∑ Pr ( x i ± Δx / 2 ) = 1
i =1 35

35

Probability Density Function, pr(x)

Cumulative Distribution Function,
Pr(x <X)
Probability density function

Pr ( xi ± Δx / 2 )
pr ( xi ) =
Δx
I I ∞
∑ Pr ( xi ± Δx / 2 ) = ∑ pr ( xi ) Δx ⎯Δx→0
⎯⎯ → ∫ pr ( x ) dx = 1
I→∞
−∞
i=1 i=1

Cumulative distribution function

Pr ( x < X ) = pr ( x ) dx
X
∫
−∞

36

36

18
 2/12/20

Probability Density Function, pr(x)

Cumulative Distribution Function,
Pr(x <X)

Pr ( x < X ) = pr ( x ) dx
X
∫
−∞

37

37

Properties of
Random Variables
• Mode
– Value of x for which pr(x) is maximum
• Median
– Value of x corresponding to 50th percentile
– Pr(x < median) = Pr(x ≥ median) = 0.5
• Mean
– Value of x corresponding to statistical average
• First moment of x = Expected value of x
“Force”
∞
x = E(x) = ∫ x pr ( x ) dx
−∞
“Moment arm” 38

38

19
 2/12/20

Expected Values
• Mean Value is the
first moment of x
∞
x = E(x) = ∫ x pr ( x ) dx
−∞

• Second central moment of x = Variance

– Variance from the mean value rather than from zero
– Smaller value indicates less uncertainty in the value
of x
∞
σ x 2 = E ⎡⎣( x − x ) ⎤⎦ = ∫ ( x − x ) pr ( x ) dx
2 2
−∞

39

39

Mean Value and Variance of a

Uniform Distribution
⎧ 0
⎪ x < xmin
⎪ 1
pr(x) = ⎨ ; xmin < x < xmax
⎪ max − xmin
x
x > xmax
⎪⎩ 0

Mean
x 1
( xmax + xmin )
xmax
x=∫ dx =
xmin ( xmax − xmin ) 2

Variance
If xmin = −xmax ! a
a
1 a 2 x3 a2
σx = ∫ x dx = =
2

2a − a 6a − a 3 40

40

20
 2/12/20

Gaussian (Normal) Random

Distribution
Unbounded, symmetric distribution
Defined entirely by its mean and standard
deviation
( x − x )2
−
1 2 σ x2
pr(x) = e
2π σ x

Mean value; from symmetry

∞
E(x) = ∫ x pr ( x ) dx = x
−∞

Variance
∞
E ⎡⎣( x − x ) ⎤⎦ = ∫ (x − x ) pr ( x ) dx = σ x2
2 2
−∞

Units of x and σx are the same 41

41

Probability of Being Close to the

Mean
(Gaussian Distribution)

• Probability of being within ±1σ x

Pr ⎡⎣ x < ( x + σ x ) ⎤⎦ − Pr ⎡⎣ x < ( x − σ x ) ⎤⎦ ≈ 68%

• Probability of being within ±2σ x

Pr ⎡⎣ x < ( x + 2σ x ) ⎤⎦ − Pr ⎡⎣ x < ( x − 2σ x ) ⎤⎦ ≈ 95%

• Probability of being within ±3σ x

Pr ⎡⎣ x < ( x + 3σ x ) ⎤⎦ − Pr ⎡⎣ x < ( x − 3σ x ) ⎤⎦ ≈ 99%

42

42

21
 2/12/20

Experimental Determination of
Mean and Variance
Sample mean for N data points, x1, x2, ..., xN
N

∑x i
x= i =1 Histogram
N
Sample variance for same
data set
N

∑( x − x)
2
i
σ x2 = i =1

( N − 1)
Divisor is (N – 1) rather than N to
produce an unbiased estimate 43

43

Log-Normal Distribution
Probability Density Function
• Variation in large
ensembles for which x > 0
• The logarithm of x is
Gaussian
• Replace x by xl in previous
equations

xl ! log x
−
( xl −xl ) Cumulative Probability Function
1 2σ l 2
pr(xl ) = e
σ l 2π

1⎛ x − xl ⎞
Pr(xl ) = ⎜ 1+ erf l
2⎝ σ l 2 ⎟⎠ 44

44

22
 2/12/20

Weibull Distribution
• Variation in life Probability Density Function
characteristics of parts or
components
• Variation in large
ensembles for which x > 0

⎧⎪ ⎡ b ⎤ ⎡ x − x ⎤b−1 ⎫⎪ − ((θ −xo ))b

b
x−x

pr(x) = ⎨ ⎢ ⎥ ⎢θ − x ⎥ ⎬ e
o o

θ −
⎩⎪ ⎣ o ⎦⎣ o⎦
x
⎭⎪
Cumulative Probability Function
−
( x−xo )b
(θ −xo )b
Pr(x) = 1− e

xo : expected minimum value

b : shape or slope parameter (k in figure)
θ : characteristic life or scale parameter (λ in figure)
45

45

Exponential Distribution
Probability Density Function
• Special case of Weibull
distribution, with b = 1, xo =
0, and x = t
• Time to failure of systems or
parts
• Modeling of independent
events that occur at a
constant average rate
Cumulative Probability Function
1 −t
pr(t) = e θ
θ
− tθ
Pr(t) = 1− e
λ = 1θ : failure rate
46

46

23
 2/12/20

Poisson Distribution
• Occurrence of isolated, Probability Mass Function
independent events whose
average rate is known
– Number of events can be
observed
– Number of non-events cannot be
observed
• Examples:
– Number of machine breakdowns
in a plant
– Number of errors in a drawing Cumulative Probability Function

e− λ y ri
pr(r = ri ) =
ri !

λ : average number of occurrences

47

47

Binomial Distribution
• The probability of r Probability Mass Function
successful outcomes in n
trials
• Examples: inspection of
parts, probability that a
system will operate correctly
⎛ n⎞ ⎛ n⎞
pr(r) = ⎜ ⎟ p rq n−r = ⎜ ⎟ p r (1− p)
n−r

⎝ r⎠ ⎝ r⎠
where
Cumulative Probability Function
⎛ n⎞ n!
⎜ ⎟=
⎝ ⎠
r r! ( − r)!
n
n = number of trials
p = probability of success
q = probability of failure 48

48

24
 2/12/20

Confidence Level
• The probability that a probability
estimate is correct, e.g.,
“The likelihood of failure is 90%,
with a confidence level of 95%”

49

49

Trials Required to Estimate Probability

Depend on Confidence Interval
1e+09

1e+07
Number of Evaluations

Interval
Width
1e+05 2%
5%
10%
1e+03 20%

100%
1e+01
1e-06 1e-05 1e-04 0.001
1 0.01 0.1
0.5
Probability or (1 - Probability)
Binomial Distribution
Required number of trials depends on outcome
probability and desired confidence interval 50

50

25
 2/12/20

How Will You Estimate the

Likelihood of Success for Project
2020 UA?

51

51

MAE 342, Space System Design

52

52

26
 2/12/20

53

53

27