NESC Develops Method for Estimating Risk When Reducing NDE 

Download PDF: NESC develops a method for estimating risks when reducing EMI

Performing a non-destructive evaluation (NDE) can have both cost and schedule impacts, leading some to wonder whether it might be possible to delimit (i.e., reduce or eliminate) NDE inspections on certain spaceflight hardware. However, this approach would be contrary to NASA Technical Standard NASA-STD-5019A, which describes spaceflight system requirements for establishing a fracture control plan, one that relies on the design, analysis, testing, NDE, and tracking of fracture-critical parts to verify damage tolerance and mitigate catastrophic failure.

Under the 5019A framework, damage below the NDE detection capability is assumed to exist, but by analysis or testing it must be demonstrated that the evaluated part survives the required service life. In practice, the role of the NDE is to eliminate defects that might otherwise result in failure. However, in some cases it could be useful to decouple NDT from the damage tolerance verification process while still ensuring the required level of safety.

NESC conducted an assessment to help answer the question of whether it was possible to find justification for achieving an equivalent risk posture without using the traditional 5019A approach to damage tolerance. The goal was to develop a probabilistic analysis method that would allow NASA programs and projects to estimate the risks associated with defining NDE requirements for single-fabricated materials. This effort included using historical data to demonstrate the method, conducting sensitivity studies, and identifying the minimum supporting data that would be required to approve a de-scoping request.

Deciphering Damage Tolerance EMI

Damage tolerance is generally treated as deterministic: an NDE detection threshold is established as a fixed defect size with an associated binary outcome (defect exists/does not exist), and failure is based on conservative analysis or testing with a binary outcome (pass/fail). However, damage tolerance is based on the following probabilities:

• PENNSYLVANIA): Probability that a defect of a given size exists,

• P(D0│A): Probability that this flaw is missed by NDE, and

• P(F│D0,A): Probability that a flaw will cause a failure given that it exists and was missed by the EMI.

These are combined into the common failure probability: P(F,D0,A) = P(F│D0,A)P(D0│A)P(A)

Damage tolerance is based on the idea that analysis and testing suggest a near-zero probability of failure below a critical initial defect size. (ACIFS) illustrated by the green (lower) arrow in Figure 1, and END results in a near-zero probability of missing a fault above a certain detectability threshold. (ANDE) illustrated by the yellow (upper) arrow in Figure 1. If these two areas overlap, then the part is damage tolerant, with a probability of failure close to zero, regardless of the underlying probability of defects existing, i.e. conservatively assuming that P(a>aCIFS)=1 for any defect size has no impact on the conclusion. However, if EMI is not taken into account, the right arrow in Figure 1 is removed and the risk will increase until it reaches a value proportional to the probability. P(a>aCIFS).

Estimate P(a>aCIFS) may be intractable without expensive, high-resolution methods to characterize the frequency of defect occurrence at a particular size for a given part. Alternatively, it may be possible to estimate Pennsylvania> ANDE)the probability that a detectable defect exists. Assuming that a part of interest is found to be damage tolerant prior to any NDE analysis (i.e., meeting NASA-STD-5019A), it can be assumed that (1) historical inspection data is available, and (2) ANDE > ACIFSdue to the required overlap in Figure 1. As such, it has been proposed that the frequency of historical discoveries can be used to estimate an upper 95% confidence limit on Pennsylvania> ANDE) and therefore an estimate of the risk associated with descoping.

To demonstrate the risk assessment framework, NESC had access to a historical NDT database comprising 33,630 bolt hole inspections over a 3 year period. In total, six crack-like features were discovered by EMI. Accounting for uncertainty due to sample size resulted in an upper confidence limit of 95%. Pennsylvania> ANDE) = 0.04% for each hole. In the proposed method, it is conservatively assumed that if a defect exceeding the CIFS exists, it will result in structural failure. Although conservative, this assumption was necessary due to the limitations of the database in that it lacked the size of detected defects. Based on this hypothesis, Pennsylvania> ANDE) = 0.0004 gives a structural reliability of approximately 0.9996 (expressed as 3.4 “nine”).

The results are illustrated graphically in Figure 2. In this case study, increasing the number of inspections in the dataset to 100,000 (i.e., multiplying by a factor of 3) increases the number slightly from nine to 3.5. At the observed NDE rejection rate, a reliability of 4 9 is not achievable, even with infinite samples and zero uncertainty. The rejection rates and sample sizes in this case study are expected to be within the order of magnitude of what would be observed and available in practice. Since 2 nines or less would equate to a significant increase over baseline risk for NASA’s human spaceflight programs, a minimum sample size of 5,000 inspections is required for an EMI rejection rate of 0.04%.

There are necessary assumptions underlying this methodology. First, time-invariant process control is necessary to ensure that probabilities estimated from historical inspections predict future probabilities after deportation. Ensuring consistency during the data collection period is a first step in verifying existing controls, and ongoing monitoring is necessary to verify that the process remains consistent over time. Second, although aggregating data across multiple parties can increase inspection sample size and reduce uncertainty in estimated rejection rates, it requires aggregation justification via qualitative and quantitative similarity assessments. The methodology developed by NESC is intended to be part of a comprehensive fracture control assessment by the NASA Fracture Control Board and the responsible technical authority.

For more information, contact Patrick E. Leser. patrick.e.leser@nasa.gov

Reference: NASA/TM-20250004074