Guiding Principles for Combination Product Reliability

Figure 1: Key reliability considerations.The challenge of the correct tools to use is compounded by the increasing complexity of devices, like reusable, software controlled, electromechanical infusion pumps, and single-use devices, such as mechanical autoinjectors. As regulators and the international standards community increase their focus on patient risks associated with critical therapies, manufacturers must adapt to deliver reliable products through early implementation of reliability engineering techniques.

Reliability Engineering

Effective reliability engineering reduces the risk of product failures and other associated negative outcomes, such as harm to patients, recalls, enhanced regulatory oversight, cost, and damage to brand reputation. There are many examples of medical device recalls that may have been prevented through application of a formal reliability program. A review of recalls for infusion pumps identifies many reliability-related issues, including switch malfunctions, unresponsive user interface, and pump housing cracks allowing fluid ingress.1  Reliability engineering is a set of tools to help manage these risks and this article discusses processes to plan and implement the use of these tools.

However, it is important to distinguish between reliability as an objective and reliability engineering as discipline. Reliability is an attribute of a product, which applies to both multi-use and single-use devices. Reliability engineering is a system of applying engineering principles to the life cycle of a product to achieve the desired reliability objectives.

It is essential that both reliability as an objective and reliability engineering as discipline are both core considerations throughout a combination product’s life cycle. The level of effort should be commensurate with the complexity and risk of the product. Reliability growth has typically been used in a system level to monitor the product improvement and maturity—from design and development to commercialization and continuous improvement management.

The reliability outcome and deliverables can be quantified in terms of patient complaint rates and product failure rates. Reliability values can also provide estimated predictions of patient complaints before product launch based on the clinical data of patient uses of combination products for therapy and outline product failure risks after launch based on failure mode and effects analysis (FMEA), engineering confirmation testing, and the verifications and validation data from design and development.

Background

A core principle of developing a reliable product is that reliability engineering needs to occur at a system level where a fundamental understanding of the design must be established based on first principles. Every decision—from the number of parts to which software version updates, technologies, or testing strategies are used—needs to incorporate reliability engineers into the decision-making process.

Reliability starts with user needs. It then gets translated into product requirements and incorporated into risk management. It is a critical component of design development activities and a cornerstone of testing. Further, it influences manufacturing processes, directs on-market activities, and is critical for patient benefit.

Reliability engagement for combination products begins at product design and development, continues through the commercialization of product launch, and involves continuous improvement of patients’ feedback and field observations. Reliability is more than testing or a demonstration. It is an application of scientific principles to a set of decisions that impacts patients, product performance, and business success.

Definition

Reliability of a manufactured product is defined as the probability that it will perform satisfactorily for a specific period, as intended, without failure under specific conditions. It is typically specified as a R(t) = x%, where t = time and x% = the probability of performing without failure throughout expected manufacturing, environmental, storage, and use condition variability.2

• 1US Food and Drug Administration. “Medical Device Recalls.” Accessed 29 April 2024. www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfRES/res.cfm
• 2U.S. Department of Defense. “Reliability Modeling and Prediction.” Electronic Reliability Design Handbook. Washington, DC, 1998. https://klabs.org/DEI/References/design_guidelines/content/mil_specs/mil_hdbk_338b_contents.pdf

Recommended Reliability Approach

We recommend that reliability be implemented within the existing design control process, as the chance of successfully integrating will be improved because the organization has already aligned around that mode of design and development. Key reliability considerations are shown in Figure 1.

Design Inputs

Reliability starts at the early stage of development as user needs, stakeholder needs, product requirements, and planning documents are established. For the reliability assessment, the design input process can be separated into three parts: a) define context of use, b) describe reliability requirements, and c) document the plan.

Define context of use

Development of the requirements and plan occurs with the context of use and should consider a broad range of issues, including the intended use of the combination product, patients, users, and risk to health if the device fails to function. Risk analysis should consider the intended use and the availability of other medical interventions when the device fails to function.

Based on these factors, the level of reliability needed will vary. For example, because alternative medical interventions may not be provided to the patient quickly enough to prevent harm when the delivery device fails, the reliability for an emergency use product will need to be high.

An example of an emergency use product is an autoinjector that contains epinephrine, a medication that can save a patient’s life by mitigating the risk and decreasing a body’s allergic reaction (relaxing the muscles in the airways to make breathing easier). If the injector device fails, the patient may not be able to access medical care before severe harm or death occurs.3

Depending on the drug product and therapeutic requirements, other therapies may have reliability requirements less strict than this. For example, although insulin is a lifesaving medication, the diabetic patient is more likely to be able to obtain other medical care to mitigate harm should their device fail to deliver the insulin.

Other factors to consider for reliability requirements include the needs of healthcare providers, payors, regulators, and the manufacturer. Details of the use profile can be compiled into a use profile table (an example is shown in Figure 2).

Describe reliability requirements

It is important to correctly and completely write the reliability requirements.4  The requirements should include the reliability target, definition of failure, shelf life (the duration of time up until the product is used or the expiry date is reached, whichever comes first), use life (the duration of time the device is used for drug delivery), and environments of use. Overall, it is critically important to avoid ambiguity and to write the specification in a verifiable manner.

Complex drug delivery systems should consider partitioning reliability requirements across subsystems (i.e., mechanical, electrical, and software) because the product requirements are partitioned into subsystem requirements. A component of reliability expectations is related to the use profile requirements. The use profile will define environmental impacts (patient uses and shelf life of storage), activation cycles, shipping conditions, and anticipated worst case conditions (i.e., drop, shock, etc.).

• 3US Food and Drug Administration. “Technical Considerations for Demonstrating Reliability of Emergency-Use Injectors Submitted Under a BLA, NDA or ANDA: Guidance for Industry and Food and Drug Administration Staff.” April 2020. www.fda.gov/media/137158/download
• 4Roush M., and W. Webb. Applied Reliability Engineering Volume II. College Park, MD: The Center for Reliability Engineering, University of Maryland, 2006.

These conditions will provide an outline for reliability evaluations throughout development. The reliability requirements and reliability planning (established at the early stage of development) become the foundation for all subsequent work and should be discussed during the design input review.

Document the plan

Reliability should be included in planning documents. It can be a part of the design and development plan, or there can be a separate reliability plan for complex drug delivery systems. The reliability plan should outline activities from the feasibility phase through development and up to the commercialization phase.

It is expected that the plan will be updated to accommodate the increased understanding of the design and performance. Each device employing different architectures, technologies, and maturity of design solutions has a substantial impact on the time required to achieve suitable on-market reliability.

Design Outputs

Once the design inputs have been specified, design output activities can be initiated: a) establish a reliability budget, b) implement reliability assessment tools, and c) develop a reliability model.

Establish a reliability budget

The reliability requirement is used to evaluate product concepts and design decisions. The reliability budget is also established during this development process. The reliability budget is an apportionment of the overall reliability requirement across the various subsystems and interfaces.

For example, if a device that consists of four subsystems has a reliability target of 95% and the reliability target for each subsystem is equally divided across the subsystems, then the target reliability for those subsystems is (.95)¼, or 98.7%. For simplicity, this apportionment example is divided equally into each subsystem. However, the budget for each subsystem can be adjusted as needed to achieve the overall system-level reliability target.

Keep in mind that decisions about how to apportion reliability to subsystems should always consider the impact on safety and effectiveness (e.g., a safety-critical subsystem may have its own independent reliability requirements irrespective of what apportionment may permit). For example, a post-injection needle safety protection feature may require 99% reliability per a risk-based or a regulatory requirement.

Therefore, although it may be mathematically possible to allocate less reliability to this function, other external requirements may impact reliability apportionment decision-making. The overall reliability of the device is determined by the product of all subsystem and interface reliability components (see equation 1). Importantly, to achieve the reliability requirement for the device, the composite reliability of the subsystems and interfaces must be met.

Equation (1)

Using functional diagrams to depict major subsystems and interfaces is often useful to gain alignment on the subsystems and interfaces that warrant attention.5  A reliability block diagram (RBD) is a diagrammatic method for showing how component reliability contributes to the success or failure of a redundant. RBD is also known as a dependence diagram. RBDs are a way of representing a system, including its subsystems and components, as a series of blocks. This is done in such a way that equipment failure rates, operating philosophies, and maintenance strategies can be quantitatively assessed in terms of the impact they are expected to have on a system. 6

An RBD is drawn as a series of blocks connected in parallel or a series configuration. Parallel blocks indicate redundant subsystems or components that contribute to a lower failure rate (see Figure 3). Each block represents a component of the system with a failure rate. RBDs will indicate the type of redundancy in the parallel path.1  For example, a group of parallel blocks could require two out of three components to succeed for the system to succeed. By contrast, any failure along a series path causes the entire series path to fail.

An RBD may be drawn using switches in place of blocks, where a closed switch represents a working component and an open switch represents a failed component. If a path may be found through the network of switches from beginning to end, the system still works. An RBD may be converted to a success tree or a fault tree depending on how the RBD is defined. A success tree may then be converted to a fault tree or vice versa.

As the product is developed, the budget can be reallocated between subsystems as needed. However, as part of the product risk management process, efforts should be made to resolve any design weaknesses that may be contributing to difficulty meeting the pre-established budget.

• 5Modarres, M., M. Kaminskiy, and V. Krivtsov. Reliability Engineering and Risk Analysis: A Practical Guide. New York, NY: Marcel Decker, Inc., 2016.
• 6Knight, K. “Why Do You Need a Reliability Block Diagram?” Arm Reliability. 11 August 2020. www.armsreliability.com/page/resources/blog/why-do-you-need-a-reliability-block-diagram
• 1 

Implement reliability assessment tools

Once the initial reliability budget has been established, engineering analysis and testing can be used to create reliability estimates. These can then be compared to the initial reliability targets. These reliability estimates are expected to mature as the designs and analyses are refined and the amount of test data increases. During this time of engineering analysis and testing, it is important to establish an understanding of the variables that impact reliability and the robustness of the design. A first principle approach is strongly suggested to understand fundamental behaviors of the design and to establish an understanding of design margin.

An assessment can be accomplished using Monte Carlo analysis and/or direct test methods. Reliability at the component level can be initiated using available support tools.7 ^, 8  A first principle approach can be used to analyze components, subsystems, and systems for reliability assessment. It allows a total system approach for analyzing individual systems and components. The design team can leverage the initial reliability estimates from the prior knowledge of each similar component, even before the design team has any physical samples for lab testing and reliability assessment.

A first principle approach also offers a prescreen tool for component-level analysis and selection. Reliability testing should be done at the subsystem level during the design and development phases and ultimately at the system level for finished product analysis and assessment. Reliability growth is a useful indicator to show the reliability improvements at each version of design and development for any device-drug delivery systems. Accelerated test methods are often used to detect failures early in development as compared to real-time product assessment from a shelf-life-based analysis.

Other valuable tools should be used to probe the reliability of the design, and often these are used as part of the product risk management process. FMEA is a structured way to identify and address potential problems or failures and their resulting effects on the system or process before an adverse event occurs. In comparison, root cause analysis is a structured way to address problems after they occur.

Reliability tools to be considered for application during development include fault tree analysis (FTA), design FMEA (DFMEA), process FMEA (PFMEA), and use-related risk analysis (URRA). These tools can be used to understand failure modes and the causes of system-level failures, which can help prioritize design analysis and testing.

Fault tree analysis (FTA)

FTA is a type of failure analysis in which an undesired state of a system is examined. This analysis method is mainly used in reliability engineering to understand how systems can fail, to identify the best ways for minimized risk, and to determine a particular system-level (functional) failure. The FTA shows causes that can lead to the failure of a system using logic gates.

An FTA is especially helpful because it assesses the causes of failure using a top-down approach considering direct causes (shown with “OR” gates) and combined causes (shown with “AND” gates). An example of an FTA is shown in Figure 4. Estimating the individual probabilities of each failure mode should improve the accuracy of the budgeted reliability.

FTA has been widely used in pharmaceutical, aerospace, nuclear power, chemical and petrochemical, and other high-hazard industries. FTA is also used in software engineering for debugging purposes and is closely related to the cause-elimination technique used for the combination products of electromechanical drug-device systems.

DFMEAs and PFMEAs

DFMEAs are a bottom-up approach to evaluate device failures. The DFMEA provides an in-depth view of component and interface failures and an opportunity to incorporate mitigations early in development. This in-depth view of component performance may identify key drivers of reliability and provide justification for greater understanding and control. Similarly, PFMEAs are a bottom-up approach to evaluate process failures and provide an in-depth view of process performance that may influence reliability and provide justification for greater understanding and control.

URRAs

URRAs are another bottom-up approach to evaluate product-use-induced failures and provide an in-depth view into how the combination products are being used by patients. The URRA shows the impacts of therapeutical risk analysis on a device-drug delivery system that may influence reliability and provide justification for greater understanding and control of combination products at design-use perspectives by the targeted patients.

Process capability studies

Manufacturing processes should be assessed using process capability studies. The allowable specification ranges should be established by the preceding engineering analysis and testing. Those variables most impactful to reliability and essential performance should be flagged and considered as a part of the control strategy. The combination of design and manufacturing characterization will help define the expected robustness of the product and its ability to meet reliability requirements.

HALT and HASS

Highly accelerated life testing (HALT) and highly accelerated stress-screen (HASS) are means to understand and improve the ruggedness of the design and to create custom production screening prior to release for verification builds. HALT involves subjecting the device to fluctuating temperature and vibration loads to determine failure modes. Resolving HALT failures and extending HALT times and/or loads is an indicator of reliability growth. HASS is testing that can be incorporated in the manufacturing process to identify manufacturing defects that could lead to reliability failures.

Budget and risk management

The reliability budget should be assessed periodically during development to reflect the current level of confidence in meeting the reliability requirement. The confidence of meeting the reliability requirement is expected to increase throughout development.

There is a direct link between reliability and risk management. The reliability information is input to the device system risk assessment. Failure effects become a part of the sequence of events and the probability of failures becomes a part of the probability of harm. Specific actions to reduce the probability of failures become a part of risk mitigation.

Develop a reliability model

The culmination of these efforts is a reliability model that is used to demonstrate that the reliability requirements are met. These modeling approaches are of particular importance for single-use devices that cannot be directly tested prior to release. Although there are different modeling tools that can be used (e.g., RBDs), we recommend FTA due to its level of detail and flexibility to adapt the design, manufacturing, and control processes where necessary to achieve the reliability specifications.

Although we are not intending to delve into how FTA is conducted, we recommend the following recommendations when developing a fault tree for a drug delivery device:

• Tie the top-level fault to the definition of failure for the product.
• Tie second-level faults to drug delivery functions that are considered important for safety and effectiveness.
• Use the FMEA and other reliability assessment tools to assure that the FTA is comprehensive.
• Assure that redundancy in the FTA is handled appropriately during analysis of the FTA.

The reliability model is not intended to be a static activity and should be updated with current information as manufacturing is scaled up and continuously improved. In this way, reliability maturity can be assessed and documented.

Design Verification

Reliability testing preparation begins with preconditioning of the test items, which should include environmental, storage, and use condition extremes and be done at the component and assembly level. The parameters should be consistent with the reliability specifications. Attention and careful justification must be given to the types of age acceleration, especially when using temperature

and the Arrhenius equation. The age acceleration study would

shorten the testing cycle; however, the materials attributes, particularly the glass transition temperature (Tg) of a specific polymeric base material, need to be considered for its accelerating study designs.

Once the polymer material has its Tg around the accelerating aging study temperature, it would introduce a ductile-brittle transition and change its material characterization and properties. The Arrhenius equation, accordingly, may not be suitable for the prediction of real-life reliability from the accelerating study results. In contrast, if a polymer material has its Tg far away from the accelerating aging study temperature, the assessment of acceleration is achieving the desired period.9

The building of verification units provides an additional opportunity to assess the process capability (Cp) and process capability index (Cpk) values of those variables that impact reliability.

Reliability tests should assess all essential performance of the design. For multi-use drug delivery systems, the reliability at end of useful life needs to be demonstrated. In addition, multi-use test samples should be run to failure, as this will establish end of life to estimate the design margin. The reliability of a durable electromechanical pump can be targeted, for example, for reliability at a 95%, 90%, and 85% reliability level at 95% confidence at one, two, and three years of reliability performance, respectively.

In addition to demonstrating that the design meets the reliability requirements, the verification results can be used to estimate the reliability performance margin and estimate field performance.

Design Validation

The validation of reliability comprises several approaches and is aimed at assuring that the established reliability specifications are meeting user needs. This is conducted through analysis of available information that includes literature, risk analyses, standards, regulatory requirements, and prospective studies (e.g., human factors, real-life handling studies, and clinical studies).

Human factors summative studies provide an opportunity to evaluate how the patient responds to the physical device, instructions for use, product labeling, product displays, and alarms. Real-life handling studies are meant to simulate actual use conditions to assess performance. Clinical studies can also be used to further validate the design. Although these studies generally do not include reliability endpoints, the information obtained should be used as part of the assessment (e.g., complaints, device failures, and use error). For example, if users interact with the device in an unexpected manner leading to a deficit of functional performance, the reliability analysis may need to be revised to account for or address the experience.

Design Transfer

Design specifications, design margin analysis, manufacturing instructions, control strategy, risk management files, manufacturing capability assessments, and estimated field performance are the basis of transferring reliability to from research and development to operations. Operations activities related to reliability include monitoring manufacturing variability, HASS results (if applied), and product on-market field performance (to compare against expected reliability performance and to determine life cycle management activities to meet and exceed reliability expectations).

On Market

Product launch after regulatory approval is a key milestone for patients to have a new medicine for the improved therapy. Healthcare providers and patients provide feedback to medical device and pharmaceutical companies. This includes their product uses in terms of the field experiences and observations of drug products, medical device for the drug delivery systems, and/or combination products. Reliability monitoring is continued from the scale-up to the technical transfer of mass production at a commercial manufacturing line.

The data from the field feedback and commercial-scale manufacturing can be used to assess both patient complaints and product failure rates. The reliability study at the on-market stage is not only to provide reliability growth assessment, but also to offer continuous improvement opportunities. The reliability outcomes of the on-market stage are the valuable knowledge and foundation that can be used to identify the critical subjects and processes for the manufacturing yield improvements and patient complaint mitigation.

Conclusion

A structured approach is essential to ensure a reliable combination product. Reliability begins with understanding the needs and establishing requirements. Reliability then gets embedded into planning and it should be characterized analytically and experimentally to fully understand the physics and the robustness of the design and manufacturing processes. Reliability to meet essential performance is demonstrated with preconditioned parts and assemblies.

Margin is shown by testing to end of life for multiuse drug delivery systems. Control strategies include those parameters necessary to meet reliability requirements and the risk management file contains the failure modes and probabilities of failures that are assessed throughout the product life cycle.

Reliability is more than testing or a demonstration; it is an approach that impacts patients, product performance, and business success. The goals of reliability deliverables and reliability engineering are to consistently assure trustworthy medical devices and robust combination products via drug-device delivery systems for targeted therapies.

• 7GE Vernova. “Reliability Analysis Software.” www.ge.com/digital/applications/asset-performance-management/apm-reliability
• 8Relyence Corporation. “Relyence Software.” https://relyence.com
• 9Hemmerich, K. J. “General Aging Theory and Simplified Protocol for Accelerated Aging of Medical Devices.” Medical Device and Diagnostic Industry. 1 July 1998. www.mddionline.com/design-engineering/general-aging-theory-and-simplified-protocol-accelerated-aging-medical-devices