Which Variables Are Critical to Field Reliability? Design of Experiments (DOE) Can Help Optimize Your Design

Engineers are often faced with the problem of optimizing a process that includes a number of input variables. It could be an analysis of a manufacturing process, or even an analysis of a business process. They’re often challenged to find ways to reduce warranty costs that result from field failures. For example, how can we better understand the field effects on an actuator assembly used in a garden tractor as a means of reducing warranty returns. One effective means is to design a special test to determine which factors are important to the life of the actuators. Design of Experiments (DOE) provides a structured means to see which factors are important to component life.

Determining Factors Important to Actuator Field Reliability

We suspected that there were four possible significant factors affecting the field reliability of the actuator that needed to be investigated. We used the Design of Experiments tool included in the Quanterion Automated Reliability Toolkit (QuART) to easily design a test and analyze the results for the field reliability investigation. The tool uses a 24-1=8 Fractional Factorial Design such that only eight test runs are required to investigate two levels for each factor, instead of the 16 that would be required to analyze all combinations. To do this, interactions among the variables were assumed to be negligible. This example represents an adaptation of actual experience for demonstration purposes. The following were the required steps:
1. Access the DOE Tool: Select the “Production” tab at the left hand side to access the “Design of Experiments” tool. Note that this tool is included under “Production” but it can be used for many other applications such as this one addressing field reliability.
Design of Experiments (DOE) is a statistically-based means of systematically determining the effects of variables on processes of interest. It is highly efficient due to reduced test time and allows many input factors to be studied at one time. Factorial Designs are a form of DoE where one or more trials are carried out with each combination of levels for the input factors while Fractional Factorial Designs are simplified approaches to Factorial Design with fewer trials including the assumption that interactions among variables are negligible. Some DOE definitions include:

  • Factor: a feature of an experiment which can be varied from one trial to another (can be qualitative or quantitative)
  • Levels of a Factor: Discrete values that an experimental factor can take on
  • Randomization: conducting an experiment in a random manner to avoid bias due to systematic effects
  • Response: result of process based on levels of factors being investigated
  • Main Effect: average effect of factor over all levels of the other factors
  • Interaction Effect: when effect of one factor is different at different levels of the other factors

For more information on DOE, the following are recommended:

  • Statistics for Experiments, Box and Hunter, Wiley
  • The Design and Analysis of Industrial Experiments, Davies, Hafner
  • Introduction to Quality Engineering, Taguchi, American


Figure 1: QuART Design Of Experiments Tab

2. Define the Problem and Decide Which Variables to Study: Our problem was to determine which factors are important to the field reliability of the actuator used on the garden tractor. From past experience and engineering judgment, four factors were selected for evaluation and the information was entered in the “Problem Definition” tab. Each factor was tested at two levels that were possible input conditions affecting reliability. The difference in geographic location was simulated by a difference in steady state operating temperature.


Figure 2: QuART Design Of Experiments Problem Definition Tab

3. Run the Tests and Input the Reliability Data (Time or Cycles, etc.): The reliability data was entered into the “Run Tests” tab of the QuART DOE tool in the rows appropriate for the specific test conditions.


Figure 3: QuART Design Of Experiments Test Results Tab

4. Examine the Results at the Desired Confidence: Once the test data was entered, the “Relationship” tab was used to view the results. Assuming that a very high confidence level (say 99%) is desired in the results, the “Acceptable Risk” window was used to choose 1% (note that risk equals 1-confidence). The right hand column indicates that only the Actuator Alignment was significant to reliability at this level of confidence.


Figure 4: QuART Design Of Experiments Relationship Tab For 1.00%

If the engineer is willing to accept a 10% risk, the risk value can be changed to see the effects. For this test at the higher risk level, the Actuator Case Material also became a significant factor.


Figure 5: QuART Design Of Experiments Relationship Tab For 10.00%

5. Interpretation of Results: For this application, because the factors had positive coefficients in the resulting relationship, it was clear that maintaining Tightly Controlled Alignment and use of High Strength Plastic were means that should be used to improve field reliability. The other factors addressed by the DOE appeared to have little significance.