Applying Taguchi Designs to EMMAQUA Weathering Experiments

Henry K. Hardcastle III
 Atlas Weathering Services Group
 July 12, 2000


 This paper investigates the effects of nine variables on weathering of a commercial polyester acrylic melamine automotive coating. The material was exposed to weathering effects using EMMAQUA natural accelerated test device. A Taguchi L16 fractional factorial experimental design was used to optimize experimental trials. The analysis of variance (ANOVA) identified the variables having the biggest effect on gloss loss. The Taguchi approach successfully categorized the effect of each of the variables using only 16 experimental trials and identified the most important variables effecting this weathering process.


 One may consider the recent history of the weathering discipline (the past 100 years or so) as an evolution of weathering experiments. This evolution trends from simpler, single variable experiments toward more stochastic and broader experimental approaches. All the different major types of weathering experiments represent important tools engineers can use for answering different types of questions regarding product development. In 1926 British statistician R.A. Fischer began applying fractional factorial approaches to agricultural experiments. In the 1960's Professor Genichi Taguchi included modified fractional factorial experiment designs for characterizing manufacturing processes in his quality engineering philosophy. Today, we present an application of modified Taguchi approaches to weathering experiments. 

 Atlas Weathering Services Group (AWSG) is currently developing a new model for weathering phenomena. One important element in the new AWSG weathering model is the analogy between manufacturing processes and weathering processes. Visualizing weathering phenomena in the context of a manufacturing process allows researchers to directly apply quality engineering tools to weathering observations. Since the Taguchi approach is such a powerful philosophy in the quality engineering of manufacturing processes, it is only natural for AWSG to apply Taguchi approaches in the development of the AWSG weathering model. 

 Manufacturing process engineers widely use fractional factorial screening experiments to "screen out" the trivial many variables from the "important few" variables (1). A small confirmation experiment (typically two to four additional trials) follows a screening experiment to confirm the results. Once the manufacturing process engineer identifies the important variables, the engineer can focus process improvement efforts in an efficient manner on those variables that the process indicates are important. This analogy described for characterizing and improving manufacturing processes applies perfectly to improving weathering performance. Weathering processes are multi-variable complex processes that are highly material dependent. Weathering investigators can use fractional factorial screening experiments to "screen out" the trivial many variables from the "important few" variables. Once a screening experiment indicates the most important variables for a particular material, a small confirmation experiment always confirms the results. Once the weathering investigator identifies the important variables, he can effect the material formulation, processing variables, and in-service environments to improve weatherability. Equally important, the investigator can optimize variables identified as trivial by the process to reduce manufacturing costs! This advantage may be especially useful in raw material cost-cutting projects. The materials weathering research efforts can then be focused in an efficient manner on the variables that the process indicates are important. AWSG has begun to apply other Taguchi tools, such as Signal-to-Noise Ratios and Taguchi Loss Function, in the research and analysis of AWSG's new weathering models (2). 

 Taguchi's fractional factorial experimental designs answer weathering research questions such as: 


  • Of the nine components in this vinyl formulation, which have the biggest effect on yellowing after five years Florida exposure, what is each component's order and magnitude of importance on yellowing, and which components can be optimized for cost without sacrificing weathering performance?
  • Of the ten major production line variables the line operator can control, on which should I establish control charting to improve quality of weatherability, and approximately what mean and tolerances should I begin with?
  • Of the major weathering agents this product will be exposed to (temperature, moisture, irradiance, pollution, abrasion, solvents, biological, cycling, etc.), which require research efforts to improve customer satisfaction for weathering performance?
  • For the major weathering failure modes I have identified in the FMEA, what are the risks associated with each?
  • For my material, which of these many weathering variables can be increased in order to accelerate weathering for test development?
  • For this vendor's candidate material, which environmental variables have the biggest effect on the system's weatherability?

 Clearly, the types of weathering research questions addressed with Taguchi's fractional factorial screening approaches represent a different level of context than single variable (3) and ramp level experiments (4), which make up the majority of weathering experiments performed today. 


  During the summer of 1999, Atlas Weathering Services Group performed several fractional factorial weathering experiments on commercially available materials. The author has used the same 10-step procedure effectively in many screening experiments, both for investigating manufacturing processes and weathering processes. This ten-step process may need modification for specific applications and is not exhaustive in detail but will serve to identify the major components and sequence for many weathering experiments. This presentation describes a straightforward ten-step procedure leaving the "why" to more esoteric and involved statistical publications. The reader should become familiar with theoretical underpinnings of these designs. 

Figure 5. Low and High Black Panel Temperature Distributions Achieved on 16 EMMAs

Step 1: 

 The general purpose of this investigation was to better understand the effect of specific pretreatment and weathering variables on appearance properties using the EMMA natural accelerated weathering device. Understanding how selected variables effect EMMA weathering may indicate: 1) which variables are most important for weathering different material types, 2) which variables are important for focused AWSG R&D efforts, and ultimately 3) which variables to control with advancements in the EMMA weathering process. Additionally, it was hoped this experiment would provide a check of "conventional wisdom" regarding weathering phenomena of these materials as described in the literature, by AWSG customers, and in previous parametric studies performed at DSET Laboratories. 

 Accelerated Weathering Device, EMMA:

 Engineers often want to accelerate UV degradation using NATURAL sunlight. Inventors developed an elegant solution to concentrate the image of several suns onto a single target area of the test material. This method became known as the "EMMA", an acronym for Equatorial Mount with Mirrors for Acceleration. This test method is detailed in 

 Figure 1. Optical System for EMMA Fresnel Reflecting Concentrator

 ASTM G90 (5). Direct solar radiation reflects off flat mirrors toward a target area. Ten flat mirrors measuring approximately 15 cm by 142 cm are laid side by side on a follow- the-sun track rack (see Figure 1). The ten mirrors are adjusted so the reflected solar image from each mirror coincides on the target area suspended above the track rack. The mirror bed tracks the sun throughout the day keeping the reflected solar image on the target. A blower forces air across the target to cool test specimens. The optical system is often described as a Fresnel reflecting solar concentrator with mirrors positioned as tangents to an imaginary parabolic trough. 

 The test specimen's optical properties (solar reflectance, absorptance and emittance), thermal conduction, ambient temperature and irradiance determine the actual temperature a material reaches. Use of proper materials for reflective mirrors represents a primary critical consideration for the EMMA test method. Mirrors must be highly reflective throughout the entire solar spectrum in order to properly concentrate both UV and visible light onto targets. The reflective material must possess robust weathering characteristics since it will undergo considerable track rack exposure. The reflective material used must maintain high specular reflectance from 300 nm to 2500 nm throughout its use on the exposure device. The next logical step in EMMA development increases levels of the critical weathering variable of moisture; hence EMMAqua = EMMA + Water. 


 AWSG has conducted a number of Taguchi Fractional Factorial experiments on a variety of materials including: current automotive paint systems, current coil coated building materials, colored-in polyethylene sheet extrusion, clear polycarbonate sheet with a UV protected surface, and injection molded clear polystyrene reference material used as indicators in Weather-O-Meters. All materials are commercially available at the time of this writing and were purchased from retail sources by AWSG. Results from one type of material often differ from results observed on different materials. This observation re-emphasizes the material dependency of weathering reactions and the need to test all materials rather than draw inferences from weathering results observed on one material to a different material. Although all the listed materials were subject to this experiment design, this presentation presents only results for a clear coat automotive coating, commercially available and currently used on American automobiles. The top coat is a clear coat polyester acrylic melamine applied over a chrome yellow basecoat and E-coat primer on top of cold rolled steel. 

 Step 2:

 Nine independently controllable variables were selected for this experiment from two types: exposure variables and pretreatment variables. 29 or 512 trials would have been required to perform this analysis using full factorial approaches. Both the pretreatment and exposure variables were included as follows:


  1. The temperature of the EMMA exposure under irradiance as indicated by metal black panel temperature instruments mounted on the EMMA target next to the specimens (TEM). This variable was controlled by changing the amount of air flow available for cooling the specimens.

  2. The strength of the radiant flux striking the surface and total radiant exposure from the solar spectral distribution. This variable will be referred to as "irradiance" for the rest of this presentation (IRR). This variable was controlled by changing the number of mirrors focused on the target area.

  3. The application of a daytime spray cycle, which wet the specimen's surfaces during periods of irradiation (SPR). This variable was controlled by turning on or turning off the water source available for the daytime spray cycle.

  4. The application of warm liquid water during periods when specimens were not being irradiated. This variable will be referred to henceforth as nighttime soak (NTS). Nighttime soak was controlled by removing the specimens from the target boards and placing them in a 40° C water bath each night and placing the specimens back on the target boards the next morning for daytime exposure.

  5. Abrasion of the specimen's surface prior to exposure (henceforth referred to as abrasion pretreatment). An AATCC Crockmeter device was modified to move the abrasive cloth back and fourth on the specimen's surface under highly repeatable conditions. The acrylic melamine clear coat was subjected to ten reciprocal strokes using 2400 grit abrasive. The abrasion pretreatment resulted in a light haze or scratching on the specimen's surface.

  6. Thermo-mechanical cyclic stressing of the specimens using a soak - freeze - thaw pretreatment (SFT). Specimens were soaked for 16 hours in a 49° C de-ionized, re-circulating water bath followed by immediate placement into a -10° C freezer for two hours followed by a six hour thaw/dry in ambient office conditions. The cycle was repeated five times. 

  7. Chemical pretreatment of the specimens was accomplished by immersing the specimens in a 40° C bath of dilute hydrogen peroxide for 24 hours (CHM). After immersion, the specimens were immediately flushed with de-ionized water for 15 minutes.

  8. A high UV irradiance pretreatment by exposing specimens to a proprietary process reported to improve weathering resistance (ARC).

  9. Oven pretreatment was accomplished by placing the specimens in an air circulating oven set to 70° C for 84 hours (OVN).

 Pretreatments were performed in a specific sequence: abrasion preceded soak - freeze - thaw which preceded chemical pretreatment which preceded UV arc which preceded oven, all of which preceded EMMA exposure. Prior to initial measurements, all specimens were thoroughly washed using a 5% mild detergent solution, gentle hand-wipe and thorough rinsing with de-ionized water after being pretreated. 

 Step 3

 High and low settings for each input variable were selected according to the schedule shown in Figure 2. This experiment utilized only two levels. 



VariableLow SettingHigh Setting

Nominal -7° CNominal +7° C

8 Mirrors (-10 %)10 Mirrors (+10 %)
Daytime Spray

No Daytime SprayWith Daytime Spray
Nighttime Soak

No Nighttime SoakWith Nighttime Soak
Abrasion Pretreatment 

Not AbradedAbraded
Soak-Freeze-Thaw Pretreat

No Soak-Freeze-Thaw PretreatSoak-Freeze-Thaw Pretreatment
Chemical Pretreatment

No Chemical PretreatmentChemical Pretreatment
High UV Pretreatment

No High UV PretreatmentHigh UV Pretreatment
Oven Pretreatment

No Oven PretreatmentOven Pretreatment

 Figure 2. High and Low Variable Settings 

 Step 4

 An L16 fractional factorial array was selected for this experiment as shown in Figure 3 (6). AWSG made an important modification to the traditional Taguchi approach in assigning variables to the L16 array for this experiment; although the L16 can theoretically handle up to 15 independent variables, we did not fully saturate the array with variables. The nine variables identified for this investigation fit into the L16 while leaving six columns blank. A traditional Taguchi approach would be to assign experimental variables to all available columns or to select a smaller array. Interactions between two or more variables, however, are very important in weathering research. Saturating arrays with variables confounds the effects of interacting variables. It seemed too early in development of AWSG's new weathering models to rule out interactions. Possibly, later in model development, we will adopt more traditional Taguchi approaches and saturate the arrays with variables. Additionally, the blank columns were used in the analysis for estimating the background variance, to check for significance of results, as well as to check for some interactions. The Taguchi Linear graph for this design appears in Figure 4 (2).



 TemperatureIrradiance Daytime Spray  Pretreat

Soak- Freeze- Thaw
Nighttime Soak  Pretreat





Trial No. 


















 Figure 3. An L16 Fractional Factorial Array: 1 = Variable at Low Setting, 2 = Variable at High Setting.

 The remaining variables for this experiment were assigned using similar justification. The final array with variable assignments is shown in Figure 3. Figure 4 shows the arrangement of variables and interactions as a linear graph. This schedule details 16 trials with unique settings of nine different variables. 

 Step  5

 The 16 trials prescribed by the experimental array were performed simultaneously on 16 different EMMA (Equatorial Mount with Mirrors for Acceleration) machines at DSET Laboratories beginning on July 29, 1999. (6). Specimens were exposed to 810 MJ/m^2 UV nominal dosage and measured for gloss. The specimens are still on exposure as of the writing of this paper. The machines utilized were quality checked throughout the exposure. Proper variable settings were maintained for each trial. Great care was utilized to insure all variables outside the scope of the experimental design were blocked across all 16 trials. At several intervals throughout the exposure, black panel temperatures were measured in the target area. A graph representative of the temperature differences is shown in Figure 5. Note that the target black panel temperature variable was controlled independently of the irradiance variable for the 16 trials. 

Step 6

 After the exposure period, specimens were removed from exposure, measured for appearance properties and compared to their initial values before exposure. The automotive coating systems were measured for 20° gloss according to ASTM D523 - 94 (7). Specimens were measured five times across the exposed surface and the mean reported for each specimen. Two specimen replicates were included in each trial. The values reported are calculated deltas between the initial measurements before exposure and final measurements after exposure. The delta values for the two specimens included in each trial are shown in Figure 6. The trial number for the output values corresponds to the trial numbers used in the fractional factorial array schedule in Figure 3. 

 Step 7

 It is important to perform quality control checks of all work throughout all phases of the experiment. A simple error in variable settings, data entry, measurements, etc., may void the orthogonal basis of the experiment and result in erroneous decisions. A thorough check of exposure conditions was performed weekly during the exposure. The scheduled pretreatments for specific trials were confirmed. All work was recorded as performed in engineering logs and written checklists were utilized throughout the work effort. Measurements were confirmed based on comparison of replicate values. 



Trial NumberDelta 20 degree Gloss Replicate "A" Delta 20 degree Gloss Replicate "B" 

 Figure 6. Delta Gloss for Exposed Replicates 

 Step 8

 Analysis of the output data was performed at two levels: 1) review of experimental grouping using graphical techniques, and 2) ANOVA. 

 1. Graphical Technique

 An analysis using statistical grouping was performed using the delta 20° gloss values. A mean was calculated for the eight sets of specimens exposed to high temperature. A mean was calculated for the eight sets of specimens exposed to the low temperature condition. These two mean values were plotted on a graph and connected with a line. Next, a mean was calculated for all specimens exposed to high irradiance. A mean was calculated for all specimens exposed to low irradiance and these two means were plotted next to the temperature variable means. This procedure was continued for all the variables included in this design. This graphing technique allowed the effects of each variable to be compared with the effects of all other variables with a single graph. Using this analysis, it was quite simple to determine which variables had the largest effect on gloss loss of the auto coating and the magnitude of the effect compared to that of the other variables. The graph obtained is shown in Figure 7.

Figure 7. Main Effects Graph for Acrylic Melamine Gloss Loss


 Because the two techniques so far do not sufficiently account for within and between trial variance, an ANOVA analysis was performed using a software package from ASD, Inc. ANOVA Analysis of Fractional Factorial Screening experiments allows experimenters to complete the following table: 



Variables Tested That Are InsignificantVariables Tested That Are Significant But UnimportantVariables Tested That Are Significant And Important


 The ANOVA performs an analysis using the F test. In an F test, an F ratio is calculated comparing variance due to treatment variables to variance due to experimental noise. The F ratio is often described as having the between column variance in the numerator and the within column variance in the denominator (8). For this analysis, we simply compare the effects caused by input variables to the background variation of the experiment. As this ratio approaches one, we say that the effects due to the input variables are not significantly different than the background variation of the experiment or that the effect of input variables is not significant. However, as the F ratio becomes larger and larger, the effects due to the input variables become more different than background experimental noise; a large F ratio indicates the effect of the input variables is significant. 

 The variation due to input variables is easily traced back through the orthogonal array. The ANOVA uses outputs from the columns to which the input variables were assigned. An estimate of the variation due to experimental noise (experimental error, background variation) comes from a combination of three sources for this analysis. First, if replicate samples are used in the experiment (n>1), the sample-to-sample variation in the results can be a good estimate of within treatment variation. Second, the columns that were left blank in the orthogonal design represent a rich source for estimating experimental error once interaction effects are ruled out. By incorporating these blank columns and using unsaturated designs during the design steps of the experiment, we are now rewarded with a very robust and statistically valid estimate of experimental error. Third, any of the input variables that are not significant can also be pooled into the estimate of experimental variance. By using all three sources to develop a pooled estimate of experimental error, a very robust appropriate experimental error term can be developed for the denominator of the calculated F ratio. It is with this experimental error term that the effects of treatments are compared to determine significance. 

 Once the F ratio is calculated from the experimental data, it may be compared to values found in the standard F distributions given the degrees of freedom for the numerator and denominator and the desired confidence level. If the data generated F ratio value exceeds the critical F ratio value (from the table of standard F distribution values) the effect of the input variable can be said to be significant (the output due to that variable exceeds what would normally be expected due to random experimental noise). 

 For this analysis, the ANOVA table, treatment variables, pooled sources of experimental variance, and calculated F ratios are shown in Figure 8. Using the F column and r column in the ANOVA table, we can begin to assign a rank order to the significant input variables and understand the magnitude of effects compared to each other for acrylic melamine gloss loss as follows: 



Variables Tested That Are InsignificantVariables Tested That Are Significant But UnimportantVariables Tested That Are Significant And Important
1. Temperature1. daytime Spray1.
2. Irradiance2. Soak-Freeze-Thaw Pretreatment2.
3. Abrasion Pretreatment3. Chemical Pretreatment3.
4. Oven Pretreatment4. Interaction of Daytime Spray and Irradiance4.


 It is important to compare the ANOVA table with the goals from Step 1. ANOVA only indicates significance, not direction. For instance, the F ratio for ARC pretreatment shows the effect of high UV pretreatment on gloss loss after 810 nominal MJ/m^2 UV EMMA exposure is significant. Only the graphs, however, indicate that no pretreatment causes greater degradation while pretreating with high UV retards degradation of gloss. 


 The ANOVA table should also be inspected for effects of interactions between input variables. Recall that these interactions should appear in the columns that were left blank. The interaction between temperature and irradiance for this experiment should show up in Column 3, the interaction between irradiance and daytime spray should show up in Column 6, and so forth. In this experiment, the F ratio values for the interaction between irradiance and daytime spray are statistically significant indicating an important interaction in this experiment. After inspecting the F ratios of these interaction columns for significant effects on the output, if no significance is found, we can contribute these blank columns to our estimate of experimental variation. 



SourcePoolDFSVFS'/td> r
TEM 115.12515.1251.5495.8990.18
IRR 118.00018.0001.8438.7740.26
SPR 172.00072.0007.37262.7741.88
6 1595.125595.12560.932585.89917.55
SFT 1120.125120.12512.299110.8993.32
NTS 1840.500840.50086.055831.27424.90
ABR 166.12566.1256.77056.8991.70
OVN 121.12521.1252.16311.8990.36
CHM 1338.000338.00034.606328.7749.85
ARC 11058.0001058.000108.3241048.77431.42
e1 115.125  -9.226-0.28
(e) 22214.8759.767 286.0128.57


 Figure 8. ANOVA Table for Polyester Acrylic Melamine 20° Gloss

 Step 9

 Once the significance of the individual treatments is understood, it is appropriate to decide which input variables are important. Significance involves a statistical exercise. Importance is a human judgment exercise and often depends on other sources of data beyond the scope the experiment. For instance, in this experiment, a difference in daytime spray alone resulted in a difference in mean 20° gloss loss of about four units (19.5 with daytime spray to 16 without daytime spray). In some type of end-uses, this magnitude of difference may cause great user consequences, including formulation and/or process changes. In other end-uses, this magnitude of difference might not be important to the end-use at all. Prior to beginning this study, the end-user of this coating was interviewed regarding criteria for importance. The customer identified differences exceeding approximately 13 delta 20° gloss units at this experimental level (810 MJ/m^2 UV) to be important. Based on this information, the significant and important information can be completed as shown in Figure 9. 



Variables Testes That Are InsignificantVariables Tested That Are Significant But UnimportantVariables Tested That Are Significant And Important
1. Temperature1. Daytime Spray1. Nighttime Soak
2. Irradiance2. Soak-Freeze-Thaw Pretreat 2. High UV Pretreatment
3. Abrasion Pretreatment3. Chemical Pretreatment 
4. Oven Pretreatment4. Interaction of Daytime Spray and Irradiance 


 Figure 9. Summary Results for Acrylic Melamine Gloss Loss 

 Step 10

 Confirmation trials should always be conducted in conjunction with fractional factorial screening experiments. Confirmation trials should also be considered as a critical part of the screening experiment. This is especially important if high levels of input variable saturation are designed into the orthogonal array and where significant interactions are identified between several input variables. Only confirmation trials can decode alias characteristic of the fractional array. Two confirmation trials were conducted with input variables set as shown in Figure 10. 



Confirmation Trials:
#1-Least Degradation Predicted#2-Most Degradation Predicted
• No Nighttime Soak• With Nighttime Soak
• With UV Pretreatment• No UV Pretreatment

 Figure 10. Confirmation Trials 

 The remaining variables investigated in the screening experiment that were identified as not important or not significant were set to optimal levels for cost for both trials as shown in Figure 11.



Confirmation Trials:
Variables optimized for cost
• High Temperature Exposure
• High Irradiance
• No Soak Freeze Thaw Pretreatment
• No Oven Pretreatment
• No Chemical Pretreatment
No Daytime Spray


  Figure 11. Confirmation Trials Optimized for Cost 

 These confirmation trials were then conducted on identical Acrylic Melamine specimens using the EMMA accelerated weathering device. The data obtained was compared to the screening experiment results. The confirmation trials at 220 MJ/m^2 UV dose compared directly with the predictions made in the original screening experiment. From this confirmation trial it was concluded that no hidden interactions, confounds or aliases were operating in the original screening experiment. An additional trial to confirm and characterize the nature of the irradiance - daytime spray interaction has just begun and will be analyzed after publication of this paper. Follow-on efforts can now be focused on those variables the weathering process has indicated are significant and important. 


  There has been an evolution in the sophistication of experimental designs for weathering tests. The vast majority of current weathering exposures utilize more fundamental designs effecting few variables. This type of experimental design requires far more trials and thus more cost, less information, and poorer quality than more sophisticated approaches using screening fractional factorial experiments. Preceding fundamental level, few variable weathering trials with Taguchi style fractional factorial screening and confirmation experiments represent an efficient, stochastic, powerful approach for improving knowledge regarding weathering's n-dimensional hyper-volume of environmental effects on materials degradation.


  1. Luftig, J.T. Experimental Design and Industrial Statistics, Level I-IV; Luftig and Associates, Inc.: Farmington Hills, NJ, 1987. 
  2. Phadke, M. Quality Engineering Using Robust Design; Prentice-Hall: Englewood Cliffs, NJ, 1989. 
  3. Duncan, R.C. Introductory Biostatistics for The Health Sciences; John Wiley and Sons: NY, 1977. 
  4. Montgomery, D.C. Design and Analysis of Experiments, 3rd ed.; John Wiley and Sons: NY, 1991. 
  5. ASTM G90-94 Practice for Performing Accelerated Outdoor Weathering of Nonmetallic Materials Using Concentrated Natural Sunlight. 1994 Annual Book of ASTM Standards, vol. 14.02. Philadelphia, PA: American Society for Testing and Materials, 1994. 
  6. Taguchi, G.; Konishi, S. Orthogonal Arrays and Linear Graphs; ASI Press: Dearborn, MI, 1987. 
  7. ASTM D 523-89 Standard Test Method for Specular Gloss. 1998 Annual Book of ASTM Standards, vol. 6.01. West Conshohocken, PA: American Society for Testing and Materials, 1998. 
  8. Levin, R.I. Statistics for Management, 4th ed.; Prentice-Hall: Englewood Cliffs, NJ, 1987.