Society
for Amateur Scientists
A formal design method| |
|
|
|
|
Society
for Amateur Scientists
A formal design method |
||
|
|||
|
|
|
Related Products: Society for Amateur Scientists
|
Sponsored by:
Design Your Experiments Part V: A formal design methodby Kevin T. Kilty Why use a formal design method? There are several reasons...
Over the next few installments I may refer to these concepts without stopping to explain them...
My objective is to measure the surface cleaning capability of a solvent such as isopropyl alcohol (IPA) in water. There is only one factor in this, which is the concentration of IPA in the water. I plan to use a high concentration and a low one; so, I am testing the factor at two levels. There is a problem with the experiment in that a surface may have distinct cleaning properties not related to concentration of IPA. This is a source of noise in the experiment I can handle with local control of error. The method of control I plan to use is to divide each test surface into two samples. I perform cleaning using one level of IPA on one sample of the pair and use the other level of IPA on the other sample. I will pair the results for analysis. The result of my experiment is N different blocks each with 2 experimental units. Replication in the experiment I get by having N experimental units for each of the two IPA concentrations. I randomly allocate IPA concentration to sample within each block, and randomly order my testing. Pairing and randomization provide local control of error. Factorial DesignI'll describe a randomized complete block design using what is known as a contrasts table to represent it. First, I decide upon the factors that my experiment will test. At the present time I'll denote these by capital letters such as A, B, C, and so forth. I'll use subscripted symbols xai, xbi, and so forth to indicate the value of a factor on the ith replication of an experiment. Factors could represent a continuous quantity like temperature or pressure, or a category like the manufacturer of a product. A factorial design uses every possible combination of treatments. The contrasts table is a way of representing this. Typically, when I run an experiment to screen many factors to find those having the most influence, I apply factors at two levels, a high level and a low one. The high level I indicate with a +1 and the low level with a -1. The exact value of the level is unimportant at this time, but as an example, suppose the factor is temperature. Then the +1 level could be 75C and the -1 level could be 45C. In other words I have scaled the actual units of a factor in such a way that 30C corresponds to +2 change in factor level. I need this information later to return to true units. The form of the contrasts table at this stage involves only the factors (I=a constant intercept) and particular levels involved in the experiment. The contrasts table helps insure that I include all factor combinations. Contrasts Table Factor Levels I A B C... ----------------------- 1 -1 -1 -1 1 +1 -1 -1 1 -1 +1 -1 1 +1 +1 -1 1 -1 -1 +1 1 +1 -1 +1 1 -1 +1 +1 1 +1 +1 +1 -----------------------At this point I see that I'll have 8 distinct treatments, so I will need materials to produce at least 8 experimental units. I'll allocate treatments randomly to the experimental units. In a completely randomized design all treatment combinations I allocate in such a way that each experimental unit has the same chance of receiving any treatment. If I have organized my experiment into blocks to help control error locally, such as a paired experiment, then I need to make up sufficient experimental units per block for me to apply all treatments per block. This is a randomized complete block design. A mathematical model Consider an experiment involving only 2 factor levels. The most general model I can produce from it is one that includes terms involving each factor raised to the first power. I cannot go beyond first power because I am testing each factor only at two levels and only a line through two points is a unique construction. I can, however, include products of factors. So a possible model from my experiment is... Yi = C0 + C1xai + C2xbi + C4xaixbi + ni where,The Y's are an outcome, or objective, of the experiment. The C=92s are coefficients which I can determine through the experiment outcome. They describe the gain that each factor supplies in controlling the outcome of the experiment. The factor n is an error or noise component. If this model is a true model, then the n's are nothing more than random variables. That is, they are realizations which follow some noise density. If the model is not a true one, then in addition to random noise the n's contain wrong model noise. Wrong model noise may look so much like random noise that I cannot tell the difference. However, I can reduce or even eliminate the wrong model noise by choosing a better model. Concept of interaction The model I introduced above has linear terms in each factor, so that I can calculate the effect of each factor in isolation. The factors in a factorial design experiment are like basis vectors in a space. The parameter associated with each factor is orthogonal to the parameter for each other factor. You can show this by using a dot product if you wish. There are also terms in the products of factors. These allow me to represent interaction of one factor with another. By interaction I mean that the effect of one factor or the other is not solely dependent on the level of that factor, but depends also on the level of another factor. The contrasts table below shows all of the factors and their various possible interactions. Depending on how many interactions I plan to calculate, a full factorial design like this might have hidden (implicit) replication in the sense that it contains more information than I need at a minimum to calculate all interactions. The minimum requirement is that I need one experiment per interaction or factor I plan to calculate. If my mathematical model has X unknown coefficients, then I need at least X experiments to calculate them. If I hope to account for experimental noise, then I need more experiments beyond this minimum, and I'll obtain them from replicating experiments at some, or all, treatments. Once I have used data to estimate parameters, some degree of freedom in the data is gone. One degree of freedom per parameter is the rule. However, I have an estimate of n for each experimental result. This means the estimates of n from one observation to the next are never fully independent of one another. Contrasts Table Factor Levels and Interactions I A B C AB AC BC ABC ------------------------------------------------- 1 -1 -1 -1 +1 +1 +1 -1 1 +1 -1 -1 -1 -1 +1 +1 1 -1 +1 -1 -1 +1 -1 +1 1 +1 +1 -1 +1 -1 -1 -1 1 -1 -1 +1 +1 -1 -1 +1 1 +1 -1 +1 -1 +1 -1 -1 1 -1 +1 +1 -1 -1 +1 -1 1 +1 +1 +1 +1 +1 +1 +1 ------------------------------------------------- The use of this model and experiment design predates the modern computer, so there are manual methods of estimating the effects of factors and the value of coefficients by hand quickly. It is more expedient to use a spreadsheet like Excel to perform a regression analysis on the experimental data. However, let me explain the manual method. In the table below I have included only the first four columns of the contrasts table, but I have added an extra column which I built from the headings of the contrasts table by noting which factor or combinations of factors have plus signs. I have added a further column for the observed effect (Y) for each treatment from the outcome of each experiment. To find the coefficient appropriate to factor A, I simply use the sign in the 'A' column as a multiplier for the corresponding experimental result. Then I add all of the effects and divide by the number of rows having the same sign for each factor. In other words, I calculate the average response at the high level minus the average response at the low level. Contrasts Table Factor Levels I A B C Y -----------------------...------------- 1 -1 -1 -1 I 20 1 +1 -1 -1 A 18 1 -1 +1 -1 B 3 1 +1 +1 -1 AB -5 1 -1 -1 +1 C 6 1 +1 -1 +1 AC 15 1 -1 +1 +1 BC 23 1 +1 +1 +1 ABC -2 -----------------------...-------------So, as an example I can find the coefficient for The 'A' factor, the coefficient in front of xa in the model, as... Effect of A = (A+AB+AC+ABC)/4 - (I+B+C+BC)/4 = (18-5+15-2)/4-(20+3+6+23)/4=-6.5I can find all of the other effects similarly. Remember that the coefficient found through this method is scaled to a total difference of 2 in the response. To convert to actual units I need to return to the original scaling I did to set up levels, and recall that a difference of 2 for my temperature response is actually 30C. Using a probability plot It is rare to compute a model by hand nowdays. Generally I use standard regression software, running as a stand alone program, or using Excel, to compute the regression coefficients. The regression software will also give me statistics regarding how well the data fit the model. However, an alternative means of evaluating the success of the experiment is available by making a normal probability plot. Let me explain this now, and save the use of a spreadsheet for later. Suppose I have a model of three factors and all possible interactions, including the interaction term for all three factors. I have 7 effects values. I can rank these from smallest to largest and graph each against a percentile rank calculated as (i-0.5)/k, where i is the rank of the factor and k is the total number of factors. I can also calculate the standard normal deviate for these ranks using a tables book. I then plot the magnitude of the effects, which I have just calculated by hand, against the normal deviate. This produces what people call a normal probability plot. If all effects land along a single line, then they are consistent with the hypothesis that all of the true coefficients in the model (the effects) are actually zero, and they are only obtaining non-zero values through noise inspired fluctuation. If one or several of the effects land far from the straight line, I mean unusually far to the right or left, then this implies effects which are over represented compared to normal fluctuations and must have a significance to them. The normal probability plot is more qualitative than it is quantitative. The analysis of variance, using as it does the F ratio test, is much better for quantitative analysis. Center runs when the factor is continuous.My experiments to this point use just two factor levels. When factors are continuous then it makes sense for me to place a third factor level at the mid-point between the two extreme values, or even take additional data between any two factor levels. What I gain by doing this is:
Ckxai2 In the next installment I'll
show an example of an experiment planned from the beginning, and the analysis
of experimental results using Excel. Reprinted from:
|
