ANNOVA Part 1

Why do we use analysis of variance (ANOVA) when we are interested in the differences among means?

ANOVA is used to compare differences of means among more than 2 groups. It does this by looking at variation in the data and where that variation is found (hence its name). Specifically, ANOVA compares the amount of variation between groups with the amount of variation within groups. It can be used for both observational and experimental studies.

When we take samples from a population, we expect each sample mean to differ simply because we are taking a sample rather than measuring the whole population; this is called sampling error but is often referred to more informally as the effects of “chance”. Thus, we always expect there to be some differences in means among different groups. The question is: is the difference among groups greater than that expected to be caused by chance? In other words, is there likely to be a true (real) difference in the population mean?

Although it may seem difficult at first, statistics becomes much easier if you understand what the test is doing rather than blindly applying it. Hopefully ANOVA will become clear by following the steps below.

The ANOVA model

Mathematically, ANOVA can be written as:

x _ij = μ _i + ε _ij

where x are the individual data points (i and j denote the group and the individual observation), ε  is the unexplained variation and the parameters of the model (μ) are the population means of each group. Thus, each data point (x_ij) is its group mean plus error.

Hypothesis testing

Like other classical statistical tests, we use ANOVA to calculate a test statistic (the F-ratio) with which we can obtain the probability (the P-value) of obtaining the data assuming the null hypothesis. A significant P-value (usually taken as P<0.05) suggests that at least one group mean is significantly different from the others.

Null hypothesis: all population means are equal

Alternative hypothesis: at least one population mean is different from the rest.

Calculation of the F ratio

ANOVA separates the variation in the dataset into 2 parts: between-group and within-group. These variations are called the sums of squares, which can be seen in the equations below.

Step 1) Variation between groups (Fig 1)

The between-group variation (or between-group sums of squares, SS) is calculated by comparing the mean of each group with the overall mean of the data.

Specifically, this is:

i.e., by adding up the square of the differences between each group mean  and the overall population mean , multiplied by sample size , assuming we are comparing three groups (i = 1, 2 or 3).

We then divide the BSS by the number of degrees of freedom [this is like sample size, except it is n-1, because the deviations must sum to zero, and once you know n-1, the last one is also known] to get our estimate of the mean variation between groups.

Step 2) Variation within groups  (Fig 1)

The within-group variation (or the within-group sums of squares) is the variation of each observation from its group mean.

SS_R = s²_group1 (n_group1 – 1) + s²_group2 (n_group2 – 1) + s²_group3 (n_group3 – 1)

i.e., by adding up the variance of each group times by the degrees of freedom of each group. Note, you might also come across the total SS (sum of ). Within SS is then Total SS minus Between SS.

As before, we then divide by the total degrees of freedom to get the mean variation within groups.

Step 3) The F ratio

The F ratio is then calculated as: 

If the average difference between groups is similar to that within groups, the F ratio is about 1. As the average difference between groups becomes greater than that within groups, the F ratio becomes larger than 1.

To obtain a P-value, it can be tested against the F-distribution of a random variable with the degrees of freedom associated with the numerator and denominator of the ratio. The P-value is the probably of getting that F ratio or a greater one. Larger F-ratios gives smaller P-values.

Assumptions of ANOVA

♦ The response is normally distributed

♦ Variance is similar within different groups

♦ The data points are independent

Reference

http://www.edanzediting.com/blog/statistics_anova_explained#.Ve60p9KqpHw