Chapter 8:  Differences Among Groups

l Techniques described in this chapter are not used by themselves to establish cause and effect, but only to evaluate the influence of the independent variable.

l Cause and effect are not established by statistics but by theory, logic and the total nature of the experimental situation.

The purpose of the statistical test is to evaluate the Null Hypothesis;  H_o.

In other words, do the levels of a treatment differ significantly at a probability level?

The tests cannot tell why they are different, though.

Besides showing that groups are different you want to show the size of the difference, and the strength of the association between the independent and dependent variables

t and the F ratios are used to determine the difference between groups.

w² is  used to estimate the degree of association (or % variance accounted for) between the independent and dependent variables. (similar to r² for the correlations)

Effect size (the standardized difference between groups) is also use to estimate meaningfulness.

Four Assumptions for the use of the t and F distributions

l Observations are drawn from a normally  distributed  population

l Observations represent random samples from populations

l The numerator and denominator are estimates of the same population variance

l The numerator and denominator of F (or t) ratios are independent.

The t and F tests are only slightly influenced by violations of these assumptions, but do not neglect them.

t test between a Sample and a Population Mean

         M - m

T = ------------------     equation 8.1

                _____

       s_m / Ö  n

 s_m = the standard deviation for the sample mean

 n = the number of obeservations in the sample

Example 8.1

Population: N = 10,000

Fitness class: n = 32

Sample Mean: M = 81

Population Mean: m = 76

Standard Deviatio s_m = 9                              __

Put data in equation  8.1: t = (81 – 76)/ (9/Ö32

t = 5/ 1.59 = 3.14

For significance of t = 3.14, check Appendix A.5

For df = n-1= 32 –1 = 31

Independent t test

Differences between two sample means

l Example: Two levels of intensity of training 40% and 70% of VO_2max on performing the 12-minute run.

How meaningful is the effect of training intensities?

Differences between groups: t = 13.81

Number of subjects in group 1: n₁= 15

Number of subjects in group 2: n₂ = 15

w² = (13.81² – 1)/(13.81² + 15 + 15 – 1)

w² = 189.72/219.72 = .86

86% of the total variance is accounted for by the difference in the treatments.

Effect Size: The standardized difference between the means

An effect size of 0.8 or greater is large, ~0.5 is moderate,  0.2 or less is low.

Dependent t Test

Two groups of scores are related in some manner.

– One group of subjects is tested twice

– Two groups of subjects are matched on one or more characteristics and are no longer independent

df = N – 1, N is the number of paired observations

One-tailed vs. Two-tailed Tests

l Appendix A.5

l If we do not know which mean will exceed the other a two-tailed test is used.

l But if our hypothesis says that one will be equal to or greater than the other, we could use a one-tailed test.

l Easier to get significance using one-tailed

Power: The probability of rejecting a Null Hypothesis when it is false

l Increasing difference between the means:

   Using stronger, more concentrated treatments, i.e., 12 week treatment instead of a 6 week treatment

2. Reducing the Variances of the scores:

    If the denominator becomes smaller the power is increased.

    Apply the treatments consistently.  The more consistent the application of the treatments the more similar the subject’s response to the dependent variable.

3. Number of participants in each group:

      The larger the number of subjects the greater smaller the denominator and the greater the t and the power is increased.

     The larger the N, the lower the t-ratio needed for significance.

  Summary: power is obtained by using strong treatments, administering those treatments consistently, using as many participants as feasible, varying alpha, or using an appropriate research design and statistical analysis

Analysis of Variance (ANOVA)

Using t-tests is a good method to determine differences between two groups, but when using more than two groups a ANOVA is used.

The groups represent levels of an independent variable.

Cardiovascular training at three levels of intensity: 40%, 60% and 80% of VO_2max

A t-test to compare groups 1&2, 1&3, and 2&3, could also be used.

This would violate the assumption that at the .05 level we expect 1 chance in 20 that we got a sampling error and a significant difference when there was none.

With each group appearing in 2 t-tests we have increased our chances of committing a type I error, rejecting the H_o when it is true.

ANOVA allows all the groups to be compared simultaneously.

Dividing the Between Groups MS by the Within Groups MS gives the F-ratio.

In Appendix A.6, the F-ratio is determined by finding the numerator df column, and the denominator df row.

MS_b df = 2; MS_w df = 12. 

(p < .05) we need an F-ratio ³ 3.88

(p < .01) we need an F-ratio ³ 6.93

Follow-up (post hoc) tests

Several follow-up tests protect the type I error

These include: Scheffe’, Tukey, Newman-Kuels, Duncan.  These are in order of the most conservative to the most liberal.  Conservative means it is more difficult to get significance, and liberal means it is easier to get significance.

Factorial ANOVA

Manipulate more than one independent variable and statistically evaluate the effects on a dependent variable.

There are more studies in the literature using the Factorial ANOVA than the simple (one-way) ANOVA

Components of a Factorial ANOVA

Main Effects are tests of each independent variable when the other is disregarded. 

In table 8.2 there are two levels of intensity; high and low, and there are two levels of fitness of participants; high fitness and low fitness

We are also interested in the interaction between levels of each factor.