Chapter 4: Formulating the Method

I. Typically this chapter contains the following sections :

Participants

Instruments and apparatuses

Procedures

Design and Analysis

II. Purpose

The purpose of the method section is to explain how the study was conducted.  The standard rule is that the description should be thorough enough for a competent researcher to reproduce the study.

III.  How to present Methodological Details

A. In Theses or Dissertations the following would be put in the appendix instead of the Methods chapter:

1. Instructions to participants

2. Samples of tests and answer sheets; i.e., questionnaires

3. Sample data recording sheet

4. Informed consent form

IV. Planning the Methods

1. The purpose of planning the method is to eliminate any alternative or rival hypotheses.  This really means that when you design the study correctly and the results are as predicted, the only explanation is what you did in the research.

Example: shoe size and math scores (figure 4.1)

2. Maxicon Principle:  Maximize the true variance due to the independent variable; minimize the error variance ( reduce all mistakes that could enter into the study to disguise the true relationship, or make sure the rival hypotheses are not the real explanations of the relationship.

V. Principles of Planning Experiments

1. Less is more, except number of participants

a. don’t take on too many variables to control or measure

b. Carefully evaluate the number of independent and dependent variables that are practically and theoretically important in your study.

2. Simple is better; KISS

a. True for design of the treatments to the analysis of data, to displaying data, to interpreting results

b. Understanding your data is an important concept

c. Plot data graphically and evaluate it carefully

VI. Describing Participants

A. Describes how and why the participants were selected, and which characteristics are pertinent to the study.

B. Are participants with special characteristics necessary for your research?

1. Age (children, elderly)      

2. Sex

3. Level of training

4. Level of performance

5. Size (weight, fatness, leanness)

6. Special types (athletes, cyclists, runners)

C. Can you obtain the necessary permission and cooperation from the participants?

D. Can you find enough participants?

E. You want to choose participants who will potentially respond to the treatments.

F. In experimental research the interaction among participants, measures, and the nature of the treatment program is essential in allowing the treatment program to have a chance to work.

G. Participants with high levels of PF or elite, skillful athletes, will not show improvement from moderate programs.

VII. Writing about the Participants

A.Exact number of subjects, and the number of those who dropped out of the study.

B. In the proposal, or plan of the study, you may not know exactly how many will volunteer, however you should know what you can accommodate, and how many you need for statistical purposes.

C.Participant characteristics like height, weight, age, VO_2max, etc. may be important for an exercise physiology study, but not relevant for a study in psychology, recreation, etc.

D. Characteristics need to be defined; i.e., “well-trained runners” is not precise enough, but “runners with VO_2max measures of over                 60 ml•kg^-1 •min^-1  “ would be better description.

E. The categorical variable age for groups 7-, 9- and 11-year olds, would be best written at 7 ±3 months, etc.  This describes the age interval better.

VIII. Protecting Particpants

Will  discuss this in Chapter 5, “Ethical Issues in Research and Scholarship”

IX. Describing Instruments

1. Information about the instruments, apparatuses, or tests used to collect data is used to generate the dependent variables in the study.

2. What is the validity and reliability of the measures?

3. How difficult is it to obtain the measures?

4. Do you have access to the instruments, tests, or apparatuses needed?

5. Do you know (or can you learn) how to administer the tests or use the equipment?

6. Do you know how to evaluate the participants test performance?

7. Will the tests, instruments, or apparatuses yield a reasonable range of scores for the participants you have  selected?

8. Will the participants be willing to spend whatever time is required for you to administer the tests or instruments?

X. Describing Procedures

1. Describe how the data were obtained, including all testing procedures for obtaining scores on the variables of interest. 

2. How tests were given and who gave them

3. Give details as to the setup of the testing situation and instructions to the subject

4. If the study is experimental, you should describe the treatments applied to the different groups

XI.  Planning Procedures

1. Collecting data

a. when, where, how much time is required

b. Do you have pilot data to demonstrate your skill and knowledge in using the tests and the equipment and how participants will respond?

c. Have you developed a scheme for data acquisition, recording , and scoring?

2. Planning the treatments (in quasi-experimental or experimental studies)

a. How long?  How intense? How often?

b. How will participants’ adherence to treatments be determined

c. Do you have pilot data to show how participants will respond to treatments and that you can administer these treatments

d. Have you selected appropriate treatments for the type of participants to be used?

3. The procedures section contains most of the details that allow another researcher to replicate the study.

a. Specific order in which the steps were undertaken

b. The timing of the study (e.g., time taken for different procedures and time between different procedures.

c. Instructions given to participants

d. Briefings, debriefings, and safeguards

XII. Pilot Work

1. Too many studies center their discussion on methodoligical faults that cause the research to lack validity.

2. Every thesis or dissertation proposal should present pilot work that verifies that all instruments and procedures will function as specified on the type of participants for which the research is intended.

3. You must demonstrate that you can use these procedures and apparatuses accurately and reliably.

XIII. Describing Design and Analysis

1. Design is the key to controlling outcomes from experimental or quasi-experimental research

2. The independent variables are manipulated in an attempt to judge their effects on the dependent variable

3. The well-designed study is one in which the only explanation for change in the dependent variable is how the participants were treated (independent variable)

4. The plans for data analysis are also presented

XIV. Establishing Cause and Effect

1. It is more than a statistical or design issue

2. The issue is logic

3. If the null hypothesis is not true, then what hypothesis is true?

4. Method of agreement:  If an effect occurs when A and B are present, and A and B have C in common, then C is the likely cause. (or part of it)

5. Method of disagreement: If the effect does not occur in E and F when C is the only common element absent, then C is the cause (or part of it)

6. The researcher’s theoretical beliefs as well as the study’s design and analysis are essential factors in establishing cause and effect.

XV. Interaction of Participants, Measurements, and Treatments

1. Correlational studies use selected participants to evaluate the relationship among two or more measures

2. Experimental studies use selected participants to evaluate whether a specific treatment causes a specific outcome.

3. There must be an appropriate range of scores

a. In correlational studies, the measures must capture the characteristics the researcher hopes to assess.

b. In experimental studies, the participants must be at the “right level” to respond to the treatments.  The treatments must be intense enough and frequent enough to produce changes that the researcher hopes to see, and the measuring instruments must be responsive to the treatment changes in participants.