Chapter
14: The Survey
l Descriptive
research is a study of status
l Used
frequently in Education and Behavioral sciences
l Survey is
the most common descriptive research method
The
Questionaire
l The
questionnaire and the interview are essentially the same
l Questionnaires
are usually answered in writing
l Interviews
are conducted orally
l Developing
a questionnaire is essentially the same for an interview
Objectives
l Clearly
defined objectives
l What
information is needed
l How will
the information be analyzed
l Analysis is
determined in the planning stage
l What are
the variables to be analyzed
Delimiting
the Sample
l A specific population is usually described
l The participants must have the answers to the
questions
l The representatives of a sample: stratified random
sample
l Selecting a sample should be based on the variables in
question; which affects the generalizability of the study.
l The representatives of a sample are more important
than the size of the sample
Constructing
a Questionnaire
l Difficult
to develop understandable questions
l Develop a
table that includes categories of responses, comparisons, and other breakdowns
of data analysis
l This will
help in knowing how to handle each answer and how each will contribute to the
objectives of the study.
l Select the
format for the questions
Open-ended
Questions
l “How do you like your job?” Easy to write, allows the
respondent latitude to express feelings and to expand on ideas.
l Most respondents do not like open-ended questions;
they require more time to answer than closed questions.
l Limited control over the nature of the response;
respondent often rambles and strays from the question, which are difficult to
categorize or synthesize for interpretation.
Closed
Questions
Ranking forces the respondent to
placed responses in a rank order according to some criterion.
Rankings can be summed and
analyzed quantitatively.
Scaled items are one of the most
common types of closed questions.
Answer to a question in terms of: Rarely, some,
often, frequently; Strongly agree, agree, undecided, disagree, strongly
disagree.
Categorical responses offer the respondent only two choices; yes/no;
agree/disagree.
1.
Do not require as much time
2.
Do not give undecided, sometimes, etc. an option.
3. Answer
flow chart.
Borg and Gall’s Rules for Construction of a Questionnaire
The items must be clearly worded so
that the items mean the same to all respondents. Avoid words that have no
precise meaning, such as “usually,” “most,” and “generally.”
Use short questions rather than
long questions because they are easier to understand
Do not use items that have two or
more separate ideas in the same questions
Avoid using negative items, such as
“Physical education should not be taught by coaches.” The negative word is often overlooked
Avoid technical language and jargon. Attempt to achieve clarity and the same
meaning for everyone.
Be careful that you do not bias the answer
or lead the respondent to answer in a certain way. Sometimes one what the
correct response to a question is from the way it is written.
Appearance
and Design
Important to the completion and return of the questionnaire
Many people have negative attitudes toward questionnaires, so researcher
must overcome this negative attitude
Making the questionnaire attractive
Should provide the name and address of the investigator.
Instructions for answering the questions are
clear and complete
Examples are provided for any items that are
anticipated to be difficult to understand
The first few questions should be easy to
understand; do not start with questions that require considerable thought or
time
Do not ask questions that you have access to
the information already.
Short questionnaires are more effective than
long ones.
Pilot
Study
Recommended for any type of research but is
imperative for a survey.
Have a few colleagues read over the
questionnaire and give critique
Second pilot study uses people from the
intended population
Results of the second are subjected to item
analysis.
Are the items clear and appropriate.
Determines whether the instructions are
adequate
Writing
the Cover letter
The effectiveness of the first mailing depends on the cover letter.
Explains the purpose and the importance of the survey in a succinct and
professional manner
Privacy and anonymity are assured
Make appeal for the respondent’s cooperation
Letters should look like they have
been individually typed and use their name.
Endorsements by agencies, associations, or institutions; or the
organizations letterhead
Whether the study is funded.
Identify yourself by name and position.
If part of a thesis or dissertation give advisor’s name.
Contact individuals by card, letter, phone, or email asking for
participation.
May induce a person to respond by giving rewards.
Request that the questionnaire must be returned by a certain date. Also,
specify the due date on the questionnaire itself.
Give about one week to respond plus mailing time.
Do not make any grammatical errors, misspelled words, or improper spacing
in the cover letter or questionnaire.
Sending
the Questionnaire
What is the best time for the initial mailing?
Include a self-addressed, stamped envelope to mail back the questionnaire.
There is considerable cost in mailing and return mailing, as well as,
printing, etc. Bulk mailing may help, or getting a sponsor to contribute to the
study financially.
Email, fax, internet are currently being used.
Follow-up
It is unlikely that you will get 100% return
off of the first mailing.
A follow-up letter is almost always needed.
Send a card after about 10 days from the
initial mailing.
Stress the importance of the study and their
contribution.
About 10 days after the card send another
cover letter, questionnaire, and self-addressed stamped envelope.
Follow-up
Letter
Be tactful, don’t chastise
Nonresponse is likely when the subject matter is controversial or
sensitive.
People who have a particular interest in the survey topic are more likely
to respond than people who are less interested.
People responding to the survey have a more favorable attitude about the
topic and themselves, than the reluctant respondents.
Usually about 20% are not returned
May do a survey among the nonrespondents by phone, or special cover letter
via registered mail. Survey about 5-10% of the nonrespondents
Compare the nonrespondents to the respondents. If they are the same you may assume that they are not different
from the initial respondents. If they
are different you may try to get more of the nonrespondents or make a note of
it in the research report.
Tracking responses and nonresponses may be done with a code number on the
questionnaire, however insure that their information is confidential. A postcard with the number can be mailed
with the questionnaire and returned with the questionnaire.
The
Personal Interview
Steps for the interview and questionnaire are essentially the same. Difference is in how data is gathered.
In an interview, the data is more reliable, and returns are greater.
Same Sampling Technique as Questionnaires
Must prepare for the interview
Rehearse the interview, one tends to get better with practice (pilot study)
Establish Rapport and make the person feel at ease.
Get permission to tape record, if you want to record the interview.
If the information must be recorded by hand, one should have a coding
system so that the amount of time to record the answers is minimal.
Do not interject your bias, or argue with respondent.
Do not stray from the principle objective of the survey, and tactfully keep
the respondent from rambling.
Do not ask yes/no questions, or insert your own point of view.
Four
types of Questions
1. Hypothetical:”Let’s suppose this
is my first day of student teaching.
What would it be like?”
2. Devil’s Advocate: “Some people
say that professional educational courses are of little value for the
student-teaching experience. What would
you say to them?”
3. Ideal position: “What would you
think the ideal student-teaching preparation program should be like?”
4. Interpretive: “Would you say that the student-teaching experience is
different from what you expected?”
Advantages
of the Interview over the Questionnaire
1. The interview
is more adaptable. Questions can be
rephrased and clarification can be sought through follow-up questions.
2. The interview
is more versatile with regard to the personality and receptiveness of the
respondent.
3. The interviewer
can observe how the person responds and can thus achieve greater insight into
the sensitivity of the topic and the intensity of feelings of the respondent. This
can add considerably to the validity of the results, as respondents’ avoiding
sensitive topics is one of the greatest threats to validity in questionnaire
studies.
4. Because each
person is contacted before the interview, interviews have a greater rate of
return. Moreover, people tend to be
more willing to talk than to fill out a questionnaire. A certain amount of ego is involved because
a person who is interviewed feels more flattered than one who receives only a
list of questions.
Telephone Interview
1. Is
less expensive than traveling
2. Can
work from a central location and monitor the quality of the interviews
3. Many
people are easily reached by phone than personal visit
4. May
reach a wider geographical area
5. People
usually respond more condidly to sensitive questions over the phone than in
personal interviews
6. Using
the computer in telephone interviewing helps with data collection.
7. There may be some bias involved in telephone interviews due to the time
of day the survey is made.
Normative
Survey
Involves establishing norms for abilities, performances, beliefs, and
attitudes.
A cross-sectional approach is used.
A questionnaire is not used but a battery of tests.
Data is analyzed by standard scores, and percentiles
Drawbacks to any normative survey is the test selection, and the
standardization of procedures.