Social Research Methods/Research Design

This chapter provides a general introduction to research design by examining several issues:
The main purposes of social research
Units of analysis
How to design a research project
The elements of research proposals
Measurement

Three Purposes of Research

Social research can serve a variety of purposes. Three of the most influential and common purposes of research are exploration, description and explanation.

Exploration involves familiarizing a researcher with a topic. Exploration satisfies the researcher's curiosity and desire for improved understanding. Exploration tests the feasibility of undertaking a more extensive study. Exploration helps develop the methods that will be used in a study.

Description involves describing situations and events through scientific observation. Scientific descriptions are typically more accurate and precise than causal ones. For example, the U. S. Census uses descriptive social research in its examination of characteristics of the U. S. population.

Explanation involves answering the questions of what, where, when, and how. Explanatory studies answer questions of why. For example, an explanatory analysis of the 2002 General Social Survey (GSS) data indicates that 38 percent of men and 30 percent of women said marijuana should be legalized, while 55 percent of liberals and 27 percent of conservatives said the same. Given these statistics, you could start to develop an explanation for attitudes toward marijuana legalization. In addition, further study of gender and political orientation could lead to a deeper explanation of this issue.

The Logic of Idiographic vs. Nomothetic Explanation

• Idiographic explanation - a "full", detailed, in-depth understanding of a case; for practical reasons, only a few subjects are studied in this way. An idiographic explanation of the marijuana legalization survey would involve a more conclusive list of factors that could influence a person's viewpoints on this issue. Therefore, an idiographic explanation would need to consider several factors, such as information from parents and previous experiences, not just political orientation.

• Nomothetic explanation - a generalized understanding of a given case, with the goal of finding new factors that can account for many of the variations in a given phenomenon; is applicable to many subjects. Regarding the survey mentioned above dealing with people's stances on marijuana legalization, a nomothetic explanation may simply suggest that political orientation is the main driving force behind people's differing opinions on this issue. Hypotheses are not required in nomothetic research.

• • There are three main criteria for nomothetic causal relationships in social research:

1) the variables must be correlated
2) the variables are nonspurious
3) the cause takes place before the effect

• • Correlation - an empirical relationship between two variables such that changes in one are associated with changes in the other, or particular attributes in one are associated with particular attributes in the other.
    

• • Spurious relationship - a coincidental statistical correlation between two variables shown to be caused by some third variable. For example, increased ice cream consumption is related to the crime rate rise. But this relationship is caused by a third variable, summertime yielding hot weather and closed schools. Therefore, for a causal relationship, variables must be nonspurious
    

• • False criteria for nomothetic causality:
    
    • Complete causation - proper nomothetic explanation is probabilistic and does not explain every single case.
      
    • Exceptional cases - exceptions do not disprove nomothetic explanation.
      
    • Majority of cases - nomothetic explanation may be applicable to only a minority of cases in a given situation.
      

• • Necessary and Sufficient Causes:
    
    • A necessary cause represents a condition that must be present for the effect to follow. Example: It is necessary for you to take college courses in order to get a degree. Take away the courses, and the degree never follows.
      ***A sufficient cause represents a condition that guarantees the effect if it is present. Example: Skipping an exam would be a sufficient cause for failing it (even though there are other ways to fail it).

Units of Analysis

A unit of analysis is used to classify what or whom is being studied. The classifications include individuals, aggregates, and social artifacts.

• Individuals: In social science research, individuals are the most commonly studied.
  
• Aggregates: can be considered groups, organizations, and social interactions.
  
• Social artifacts: objects, such as paintings, articles, and diaries.
  
• Social interactions: interactions among individuals or aggregates.
  
  • Examples: school children(individuals); elementary schools(aggregate-groups); education (aggregate-organization); journal (artifact); class attendance (social interaction).
  • Particularly concerning groups, one can derive certain characteristics of a social group by observing the behaviors of individual members. (note: street gangs can imply all gangs/social groups and can be specified by city, sizes, locations, etc.) Organizations can be generalized too either by grouping organizations together or segregating a single organization by itself. In the context of corporations, and individual corporation can be studied via the employees (total employment, number of ethnic minority groups), gross assets, net annual profits, etc.

• Faulty Reasoning about Units of Analysis:
  • The ecological fallacy is the assumption that something learned about an ecological unit says something about the individuals making up that unit.

Example: If we found that suicide rates are higher in Protestant countries than in Catholic ones, we could not draw the conclusion that more Protestants commit suicide than Catholics; this would be an ecological fallacy.

• • Reductionism involves attempts to explain a particular phenomenon in terms of limited and/or lower-order concepts.
    

Example: For many social scientists, the field of sociobiology (social behavior can be explained solely in terms of genetic characteristics and behavior) is too limited and is an example of reductionism.

The Time Dimension

Cross Sectional Study: a study based on observations representing a single point in time; a cross section of a population. Example-The amount of people who registered to vote

Longitudinal Study: a study based on data that is collected at several different times. Example- The Tuskegee Experiment

• There are three types of longitudinal studies:
  
  • Trend Study: A type of longitudinal study in which a given characteristic of some population is monitored over time. Example: The series of Gallup Polls showing the electorate's preferences for political campaign, even though different samples were interviewed at each point
    
  • Cohort Study: A study in which some specific subpopulation, or cohort, is studied over time, although data may be collected from different members in each set of observations. Example: A study on the occupational history of the class of 1970 in which questionnaires were sent every five years
    
  • Panel Study: A type of longitudinal study, in which data are collected from the same set of people (the sample or panel) at several points in time

• Longitudinal studies do not always provide a feasible or practical means of studying processes that take place over time. Sometimes cross sectional-data can be used
  

-to imply processes over time on the basis of simple logic
-to make logical inferences whenever the time order of variables is clear
-ask individuals to report their past behaviors -cohort analysis to infer processes over time

How to Design a Research Project

Steps for designing a research project:
1) Define the purpose of your project (exploratory, descriptive, or explanatory?)
2) Specify the meaning of each concept being studied
3) Select a research method
4) Determine how you will measure the results
5) Determine the unit of analysis
6) Collect empirical data
7) Process the data
8) Analyze the data
9) Report your findings

The problem with a simple definition:
A real definition does not exist (fallacy of reification), as it mistakes our theoretical construct for a real entity.
A nominal (conceptual) definition is one that is simply assigned to a term without any claim that the definition represents a “real” entity.
An operational definition specifies precisely how a concept will be measured – that is, the operations we will perform.

The problem with an advanced definition:
Some conceptual accuracy is lost at every step along the way.
The meaning of measures are also highly contextual.

Conceptualization
Once you've decided on a purpose for your research and the type of research you will do (exploratory, descriptive, or explanatory) the next step in designing a research project is conceptualization- the mental process whereby fuzzy, imprecise and abstract notions (concepts) are made more specific and precise.
During this step, researchers specify definitions of concepts that will be used to examine a topic. For example, concepts like education, prejudice, and poverty need to be made more specific and precise before they can be used to understand a topic.

• Concepts have indicators and dimensions. An indicator is something the researcher has chosen to recognize as a reflection of a variable being studied. Example: if you're going to study how college students feel about abortion and why, the first thing you'll have to specify is what you mean by "the right to abortion" (because support for abortion often varies according to conditions). A dimension is a specifiable aspect of a concept. Example: dimensions of religiosity: belief, ritual, devotional, knowledge.

Operationalization
Operationalization is the development of specific research procedures that will result in empirical observations representing those concepts in the real world. Example: If you decided to use a survey to study attitudes about abortion rights, part of operationalization is determining the wording of questionnaire items. Some important questions to consider when doing operationalizations: How broad is the concept we want to study? How are we going to define (operationalize) or variables and attributes?

• Example of Conceptual vs Operational definitions: Weight
  
  • Conceptual definition: a measurement of gravitational force action on an object.
    
  • Operational definition: a result of measurement of the object on a Newton spring scale.
    

Choice of Research Method
Each research method has its strengths and weaknesses which need to be considered when choosing what is most appropriate for your study. Example: A survey might be the most appropriate method for studying attitudes towards abortion rights.

Population and Sampling
The population for a study is that group about whom we want to draw conclusions.
The sample is the group you select to be representative of that population. Example: For the abortion study, your population might be college students, and your sample might be 200 Pitt students.

Observations
The next step is to collect empirical data. Example: To conduct a survey on abortion, you might want to print questionnaires and mail them to a sample selected from the student body.

Data Processing
You next need to process your data so that it is interpretable. Example: Coding responses on the survey and transferring the information to a computer.

Analysis
The next step is to interpret the data for the purpose of drawing conclusions. Example: Calculate the percentages of students who favored or opposed each of the several different versions of abortion rights.

Application
Determine how your research and the conclusions you made can be used. Example: Prepare or publish a written report on your findings of abortion rights attitudes and discuss how they might apply to policy goals. Give suggestions for future research.

The Research Proposal:
It is often necessary to create an outline or layout of one's research plan, in the form of a "research proposal". This is beneficial to the researcher because it serves as an aid in planning. In addition, it makes it easy for others to understand and critique a researcher's ideas before they are carried out. Some common elements of a research proposal (and questions that it should answer) include:

• Problem/Objective: What are you planning to study, and why does it need to be studied?
• Literature Review: What previous research exists regarding this topic? What can you learn from existing research or theories pertaining to your topic? Will your study be able to improve or contribute to what already exists?
• Subjects for Study: Whom or what will you be studying, and how do you plan to get in touch with them? How will your research affect those whom you will be studying? Are you sure that your research will not be harmful to them? Is it ethical?
• Measurement: What are the key measurements (variables) pertaining to your study? How do you plan to define and measure them?
• Data-Collection Methods: How do you plan to collect data for your study? Will you use an experiment or survey, etc.?
• Analysis: What kind of analysis are you going to utilize? Are you planning to describe phenomena in detail, or will you attempt to explain the reasoning behind such phenomena?
• Schedule: What is a proposed timeline for the various stages in this project?
• Budget: Roughly, how much money do you estimate will be necessary for this project? Over the course of the project, where should the money be allocated?

The Ethics of Research Design
It is important to consider ethical conerns when you plan your research design, so that
-the subject's privacy is concerned
-the subject's well-being is protected
It may be appropriate and necessary for your design to be reviewed by an Institutional Review Board (IRB).

Measurement

Measurement: careful, deliberate observations of the real world for the purpose of describing objects and events in terms of the attributes composing the variable. Social scientists measure:

• Direct observables: physical characteristics (sex, height, skin color)of a person being observed and/or interviewed
  
• Indirect observables: characteristics of a person as indicated by answers given in a self-administered questionnaire (age, place of birth, education)
  
• Constructs: level of alienation, as measured by a scale that is created by combining several direct and/or indirect observables

Levels of Measurement - all measurements in science are conducted using 4 different types of scales:

• Nominal Level -variables with attributes of exhaustiveness and mutually exclusiveness. Examples: gender, religious affiliation, college major, hair color, birthplace, nationality
  
• Ordinal Level -variables with attributes we can logically rank in order. Examples: socioeconomic status, level of conflict, prejudice, conservativeness, hardness
  
• Interval Level - variables for which the actual distance between attributes has meaning. Examples: temperature (Fahrenheit), IQ score
  
• Ratio Level - variables whose attributes meet the requirements of an interval measurement and has a true zero point. Examples: age, length of time, number of organizations, number of groups
  
• Implications: Analyses require minimum levels of measurement. And some variables can be treated as multiple levels of measurement.

Measurement Quality:
While conducting an experiment or study, the quality of the measurements are very important.

• Precision - Precise measures are superior to imprecise ones. Precision is not the same as accuracy.
  
• Reliability - suggests that the same data would have been collected each time in repeated observations of the same phenomenon. There are different kinds of reliability: Stability, representative, and equivalence.
  
  • Stability reliability consists of remeasuring the data over and over again in hopes of getting the same result.
    
  • Representative reliability focuses on whether or not the data collected is the same when dealing with different "sub groups" in a certain population.
    
  • Equivalence reliability deals with multiple indicators such as questions or coders and focuses on whether or not these different indicators can yield the same results.
    
  • Ways to improve reliability:
    
    • Make sure that measures capture only the concept of interest
      
    • Increase the level (range) of measurement of the instrument
      
    • Make use of multiple indicators
      
    • Make use or pretests / pilot studies
      

• Validity -A term describing a measure that accurately reflects the concept it is intended to measure. There are four types of validity: face validity, criterion-related validity, construct validity, and content validity.
  
  • Face validity – the quality of an indicator that makes it a reasonable measure of some variable. It relies on the reader’s common sense to make a judgement.
    
  • Criterion-related validity – the degree to which a measure relates to some external criterion
    
  • Construct validity – asks whether the various measures for a given concept all seem to correspond to the same thing
    
  • Content validity – the degree to which a measure covers the concept it operationalizes

Problem: Validity and reliability can interfere with one another. Repeated measures should be taken to ensure the highest levels of both.