RESEARCH METHODS AND DATA ANALYSIS

Heuristics and Biases Reading

Representativeness
• Insensitivity to sample size
• Misconception of chance: deviations are not corrected as a chance process unfolds,
merely diluted
• Insensitivity to predictability
• Illusion of validity
• Misconception of regression

Availability
Cognitive shortcut that relies on what immediately comes to mind to make quick decisions
and judgments
• Biases due to retrievability of instances (familiarity)
• Biases due to effectiveness of a search set
• Biases of imaginability
• Illusory correlation

Adjustment and Anchoring

• Insufficient adjustment
• Biases in the evaluation of conjunctive and distinctive effects
• Anchoring in the assessment of subjective probability distributions
• Biases due to the retrievability of instances

Lazarsfeld Reading: The American Soldier

Hard to examine the world of social events:
More difficult to realize ideas of right and wrong vary in different cultures, people change
behaviour depending on group they are with
è Surveys only complicate behaviour that are already obvious

Obvious: every human reaction is convincible, important to know which reactions actually
occur most frequently and under what conditions

A) Primary groups shape attitudes of the individual

B) Primary groups provide emotional support (groups in battle showed a especially high
degree of loyalty and solidarity)
C) Emotional disturbances from loss of contact with the group

Hard to investigate effects of primary groups; 4 methods

• Study of people who have been brought into new situations

• Repeated interviews with the same persons
• Comparison. Of past and present attitudes
• Experimental procedures
Frame of reference

a) Selective perception: people see the world as they want to

b) Level of expectation: promotions boosted morale, higher expectations that
opportunities available. Greater average opportunity = greater average
disappointment
c) Relative deprivation: people don’t suffer in an absolute way. Compare their lot with
that of other people of their kind
d) Role of uncertainty: importance of a clear frame of reference

Role and position: army gave rise to rapid changes in position (change of roles) -> affected
outlook (change in responses to miliary affirmations depending on rank/role)

(Could be dependent on attitude: more conformist attitudes = more likely to be promoted

Differences in attitudes and misunderstanding and mistaken judgement: misjudgment of one

group by another

Cognitive Biases systematic errors in thinking that affect judgement and decision making

Positive Decision making/thinking is faster and more efficient

Negative Can lead to (serious) errors in judgement

Confirmation bias
The tendency to seek out, interpret, and create information that confirms our beliefs, and
minimize discount or ignore information that disconfirms our beliefs

What can it lead to?

1. Influence how we gather info / prevent objectivity
2. Influence how we interpret and recall information: remember things that support
your belief system

Self-fulfilling prophecy
A belief that leads to its own fulfilment
One’s expectations about ourselves/others lead us to behave in ways that confirm those
expectations

Beliefs (about ourselves) – Actions (towards others) – Others’ beliefs (about us) – Others’
actions (towards us)

è Assumptions influence behaviour and cause the predicted outcome

Pygmalion Effect: positive SFP
Golem effect: negative SFP
Belief Perseverance Error the tendency to retain our beliefs despite facing evidence to the
contrary
è Facilitates maintenance of stereotypes
è Often leads to more extreme evaluations
è Can also lead to prejudicial behaviour towards others
Confirmation bias results in belief perseverance error

Overconfidence Effect the tendency to be more confident than correct

è Overestimating the accuracy of one’s beliefs, predictions, and knowledge
è Mistaken judgements are typically recalled as times when one was “almost right”
è Cognitive bias leading to errors in prediction and judgement

Motivated Social Perceivers

è We seek to understand the causes of our own behaviour
è We seek to gain insight into the causes of the behaviour of others

Attribution theories how individuals perceive the information they receive, interpret events,
and how these form causal judgements

In Western individualistic cultures we tend to…

• Underestimate the role of situational factors (environment)
• Overestimate the role of dispositional factors (individual characteristics eg traits,
attitudes, internal states)

Why do we make attributions?

1. Explain the causes of behaviour and outcomes
2. Predict the future behaviour of others
3. Respond to social situations in an appropriate way

Biases within attribution theories

Fundamental Attribution Error tendency to emphasize dispositional factors to explain

someone else’s negative behaviour
è Tendency to attribute behaviour of others to dispositional factors
è Tendency to ignore situational determinants of others behaviour

Why does it occur?

People tend to automatically make dispositional attributions and then only later use
situational information to discount/correct their initial evaluation
è Corrections are more likely if contextual information is very compelling/salient
è We are especially likely to make dispositional attributions when events are
negative/unexpected

Cultural differences in FAE

Miller (1984)
Independent (internally focused): focus on personal attributions
Interdependent (externally focused, social roles): focus on situational attributions
è As people age, difference in attributions between cultures increases

Actor-Observer Bias
• Tendency to attribute other’s behaviour to dispositional factors
• Tendency to attribute own behaviour to situational factors

Actor: attention focused on the situation, hence focus on situational factors

Observers: attention focused on individual, hence focus on dispositional factors

Self-Serving Attributional Bias tendency to attribute our successes to internal traits and
talents & attribute our failures to external factors beyond our control

Self-enhancing bias: attributing success to personal traits provides us with a boost in

confidence
Self-protecting bias: Attributing failure to situational factors helps absolve us from personal
responsibility

Self-Serving Blindness: 64% doctors thought pharmaceutical gifts affected other doctors but
only 16% thought these gifts influenced their own actions

Hindsight Bias tendency to assume one knew all along that a given outcome was likely
Post-hoc explanation
One becomes convinced they accurately predicted an event before it occurred. It causes
overconfidence in one's ability to predict other future events and may lead to unnecessary
risks
è Easier to avoid hindsight bias if one has explicit casual models of the world, if many
factors could be relevant (inter-relatedness), it’s easier to imagine multiple outcomes

False Consensus Effect tendency to overestimate how much others agree with our beliefs,
attitudes, values and behaviour
è Can lead to people overvaluing the correctness of their own opinions

False Uniqueness Effect tendency to think that our attributes and traits are more uncommon
and rare than they actually are

Why does it occur?

Availability Heuristic
When estimating commonality, we tend to defer to examples that come to mind most
easily/quickly
• Our opinions are more accessible/salient
• People we spend the most time with have very similar beliefs and attitudes
(family/friends)
Self-Esteem
Believing most others are like us, allows us to feel normal and maintain a positive self-image,
avoids feelings of deviancy

What is science?
• Open skepticism
• Question authority
• Intellectual honesty

Science helps build explanations that are consistent and predictive vs conflicting and posthoc
(hindsight)
• Replication and generalizability
• Creating and testing theories

Scientific Inquiry
Facts
Objective: can be confirmed by others
Observation: can be direct or indirect

Hypothesis
Logic: well-reasoned predictions about events
Falsification: can be confirmed/disconfirmed

Theory
Framework: organized ideas to explain facts
Predictions: expectations of specific outcomes

Is Psychology a Science?
Psychology is the objective and systematic study of how people behave and think
Scientifically investigating the psychology of individuals and groups require the use of
systematic, empirical methods to test hypothesis and theories

The Scientific Method

1. Make an observation
2. Formulate a hypothesis
3. Background research
4. Design/conduct study
5. Analyze the data
6. Conclusion
7. Communication

Goals of Research: to better understand human behaviour

• Describe: identify and classify regularly occurring sequences of events, limited utility
• Explain: suggest why events occurred, understand at a deeper level
• Predict: identify regularities and predictable relationship that exists between variables
 • Control: regulate occurrence of a phenomena

Types of Research

Basic designed to increase understanding of fundamental psychological phenomena

Applied designed to provide solutions to real-world problems to improve society

Qualitative (non-numerical) narrative descriptions, content analyses, interviews/focus groups

Quantitative (numerical) manipulation/measurements, random assignment, control group

Empirical Questions can be answered by collecting data on a given phenomenon, use

precisely defined terms
Operational definitions specific procedures for manipulating or measuring a conceptual
variable
Converging operations understanding increases as studies with different operational
definitions converge on same result

Hypothesis formed based on empirical questions: specific, informed, testable predictions of

the outcome of the conditions in a research design
Theories formed after hypothesis are made & tested: a set of related assumptions from
which testable predictions can be made

Hypothesis – theory – more hypothesis – empirical research

1. Research confirms hypothesis: confidence increases
2. Research does not support hypothesis: confidence decreases, revise theory

Pseudoscience claims presented as scientific but lack several qualities of science

• Advances no true knowledge
• Disregards facts that contradict their claims
• Does not challenge its own assumptions
• Offers vague and incomplete explanations
• Unsound or absent logic

Validity

Content validity extent to which a measure represents all aspects of a construct

Criterion validity whether a test/measure can successfully predict some future behaviour
Construct validity the extent to which a measure/manipulation adequately
measures/manipulates what it was intended to measure/manipulate
Convergent validity whether two or more measures that theoretically should be related are
related (self-esteem & confidence)
Discriminant validity wither two or more measures that theoretically should not be related
are in fact not related (self-esteem and religiosity)
Internal validity whether we can be confident that changes in IV caused changes in DV
(Properly controlled)
External validity the extent to which results would also be founds for the same/other people
in other contexts, cultures, etc (generalized, natural & realistic)
Statistical validity the extent to which conclusions drawn from a statistical test are accurate
and reliable
• Different statistic required to answer different questions
• Adequate sample size (power) needed to properly test hypothesis
• Power: likelihood of detecting a real effect (testing causality)

Null hypothesis no change, DV will not change IV

Type 1 error false positive (reject null hypothesis when it is true) no dif but you say there is
Type 2 error false negative (failure to reject the null when it is false) dif bf but you say there’s
not

Importance of replication, science has no value without replication

Methodological: exact

Features of research design

1. A clear statement of research problem
2. Procedures for gathering information
3. Identify a population / select a sample
4. Variables to study
5. Statistical techniques to analyze data

à Effect appears, theorized effect can be obtained. Now replicate / Appears here, doesn’t
mean it will appear there (generalizability)
à Effect does not appear, does mean underlying phenomenon is not true

Descriptive Studies

Case studies: descriptive & exploratory analysis of a person, group or event that

Strengths
• Good way to generate hypothesis
• Provide data other methods can’t
• Provide illustrative anecdotes

Weaknesses
• Incomplete/ unrepresentative info
• Can only rely on self-report data
• Can be very subjective
• Cannot test theories/treatments

Survey research method used to collect information from a predetermined sample of people
è Gain insights into attitudes, beliefs, values and behaviors related to a particular topic

STRENGTHS WEAKNESSES
Good way to generate hypothesis Self-report data (misleading, subject to
biases)
 Provide info about many people (high Can be very subjective (biased
willingness to complete) wording/order effects) (extreme)

Fast and cost-effective (can be done online) Cannot test theories/hypothesis (can’t
establish cause & effect)

Naturalistic Observation observing and recording behaviour in naturally occurring situations

without trying to manipulate or control the situation

STRENGTHS WEAKNESSES
Good way to generate hypothesis (insight Some phenomena may be difficult to study
for future studies) unobtrusively

High ecological validity, provide info about Can be very subjective (behaviour open to
people in natural environment interpretation)

Cannot test theories/hypothesis (can’t

establish cause & effect)

Correlational measure two or more variables and their relationship to each other
• If x and y vary together, imply they are related

Correlation Coefficient a numerical representation of the strength and direction of the

relationship between 2 variables
Perfect positive +1 / Perfect negative -1
è Represented with a scatter plot

HOWEVER, correlation could be explained by a 3rd variable

What can you do?

• Rule out other variables: try to address 3rd variable problem (do this statistically by
including covariates)
• Measure both variables multiple times: try to address 3rd variable problem (look for
trends in relationship between all variables over time)
• Use cross-sectional design – helps control overlap of X and Y at time 1 or over time to
isolate the influence of x on y

Experimental Studies

Type of design that allows the most control over the experimental situation
Manipulation of variable (IV) and measurement of response (DV)

Features of experimental studies

Random assignment each ppt in the study has same chance of being in any of the conditions
- Eliminate researcher/selection bias
Experimental group ppts who will receive the treatment under investigation
Control group ppts who do not receive the treatment under investigation
Independent variable factor you manipulate to see if it affects DV
Dependent variable factor you measure to see if its affected by IV
Participant variable factor that characterizes pre-existing differences among ppts (self-
esteem, previous experience)

Threats to Experimental Studies

1. Confounding/extraneous variables: a factor other than IV that varies conditions in a
study – cannot be ruled out as potentially causing changes in DV

STRENGTHS WEAKNESSES
Good way to generate hypothesis (insight Artificial nature of experiments (lack
for future studies) ecological validity)

Conclusions about cause & effect can be Ethical & practical issues (issue of
made deception)

Ensuring Equal Groups

Placebo a substance or treatment that appears identical to the actual treatment but lacks the
active substance
Sometimes given to control group (medical trials – best current option given to control
group)

Issues in Experimental Design

Participant Expectancy Effects changes in behaviour that occur because the participant
believes that his/her behaviour should change

Solutions
1. Single Blind Study ppt doesn’t know which conditions they are in, hypothesis, what’s
being manipulated etc.
BUT researcher does and can manipulate who goes into which condition
2. Double Blind Study same but researcher didn’t influence who goes into groups and
how ppts respond, randomly assigned, researcher does not interfere

How do we know we succeed?

1. Include manipulation checks to evaluate (construct) validity
2. Consider how MC might affects things. Also use behavioral MC (manipulation checks)
Alternative explanations
Beta Problem valid hypothesis is not supported by data. Failure to explore alternative
hypothesis

So what should you do?

- Do more than just your manipulation check
- Try to rule out counter explanations (eg use measures of rival constructs)
- Use well-validated measures/manipulations to address validity issues (eg taken
existing research vs creating your own)

Meta Analysis a statistical technique for combining all published research results on one
question and drawing a conclusion

PERFORMANCE IN A SPORT

To what extent does education level and job experience affect income?

1. IV
Education Level
Condition 1: High (masters and/or PHD)
Condition 2: Low (undergrad)

Job Experience
Condition 1: high (specialized)
Condition 2: low (general)

2. Manipulation
Participants will be randomly assigned to one of the four conditions
- High Education Level/Specialized Job Training
- High Education Level/General Job Training,
- Low Education Level/Specialized Job Training,
- Low Education Level/General Job Training)

3. DV
Annual income measured by annual paycheck/ tax file

4. Measurement: Income will be measured in dollars. The scale will be continuous, and
the actual amount of income will be recorded.

5. Control Groups:

- Education Level: A control group will have a moderate education level.

- Control Group Manipulation: Participants in the control group will have an education
level that is neither very high nor very low.
- Training Program: A control group will follow a standard, industry-standard training
program.
- Control Group Manipulation: Participants in the control group will receive a standard,
industry-standard job training program without specialization.