4: Experimental research Flashcards Preview

ETH1: 3.3 EMM > 4: Experimental research > Flashcards

Flashcards in 4: Experimental research Deck (13)
Loading flashcards...

Experimental research - goal:

establish causal relationships


Types of research Qs:

  • descriptive; eg. polls are just descriptive
  • relational > correlational relationships
  • causal > causal relationships


spurious relationships =

correlation confused with causation


3 necessary but not sufficient conditions to infer causality b/w X & Y:

+ problems

+ difficult point in research design

>>> overarching epistemologic problem

  • co-variation = X & Y happen together
  • time order: X before Y
  • exclude alternative possible causes

+ reverse causation: Y actually causes X, or Z causes both

+ isolate potential cause from other factors

>>> causality can be inferred, but never proven 100%


3 steps to evaluate a treatment with an experiment:

+ 4 entities involved


  • manipulate one or more IVIndependent Variables, eg who gets the treatment (in most basic design) in your TU Test Unit = population sample being tested
  • measure the effect on the DV = Dependent Variable(s)
  • control for the effect of EV = Extraneous Variables

+ IV, DV, EV & TU


Taxonomy of experimental designs

Experimental designs

  • Quasi-experiment w/o randomization (of sample assignment)
    • field
    • combined
    • lab
  • (True) Experiment w randomization (of sample assignment)
    • field
    • combined
    • lab


tradeoff of field VS lab experiments 

realism --> generalizability = external validity


control & ease of implementation --> internal validity




goal n how

of assignment to one or the other condition;

the goal is to achieve internal validity because probabilistic assignment tends to produce similar populations


8=1+3+4 Threats to internal validity from extraneous variables

+ 1 important countermeasures

1 before exp:

  • selection bias = non-random assignments of treatments, eg doctors giving them to the neediest

3 experiment-related:

  • socially desirable behavior (= wanting to look good) and/or demand effects (= giving researchers what they want)
    >>> need to include measures of social desirability, coz this is usually the biggest problem in social R!
  • instrumentation = changes in instruments, observers (eg changing confederates in the exp.) or scores themselves
  • testing effects = behavior changes due to test

4 time-related:

  • history = specific events that happen at the same time
  • maturation = changes influencing test units w time; 
    similar to history, but more vague
  • mortality = loss of test units durign experiment
  • regression to the mean = probabilistic issue of test units w extreme scores moving closer to avg during



>>> randomization helps against these effects


4 ways to control for extraneous variables

  • randomization <<< best way, but not always possible
  • matching = knowing the domain >>> find comparable pairs among test units >>> reduces sample size
  • statistical control = measure & analyse confounding factor
  • design control = add another experimental condition to manipulate


2 truly experimental designs

>>> possible problems

  • pretest-posttest control group >>> testing effects cannot be excluded
  • posttest-only control group >>> different populations at outset cannot be excluded


not truly experimental designs (in a table):

w differences wrt true exps

+ usually...

without control group:

  • One-shot case study >>> no Control Group, no randomization
  • One-group pretest-posttest design >>> no Control Group, no randomization

with control group:

  • Static group design >>> no randomization

  • Nonequivalent-groups (pretest-posttest) design >>> maybe multiple diffs bw experimental & control group

+ usually these are natural settings


Statistical experimental designs:

typical property, =

factorial design w several groups covering all the combinations of manipulated variables