5: Measurement, scaling & sampling

Question 1

Measurement & scaling: defs + why?

2 reasons

Answer 1

measure = assign numbers

scale = create a continuum upon which measured objects are located

Reasons:

1. to apply statistics
2. to communicate more clearly

Question 2

4 (5) types of scales, in order from lower to higher level of measurement

Answer 2

1. nominal scale = mutually exclusive and exhaustive categories
2. ordinal scale = numbers expressing ranking

(2. 5) rating scale = like preference ranking… it is in between, and really an ordinal scale, but often interpreted as interval –> to do statistics
3. interval scale = numbers with intervals being equal
4. ratio scale = interval scale with an absolute zero –> you know ratios between ratios, e.g. Kelvin VS Celsius degrees

Question 3

6 steps of measuring: 3+2+1

Answer 3

before:

1. Identify the concept of interest
   * (can be simple like gender or complex like intelligence)*
2. Develop a construct
3. Define the concept constitutively
   * (the central idea under study)*

​during:

1. Define the concept operationally
   * (operational definition what to measure)*
2. Develop a measurement scale

after:

1. Evaluate the reliability and validity of measurement

Question 4

effect of using brackets, e.g. for age,
on the measurement level

Answer 4

–> moving down from ratio scale to ordinal scale

…it is better to ask for the age number instead!

Question 5

scaling technique 3-level taxonomy (for rating scales)

Answer 5

• comparative
  
  • pair comparison
  • rank order
  • constant sum
• non-comparative
  
  • itemized rating scale
    
    • semantic differential = 7-point scale of bipolar labels, eg hot VS cold, careful VS careless
    • Stapel = 10-point unipolar scale (on attribute), from -5 to +5 with no neutral zero rating
    • Likert (‘li:kert) = degree of agreement from 1=strongly disagree to 5/7=strongly agree
  • continuous rating scale

Question 6

when to ask w comparative scales? 3 reasons

Answer 6

• when you need a decision, eg on preferences
• when you are interested in the effects of small differenes
• to avoid carry-over effects (respond as above) and halo effects

Question 7

how to design a scale?
4=2+2 issues to consider

Answer 7

2 item Nr-related factors:

• how many items? odd (w neutral) or even (–> take sides)?
  
  • suggestion: do not force to take sides, offer a neutral or ‘do not know’ answer
  • relatedly: do not force answers, esp. on sensitive Qs where ppl might not want to answer, like income; rather, request –> it is softer
• constant number of scale categories: do not switch from 5- to 7-point scale mid-survey, coz it confuses participants and makes your analysis more difficult to standardize

2 scale type-related factors:

• switch bw scale types instead, while keeping the same Nr of options, to avoid boredom & carry-over or halo effects
• tradeoff bw single-item or multi-item scales: multi-items means several similar Qs to measure all facets more precisely, but it is fatiguing for participants

Question 8

scale evaluation: is our measurement good?

2 criteria

2 types of errors

Answer 8

2 criteria:

• validity –> am I measuring the intended construct?
• reliability –> repeatability

NB: validity presupposes reliability.

2 error types:

• random errors = noise
• systematic errors

Question 9

sampling:

key issue

goal

2 methods to achieve it

Answer 9

key issue –> are we correctly representing the correct target population?

goal –> external validity

methods to achieve it –> correct sample selection & sample size

Question 10

4 basic concepts in sampling + 2 errors

Answer 10

population –> sample frame

sample = set of (sample units)

sampling error = units chosen outside, or otherwise not representatively of the target population

sample frame error = differences bw population and sample frame

Question 11

sample size tradeoff

Answer 11

the larger,
the more accurate, but also the more expensive