Testing for differences

Question 1

What are some parametric tests?

Answer 1

Paired t-test
two-sample test
one-sample test

Question 2

What are some non-parametric tests?

Answer 2

Mann whitney U
wicoxon
one-sample sign test

Question 3

What is the quantifying difference?

Answer 3

Cohen’s d

Question 4

what is the purpose of parametric T-tests?

Answer 4

testing for differences
hypotheses testing
can be one or two tailed tests

Question 5

What are the assumptions of the T-tests?

Answer 5

continuous data
follows a normal distribution
variances are constant (standard deviations)

Question 6

What happens in the assumptions of a T-test dont hold?

Answer 6

the statistical test is in doubt and the p-value may be wrong
Robust test can withstand some deviation
try to transform the data
Use a non-parametric test instead

Question 7

When would you use the one sample t-test?

Answer 7

used to determine if mean of a sample is difference from an expected value

Question 8

When would you use a paired t-test?

Answer 8

test for differences between two sets of paired observations

Question 9

When would you used the two-sample t-test?

Answer 9

test whether the means of two independent sets of measurements are different

Question 10

What are the assumptions of the paired t-test?

Answer 10

check that appropriate t-test assumptions are met.

Question 11

What is the null hypothesis of the paired t-test?

Answer 11

The mean change or difference is zero

Question 12

What is the rationale of the paired t-test?

Answer 12

calculate the difference between each pair of points, then determine if the mean of these values is different from zero

Question 13

What is the method of using the paired t-test?

Answer 13

– Calculate the difference (d) between each pair of measurements
– Calculate the mean difference and standard error – Calculate the test statistic (|t|) and compare to the critical value

Question 14

How do you interpret results compared to the critical value for the paired t-test?

Answer 14

– If |t| ≥ critical value, the null hypothesis is rejected, concluding the mean difference is “significantly” different from zero
– If |t| < critical value, there is not enough evidence to reject the null hypothesis, concluding the mean difference is not significantly different from zero

Question 15

What are the assumptions of the two-sample t-test?

Answer 15

check appropriate t-test assumptions are met

samples are independent of each other

Question 16

what is the null hypothesis to the two-sample t-test?

Answer 16

the difference between the two means is zero

Question 17

What is the rationale with the two-sample t-test?

Answer 17

it compares the means from two independent samples, if the samples are not independent then the test will not be valid.

Question 18

what is the null hypothesis for the one-sample t-test?

Answer 18

There is no difference between the sample population mean and an expected (given) value

Question 19

What is the rationale for the one-sample t-test?

Answer 19

The test calculates how many standard errors the sample mean is away from the expected value (E)

Question 20

How do you interpret the results for the one sample t-test?

Answer 20

The further away the mean is from the expected value, the larger the value of t, and the less probable it is
• The value of t can be positive or negative, it is the absolute value |t| of the difference that counts
• If |t| >critical value, then the difference is statistically significant

Question 21

What is Cohen’s d looking at?

Answer 21

an effect size measurement that tells us the magnitude of an experimental treatment.

Question 22

how do you calculate Cohen’s d?

Answer 22

𝑑 = (𝑀𝑔𝑟𝑜𝑢𝑝1−𝑀𝑔𝑟𝑜𝑢𝑝2) / 𝑆𝐷𝑝𝑜𝑜𝑙𝑒d

Question 23

What are the results of cohens d?

Answer 23

d = 1, then groups’ means differ by 1 SD  
d = 0.5, the means differ by half an SD  
d = 2, means differ by 2 SDs

Question 24

How do you interpret cohens d?

Answer 24

d <0.2, difference is minor, even if significant
d = 0.2, considered a “small” effect size
d = 0.5, considered a “medium” effect size
d = 0.8, considered a “large” effect size

Question 25

What is absolute Cohen’s d value and what does it mean?

Answer 25

• Absolute Cohen’s d value = 1.3409 • Means differing by 1.3 standard deviations
• Indicates effect size is very large
• Concluded that difference in pH between control and treatment groups is very large and consistent enough to be very important

Question 26

When would you use non-parametric tests?

Answer 26

• Used when underlying assumptions required for parametric tests don’t apply
• OR if data is not the correct type (i.e. categorical ordinal data, etc.)
• No assumptions are made about the underlying distributions of the sampled populations

Question 27

What are the advantages of the non-parametric tests?

Answer 27

– Allow hypothesis testing where structure of population sampled is unknown
– Generally quicker and easier to use
– Can analyse data that consists of ranks (i.e. ordinal scale data) or classifications (i.e. nominal scale data)

Question 28

What are the disadvantages of the non-parametric tests?

Answer 28

– Wastes data if you could apply a parametric method instead
– Not sensitive so important effects can be missed
– Some methods are laborious for large samples

Question 29

What is the Mann-Whitney U test?

Answer 29

• Unpaired t-test equivalent
• Ranks of measurements used, not the measurements themselves
• Data ranked either from lowest to highest, or highest to lowest
• One of the most powerful non-parametric tests
• Can perform one- or two-tailed tests

Question 30

What is the Wilcoxon Test?

Answer 30

• Paired t-test equivalent
• If the d values are not from a Normal distribution then the Wilcoxon test is applicable
• Procedure involves calculating the differences, d, as does the paired t-test
• Uses ranks of the differences

Question 31

What is the one sample sign t-test?

Answer 31

• One sample t-test equivalent
• Uses ranks and not the actual measured values
• Compares against an expected value, E
• This is a special case of the Wilcoxon paired test but substitutes the expected value (E) for one of the two samples