15: Tests of significance: the basics

Chapter 15
Tests of Significance:
The Basics
1/11/2017
Basics of Significance Testing
1
Significance Testing
• Also called “hypothesis testing”
• Objective: to test a claim about parameter
μ
• Procedure:
A.State hypotheses H0 and Ha
B.Calculate test statistic
C.Convert test statistic to P-value and interpret
D.Consider significance level (optional)
1/11/2017
Basics of Significance Testing
2
Hypotheses
• H0 (null hypothesis) claims “no difference”
• Ha (alternative hypothesis) contradicts the null
• Example: We test whether a population gained
weight on average…
H0: no average weight gain in population
Ha: H0 is wrong (i.e., “weight gain”)
• Next  collect data  quantify the extent to
which the data provides evidence against H0
Basic Biostat
9: Basics of Hypothesis Testing
3
One-Sample Test of Mean
• To test a single mean, the null hypothesis is
H0: μ = μ0, where μ0 represents the “null value”
(null value comes from the research question, not from
data!)
• The alternative hypothesis can take these forms:
Ha: μ > μ0 (one-sided to right) or
Ha: μ < μ0 (one-side to left) or
Ha: μ ≠ μ0 (two-sided)
• For the weight gain illustrative example:
H0: μ = 0
Ha: μ > 0 (one-sided) or Ha: μ ≠ μ0 (two-sided)
Note: μ0 = 0 in this example
1/11/2017
Basics of Significance Testing
4
Illustrative Example: Weight Gain
• Let X ≡ weight gain
• X ~N(μ, σ = 1), the
value of μ unknown
• Under H0, μ = 0
• Take SRS of n = 10
• σx-bar = 1 / √(10) =
0.316
• Thus, under H0
x-bar~N(0, 0.316)
1/11/2017
Figure: Two possible
xbars when H0 true
Basics of Significance Testing
5
One-Sample z Statistic
Take an SRS of size n from a Normal population.
Population σ is known. Test H0: μ = μ0 with:
x  μ0
z
σ
n
Equivalent ly z 
x  μ0
x
For “weight gain” data, x-bar = 1.02, n = 10, and σ = 1
x  μ0
1.02  0
z

 3.23
σ
1
10
n
1/11/2017
Basics of Significance Testing
6
P-value
• P-value ≡ the probability the test statistic would
take a value as extreme or more extreme than
observed test statistic, when H0 is true
• Smaller-and-smaller P-values → stronger-andstronger evidence against H0
• Conventions for interpretation
P > .10  evidence against H0 not significant
.05 < P ≤ .10  evidence marginally significant
.01 < P ≤ .05  evidence against H0 significant
P ≤ .01  evidence against H0 very significant
1/11/2017
Basics of Significance Testing
7
P-Value
Convert z statistics to P-value :
• For Ha: μ > μ0
P = Pr(Z > zstat) = right-tail beyond zstat
• For Ha: μ < μ0
P = Pr(Z < zstat) = left tail beyond zstat
• For Ha: μ μ0
P = 2 × one-tailed P-value
1/11/2017
Basics of Significance Testing
8
Illustrative Example
• z statistic =
3.23
• One-sided
P = P(Z > 3.23)
= 1−0.9994 =
0.0006
• Highly
significant
evidence
against H0
1/11/2017
Basics of Significance Testing
9
Significance Level
• α ≡ threshold for “significance”
• We set α
• For example, if we choose α = 0.05, we require
evidence so strong that it would occur no more
than 5% of the time when H0 is true
• Decision rule
P ≤ α  statistically significant evidence
P > α  nonsignificant evidence
• For example, if we set α = 0.01, a P-value of
0.0006 is considered significant
1/11/2017
Basics of Significance Testing
10
Summary
1/11/2017
Basics of Significance Testing
11
Illustrative Example: Two-sided test
1.
Hypotheses: H0: μ = 0 against Ha: μ ≠ 0
2.
Test Statistic:
z
x  μ0
σ

 3.23
1
n
3.
1.02  0
10
P-value: P = 2 × Pr(Z > 3.23) = 2 × 0.0006 =
0.0012
Conclude  highly significant evidence against
H0
1/11/2017
Basics of Significance Testing
12
Relation Between Tests and CIs
• For two-sided tests, significant results at the αlevel  μ0 will fall outside (1–α)100% CI
• When α = .05  (1–α)100% = (1–.05)100% =
95% confidence
• When α = .01, (1–α)100% = (1–.01)100% =
99% confidence
• Recall that we tested H0: μ = 0 and found a twosided P = 0.0012. Since this is significant at α =
.05, we expect “0” to fall outside that 95%
confidence interval … continued …
1/11/2017
Basics of Significance Testing
13
Relation Between Tests and CIs
Recall: xbar = 1.02, n = 10, σ = 1. Therefore, a 95%
CI for μ =
x z

σ
n
 1.02  1.96
1
10
 1.02  0.62
 0.40 to 1.64
Since 0 falls outside this 95% CI  the test of H0: μ
= 0 is significant at α = .05
1/11/2017
Basics of Significance Testing
14
Example II: Job Satisfaction
Does the job satisfaction of assembly workers differ when
their work is machine-paced rather than self-paced? A
matched pairs study was performed on a sample of
workers. Workers’ satisfaction was assessed in each
setting. The response variable is the difference in
satisfaction scores, self-paced minus machine-paced.
The null hypothesis “no average difference” in the
population of workers. The alternative hypothesis is “there
is an average difference in scores” in the population.
H0: m = 0
Ha: m ≠ 0
This is a two-sided test because we are interested in
differences in either direction.
1/11/2017
Basics of Significance Testing
15
Illustrative Example II



Job satisfaction scores follow a Normal
distribution with standard deviation  = 60.
Data from 18 workers gives a sample mean
difference score of 17.
Test H0: µ = 0 against Ha: µ ≠ 0 with
x  μ0
17  0
z

 1.20
σ
60
n
18
1/11/2017
Basics of Significance Testing
16
Illustrative Example II
• Two-sided P-value
= Pr(Z < -1.20 or Z > 1.20)
= 2 × Pr (Z > 1.20)
= (2)(0.1151) = 0.2302
• Conclude: 0.2302
chance we would see
results this extreme
when H0 is true 
evidence against H0 not
strong (not significant)
1/11/2017
Basics of Significance Testing
17
Example II: Conf Interval Method
Studying Job Satisfaction
A 90% CI for μ is
xz
 σ
n
 17  1.645
60
 17  23.26
18
 6.26 to 40.26
This 90% CI includes 0. Therefore, it is plausible
that the true value of m is 0  H0: µ = 0 cannot be
rejected at α = 0.10.
1/11/2017
Basics of Significance Testing
18