docBootstrap Distributions Or: How do we get a sense of a... distribution when we only have ONE sample?
download 
Report Transcription
Bootstrap Distributions Or: How do we get a sense of a... distribution when we only have ONE sample?
Bootstrap Distributions Or: How do we get a sense of a sampling distribution when we only have ONE sample? Suppose we have a random sample of 6 people: Bootstrap Sample: Sample with replacement from the original sample, using the same sample size. Original Sample Bootstrap Sample Original Sample Create a βsampling distributionβ using this as our simulated population Create a bootstrap sample by sampling with replacement from the original sample. Compute the relevant statistic for the bootstrap sample. Do this many times!! Gather the bootstrap statistics all together to form a bootstrap distribution. Original Sample Sample Statistic Bootstrap Sample Bootstrap Statistic Bootstrap Sample Bootstrap Statistic . . . Bootstrap Sample . . . Bootstrap Statistic Bootstrap Distribution Example: Atlanta Commutes Whatβs the mean commute time for workers in metropolitan Atlanta? Data: The American Housing Survey (AHS) collected data from Atlanta in 2004. Sample of n=500 Atlanta Commutes CommuteAtlanta Dot Plot n = 500 π₯ =29.11 minutes s = 20.72 minutes 20 40 60 80 100 120 140 160 Time Where might the βtrueβ ΞΌ be? 180 How can we get a confidence interval from a bootstrap distribution? Method #1: Use the standard deviation of the bootstrap statistics as a βyardstickβ Using the Bootstrap Distribution to Get a Confidence Interval β Version #1 The standard deviation of the bootstrap statistics estimates the standard error of the sample statistic. Quick interval estimate : ππππππππ ππ‘ππ‘ππ π‘ππ ± 2 β ππΈ For the mean Atlanta commute time: 29.11 ± 2 β 0.92 = 29.11 ± 1.84 = (27.27, 30.95) Using the Bootstrap Distribution to Get a Confidence Interval β Version #2 95% CI=(27.35,30.96) Chop 2.5% in each tail Keep 95% in middle Chop 2.5% in each tail For a 95% CI, find the 2.5%-tile and 97.5%-tile in the bootstrap distribution 90% CI for Mean Atlanta Commute 90% CI=(27.64,30.65) Chop 5% in each tail Keep 90% in middle Chop 5% in each tail For a 90% CI, find the 5%-tile and 95%-tile in the bootstrap distribution Bootstrap Confidence Intervals Version 1 (Statistic ο± 2 SE): Great preparation for moving to traditional methods Version 2 (Percentiles): Great at building understanding of confidence intervals Thatβs all folks StatKey can be found at www.lock5stat.com Playing with StatKey! See the purple pages in the folder. We want to collect some data from you. What should we ask you for our one quantitative question and our one categorical question? What quantitative data should we collect from you? A. What was the class size of the Intro Stat course you taught most recently? B. How many years have you been teaching Intro Stat? C. What was the travel time, in hours, for your trip to Boston for JMM? D. Including this one, how many times have you attended the January JMM? E. ??? What categorical data should we collect from you? A. B. C. D. E. Did you fly or drive to these meetings? Have you attended any previous JMM meetings? Have you ever attended a JSM meeting? ??? ??? How do we assess student understanding of these methods (even on in-class exams without computers)? See the green pages in the folder. Paul the Octopus http://www.youtube.com/watch?v=3ESGpRUMj9E http://www.cnn.com/2010/SPORT/football/07/08/germany.octopus.explainer/index.html Paul the Octopus β’ Paul the Octopus predicted 8 World Cup games, and predicted them all correctly β’ Is this evidence that Paul actually has psychic powers? β’ How unusual would this be if he were just randomly guessing (with a 50% chance of guessing correctly)? β’ How could we figure this out? Simulate! β’ Each coin flip = a guess between two teams β’ Heads = correct, Tails = incorrect β’ Flip a coin 8 times and count the number of heads. Remember this number! Did you get all 8 heads? (a) Yes (b) No Hypotheses Let p denote the proportion of games that Paul guesses correctly (of all games he may have predicted) H0 : p = 1/2 Ha : p > 1/2 Randomization Distribution β’ A randomization distribution is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis were true β’ A randomization distribution is created by simulating many samples, assuming H0 is true, and calculating the sample statistic each time Randomization Distribution β’ Letβs create a randomization distribution for Paul the Octopus! β’ On a piece of paper, set up an axis for a dotplot, going from 0 to 8 β’ Create a randomization distribution using each otherβs simulated statistics β’ For more simulations, we use StatKey p-value β’ The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true β’ This can be calculated directly from the randomization distribution! StatKey ο¨ 2ο© 1 8 ο½ 0.0039 Randomization Test β’ Create a randomization distribution by simulating assuming the null hypothesis is true β’ The p-value is the proportion of simulated statistics as extreme as the original sample statistic Coming Attractions - Friday β’ How do we create randomization distributions for other parameters? β’ How do we assess student understanding? β’ Connecting intervals and tests β’ Technology for using simulation methods β’ Experiences in the classroom Using Randomization Methods to Build Conceptual Understanding of Statistical Inference: Day 2 Lock, Lock, Lock, Lock, and Lock Minicourse- Joint Mathematics Meetings Boston, MA January 2012 Cocaine Addiction β’ In a randomized experiment on treating cocaine addiction, 48 people were randomly assigned to take either Desipramine (a new drug), or Lithium (an existing drug) β’ The outcome variable is whether or not a patient relapsed β’ Is Desipramine significantly better than Lithium at treating cocaine addiction? R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R 1. Randomly assign units to treatment groups Desipramine R R R R Lithium R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R 2. Conduct experiment 3. Observe relapse counts in each group R = Relapse N = No Relapse 1. Randomly assign units to treatment groups Desipramine Lithium R R R R R R pΛ new ο pΛ old R R R R R R R R R R N R N R R R R R R R N R N R N N N N R R R R R R N N N N N N 10 18 ο½ ο 24 24 ο½ ο.333 N N N N N N 10 relapse, 14 no relapse 18 relapse, 6 no relapse Randomization Test β’ Assume the null hypothesis is true β’ Simulate new randomizations β’ For each, calculate the statistic of interest β’ Find the proportion of these simulated statistics that are as extreme as your observed statistic R R R R R R R R R R R R R R R R N N R R R R R R N N N N N N R R R R R R N N N N N N N N N N N N 10 relapse, 14 no relapse 18 relapse, 6 no relapse R R R R R R R R R R R R R R R R N N R R R R R R N N N N N N R R R R R R N N N N N N N N N N N N Simulate another randomization Desipramine Lithium R N R N R R R R R R R N R R R N R N N N R R 16 relapse, 8 no relapse pΛ N ο pΛ O 16 12 ο½ ο 24 24 ο½ 0.167 N N N R N R R N N N N R N R R N R N R R R R 12 relapse, 12 no relapse Simulate another randomization Desipramine Lithium R R R R R R R N R R N N R R N R N R R N R N R R 17 relapse, 7 no relapse pΛ N ο pΛ O 17 11 ο½ ο 24 24 ο½ 0.250 R R R R R R R R R R R R R R R R R R N N N N N N 11 relapse, 13 no relapse Simulate! β’ Combine everyone into one group, and rerandomize them into the two groups β’ Compute your difference in proportions β’ Create the randomization distribution β’ How extreme is the observed statistic of -0.33? β’ Use StatKey for more simulations StatKey Proportion as extreme as observed statistic observed statistic The probability of getting results as extreme or more extreme than those observed if the null hypothesis is true, is about .02. p-value Cocaine Addiction You want to know what would happen β’ Why did you re-deal your cards? β’ by random chance (the random allocation to treatment groups) β’ Why did you leave the outcomes (relapse or no relapse) unchanged on each card? β’ if the null hypothesis is true (there is no difference between the drugs) How can we do a randomization test for a mean? Example: Mean Body Temperature Is the average body temperature really 98.6oF? H0:ΞΌ=98.6 Ha:ΞΌβ 98.6 Data: A random sample of n=50 body temperatures. Dot Plot BodyTemp50 n = 50 π₯ =98.26 s = 0.765 96 97 98 99 BodyTemp 100 Data from Allen Shoemaker, 1996 JSE data set article 101 Key idea: Generate samples that are (a) consistent with the null hypothesis (b) based on the sample data. How to simulate samples of body temperatures to be consistent with H0: ΞΌ=98.6? Randomization Samples How to simulate samples of body temperatures to be consistent with H0: ΞΌ=98.6? 1. Add 0.34 to each temperature in the sample (to get the mean up to 98.6). 2. Sample (with replacement) from the new data. 3. Find the mean for each sample (H0 is true). 4. See how many of the sample means are as extreme as the observed π₯ =98.26. Letβs try it on StatKey. How can we do a randomization test for a correlation? Is the number of penalties given to an NFL team positively correlated with the βmalevolenceβ of the teamβs uniforms? Ex: NFL uniform βmalevolenceβ vs. Penalty yards r = 0.430 n = 28 Is there evidence that the population correlation is positive? Key idea: Generate samples that are (a) consistent with the null hypothesis (b) based on the sample data. H0 : ο² = 0 r = 0.43, n = 28 How can we use the sample data, but ensure that the correlation is zero? Randomize one of the variables! Letβs look at StatKey. Playing with StatKey! See the orange pages in the folder. Choosing a Randomization Method Example: Word recall A=Sleep 14 18 11 13 18 17 21 9 16 17 14 15 mean=15.25 B=Caffeine 12 12 14 13 6 18 14 16 10 7 15 10 mean=12.25 H0: ΞΌA=ΞΌB vs. Ha: ΞΌAβ ΞΌB Reallocate Option 1: Randomly scramble the A and B labels and assign to the 24 word recalls. Resample Option 2: Combine the 24 values, then sample (with replacement) 12 values for Group A and 12 values for Group B. Question In Intro Stat, how critical is it for the method of randomization to reflect the way data were collected? A. Essential B. Relatively important C. Desirable, but not imperative D. Minimal importance E. Ignore the issue completely How do we assess student understanding of these methods (even on in-class exams without computers)? See the blue pages in the folder. Collecting More Data from You! Rock-Paper-Scissors (Roshambo) Play a game! Can we use statistics to help us win? Rock-Paper-Scissors Which did you throw? A). Rock B). Paper C). Scissors Rock-Paper-Scissors Are the three options thrown equally often on the first throw? In particular, is the proportion throwing Rock equal to 1/3? What about Traditional Methods? Intro Stat β Revised the Topics β’ β’ β’ β’ β’ β’ Data production (samples/experiments) Descriptive Statistics β one and two samples Bootstrap confidence intervals Randomization-based hypothesis tests Normal/sampling distributions Confidence intervals (means/proportions) β’ Hypothesis tests (means/proportions) β’ ANOVA for several means, Inference for regression, Chi-square tests Transitioning to Traditional Inference AFTER students have seen lots of bootstrap distributions and randomization distributionsβ¦ Students should be able to β’ Find, interpret, and understand a confidence interval β’ Find, interpret, and understand a p-value Bootstrap and Randomization Distributions Measures from Scrambled Collection 1 Measures from Scrambled RestaurantTips Slope :Restaurant tips -60 -40 -20 0 20 slope (thousandths) 40 Dot Plot Correlation: Malevolent uniforms Dot Plot 60 -0.6 -0.4 -0.2 0.0 r 0.2 0.4 All bell-shaped What do you Mean :Body Temperatures Diff means: Finger taps distributions! notice? Measures from Sample of BodyTemp50 98.2 98.3 98.4 Dot Plot Measures from Scrambled CaffeineTaps 98.5 98.6 Nullxbar 98.7 98.8 Proportion : Owners/dogs 0.4 0.5 phat 0.6 98.9 Dot Plot 99.0 -4 Measures from Sample of Collection 1 0.3 0.6 -3 -2 -1 0 Diff 1 2 3 4 Dot Plot Mean : Atlanta commutes Measures from Sample of CommuteAtlanta 0.7 0.8 26 27 28 29 xbar Dot Plot 30 31 32 The students are primed and ready to learn about the normal distribution! Transitioning to Traditional Inference β’ Introduce the normal distribution (and later t) β’ Introduce βshortcutsβ for estimating SE for proportions, means, differences, slopeβ¦ Confidence Interval: ππππππ ππ‘ππ‘ππ π‘ππ ± π§ β β ππΈ Hypothesis Test: ππππππ ππ‘ππ‘ππ π‘ππ β ππ’ππ πππππππ‘ππ ππΈ Confidence Intervals 95% -z* z* Hypothesis Tests 95% Area is p-value Test statistic Yes! Students see the general pattern and not just individual formulas! Confidence Interval: ππππππ ππ‘ππ‘ππ π‘ππ ± π§ β β ππΈ Hypothesis Test: ππππππ ππ‘ππ‘ππ π‘ππ β ππ’ππ πππππππ‘ππ ππΈ Connecting CIβs and Tests Measures from Sample of BodyTemp50 Dot Plot Randomization body temp means when ΞΌ=98.6 98.2 98.3 98.4 98.5 Measures from Sample of BodyTemp50 98.6 xbar 98.7 98.8 98.9 99.0 Dot Plot Bootstrap body temp means from the original sample 97.9 98.0 98.1 98.2 98.3 98.4 bootxbar 98.5 98.6 98.7 Whatβs the difference? Fathom Demo: Test & CI Sample mean is in the βrejection regionβ βΊ Null mean is outside the confidence interval Technology Sessions Choose Two! (The folder includes information on using Minitab, R, Excel, Fathom, Matlab, and SAS.) Student Preferences Which way did you prefer to learn inference (confidence intervals and hypothesis tests)? Bootstrapping and Randomization 39 67% Formulas and Theoretical Distributions 19 33% Student Preferences Which way do you prefer to do inference? Bootstrapping and Randomization 42 72% Formulas and Theoretical Distributions 16 28% Student Preferences Which way of doing inference gave you a better conceptual understanding of confidence intervals and hypothesis tests? Bootstrapping and Randomization 42 72% Formulas and Theoretical Distributions 16 27% Student Preferences LEARN inference AP Stat Simulation 13 Traditional 15 No AP Stat 26 4 DO inference Simulation Traditional AP Stat No AP Stat 18 24 10 6 UNDERSTAND Simulation Traditional AP Stat No AP Stat 17 25 11 5 Thank you for joining us! More information is available on www.lock5stat.com Feel free to contact any of us with any comments or questions.
