BIVARIATE CATEGORICAL DATA bivariate data

BIVARIATE CATEGORICAL DATA
Recall from section 2.1 that we learned how to represent a categorical variable numerically in a
table and graphically in a barplot. Suppose we have bivariate data, data from two variables,
which we’d like to analyze.
Tables
Suppose we are doing a technology survey of STAT 200 students. Specifically, who owns a cell
phone, and who owns a laptop? Fill in the data in this two-way contingency table.
Owns Laptop
Yes
No
Owns
Cell
Phone
Yes
No
This table demonstrates the full distribution of data at our disposal. But if we shift our focus to
only one of the variables independent of how the other behaves, that is known as a marginal
distribution. For a marginal distribution, we simply sum the values of the rows (or columns) for
variable of interest.
Owns Laptop
Yes
No
Owns
Cell
Phone
Yes
Marginal Distribution
for Cell Phone
No
Marginal Distribution for Laptop
At other times, we are interested in how a variable behaves given certain information regarding
another variable. This is known as the conditional distribution because we are studying a
variable conditioned on the behavior of another. Let’s look at the conditional distribution of
laptop ownership. That is, the distribution of laptops, conditioned on whether the subject owns a
cell phone.
Owns Laptop
Yes
No
Owns
Cell
Phone
Yes
No
Now you construct the conditional distribution of cell phone ownership, given whether the
subject owns a laptop.
Owns Laptop
Yes
No
Owns
Cell
Phone
Yes
No
Barplots
Since we are dealing with categorical variables, the barplot will be our most reliable form of
graphing. First, we choose one of the variables to be represented on the x-axis, and we list the
factors of that variable on the x-axis. Following that we have two options.
•
•
Segmented Barplot – This plot has one main bar displaying the total frequency for each
factor on the x-axis. The frequencies of the second variable conditioned on the first are
segmented in that one bar.
Side-by-side Barplot – This plot has one for each factor of the second variable above
each factor on the x-axis. The frequencies of the second variable conditioned on the first
are each drawn in their own bar and displayed side-by-side.
Construct a segmented barplot and a side-by-side barplot for the technology survey results.
Examples in R
Be sure to review 3.1.1 in the text regarding the rbind, cbind, and matrix functions to enter
your own table data. But here, we’ll use the UScereal (MASS) dataset which contains
information about cereals on a shelf in a grocery store. Load the dataset and attach the variables.
>
Make a frequency table displaying the relationship between the mfr and shelf variables and
store it in the variable name x.
>
Are there any obvious differences between manufacturers? This may be difficult to tell in the
above table, so let’s use the margin.table function to look at the marginal distribution of the
row variable mfr. You can also see both marginal distributions using addmargins(x).
> margin.table(x, 1)
How would you look at the marginal distribution of the column variable shelf using
margin.table? Anything worth noting there?
>
We might be interested in making sure that the cereal companies are all getting a fair shake, and
that the brands are evenly distributed among the shelves. We must look at the conditional
distributions to help analyze that. Use the prop.table function.
> prop.table(x)
# What does this do?
Now, look at the conditional distribution of the shelf given the row variable, mfr. For each of
the given manufacturers, what is the proportion of its brands that appear on the lowest shelf?
> prop.table(x, 1)
#
G
K
N
P
Q
R
Now, look at the conditional distribution of the manufacturer variable given the column variable,
shelf number. What do you observe?
> prop.table(x, 2)
COMPARING INDEPENDENT SAMPLES
We will formalize our definition of independence later. But for now, you may think of two
samples as being independent if they are taken in such a way that knowing the distribution of
one sample doesn’t affect our knowledge of the other.
Though the samples are independent, it is still possible that share similar features. For example,
consider the ACT scores at two distinct high schools. Do the have a similar center? Similar
spread? Similar shape? All are possible, but only answerable through statistical analysis.
In our recurring example, we’ll look at the dataset twins (UsingR) containing the IQ scores for
27 pairs of identical twins separated near birth. Are these samples independent?
Stem-and-leaf plots can be drawn side by side on the same “stem” to note general distinctions.
R does not have a built-in function for this.
Histograms are perhaps the most useful means of displaying univariate data, but it becomes
difficult to create and read histograms comparing two variables. Recall, that the y-axis of a
histogram measures density. We can create densityplots to outline the behavior of a histogram.
What do you observe in the twins data?
> plot(density(Biological))
> lines(density(Foster), lty=2)
Boxplots also lend themselves well to displaying two variables graphically and side-by-side.
Does this graph support your observations from the densityplots?
> boxplot(Biological, Foster, horizontal=T)
A quantile-quantile plot displays the quantiles of one dataset against the same quantiles of the
other as points. If the distributions have a similar shape, then the quantile values will be similar
at each proportion, and thus the qq-plot will be roughly a straight line. If the shapes are
different, the points will not be linear.
Before looking at the qq-plot for the twins, do you expect the points to generally fall into a line
or not?
> qqplot(Biological, Foster)
SCATTERPLOTS
A scatterplot is a graph used for showing the relationship between two variables. One variable is
assigned to the x-axis and the other is assigned to the y-axis. The convention is to call the x
variable the independent variable and the y variable the dependent variable. Usually the
independent variable is thought to influence the dependent variable. The corresponding function
in R is simply plot().
Example 1: Construct a scatter plot for the Exam 1 and Exam 2 scores of 7 students:
Exam 1 Exam 2
55
62
60
50
70
65
80
70
85
95
90
80
100
90
100
90
Exam 2
Student
1
2
3
4
5
6
7
80
70
60
50
50
60
70
80
Exam 1
90
100
There is a positive association between x and y when the pattern of the points slopes upward.
As x increases, y tends to increase.
There is a negative association between x and y when the pattern of the points slopes downward.
As x increases, y tends to decrease.
Is there a positive or negative association between exam 1 and exam 2 in the example above?
What about in the twins (UsingR) dataset?
> plot(Biological, Foster)
For the following pairs of variables state whether you think the association is positive, negative,
or neither.
• Height and weight
• Weight of a car and how many miles per gallon it gets
• Years of Education and Income
• Height and GPA among college students
• Temperature in Fahrenheit and temperature in Celsius.
• # right and # wrong on a test
CORRELATION
The correlation coefficient measures the strength of the linear association between two
variables X and Y. It measures how tightly points are clustered around a line, but does not
measure clustering around a curve.
The sample correlation coefficient is defined by
Cov(X, Y)
∑ ( xi − x )( yi − y ) = 1  ( xi − x )  ( yi − y ) 
r = cor(X, Y) =
=
∑

Var(X)Var(Y)
∑ ( xi − x ) 2 ∑ ( yi − y ) 2 n − 1  s x  s y 
The corresponding function in R is cor( ). If any of your variables has missing values, use the
command cor(x,y,use=’complete.obs’).
The correlation coefficient is always between –1 and 1.
The closer the points hug a line with a positive slope, the closer r is to +1. The closer the points
hug a line with a negative slope the closer r is to -1. A correlation of 1 or -1 means you can
perfectly predict one variable knowing the other. If there is no association between X and Y then
the correlation coefficient is near 0, and the scatterplot has no ascertainable pattern.
Look at the variables at the bottom of the last page. For which pair does r = 1? r = -1? r = 0?
The correlation coefficient has no units and the same under location-scale transforms of X
and Y.
• Adding a constant to all x or y values does NOT change r.
• Multiplying all x or y values by a positive constant does NOT change r. What does
multiplying by a negative constant do?
• Interchanging all x and y values does not change r. (The correlation between height and
weight is the same as the correlation between weight and height)
• Changing units does NOT change r. (So correlation between height in inches and weight
in lbs. is the same as between height in meters and weight in kgs.)
Outlier Issue
Including outliers may ruin the correlation even though the rest of the points show a strong linear
relation. But outliers should only be thrown out for good reason. The correlation coefficient
should be used with caution when there are outliers.
Correlation is Not Causation
An association may be found between two variables, but that does not mean one causes the other.
• Shoe size and reading level are highly correlated among elementary school kids. Does
that mean that big feet help students learn to read?
• Surveys have shown that people who use artificial sweeteners tend to be heavier than
people who use sugar. Does that mean that sweeteners cause weight gain?
Example: Try to match the plot with the correlation coefficient to which it is nearest. The
choices for r are -1, -0.90, -0.50, 0, 0.50, 0.90, and 1.
B)
C)
D)
G)
H)
0
1
2
3
4
5
6
A)
E)
F)
For some extra practice, go to http://www.stat.uiuc.edu/~stat100/cuwu/Games.html, and click on
the “Correlations” link to play the Guessing Correlations game.
Computing the Correlation Coefficient

x −x
 Std. Units = z = i

1. Convert x-values and y-values to standard units.
s
x


2. Multiply each zx value by each corresponding zy value.
3. The correlation coefficient is the “adjusted” average (divide by n-1) of the products.
Calculate the correlation coefficient for scores on two quizzes for 5 students.
Quiz 1 (x)
Quiz 2 (y)
1
4
2
5
4
9
2
5
6
7
x = 3, s x = 2
y = 6, s y = 2
Zx
Correlation coefficient (r) = ________________
Zy
Sum of products =
Zx * Zy
SIMPLE LINEAR REGRESSION
Recall the scatterplot from the twins (UsingR) dataset and note several important features.
70
80
90
Foster
100
110
120
130
IQ Scores for Separately Raised Identical Twins
70
80
90
100
110
120
130
Biological
•
•
•
•
•
•
Two variables are associated with each observation/subject.
There appears to be a rough trend or relationship between the two variables.
This relationship is not exactly precise in that there exists substantial variation or scatter.
We may want to summarize this relationship with a linear equation with slope and
intercept terms.
We may want to characterize the variability about this linear equation.
We may want to know whether or not any apparent trend is statistically significant.
Linear regression is a statistical method of assessing these features. A simple linear
regression model is a summary of the relationship between a dependent variable (a.k.a.
response) Y and a single independent variable (a.k.a. predictor or covariate) X. Y is assumed to
be a random variable while, even if X is a random variable, we condition on X (that is, assume X
is fixed). Essentially, we are interested in knowing the behavior of Y given we know X.
Given X, the linear regression model assumption is simply that Y = β 0 + β1 X + ε , where e is a
random variable with mean 0. This line is the population regression line while β 0 and β1 are the
population regression coefficients. When we estimate β 0 and β1 based on a sample of X and Y
pairs, their estimates, βˆ and βˆ , are called estimated regression coefficients or just regression
0
1
coefficients.
Once estimates βˆ0 and βˆ1 of β 0 and β1 have been computed, the predicted value of yi given xi is
obtained from the estimated regression line: yˆ = βˆ + βˆ x where yˆ is the prediction of the true
i
value of yi , for observation i, i = 1, …, n.
0
1 i
i
Finding the Regression Line
There are several methods to finding a reliable regression line, but perhaps the most common is
the least squares method. This method chooses the line (and thus the coefficients) so that the
sum of the squared residuals is as small as possible. When this is true, the regression coefficients
are found by these equations:
βˆ1 = ∑
( xi − x )( yi − y )
∑ ( xi − x )2
=r∗
sy
sx
, and βˆ0 = y − βˆ1 x .
Let’s look more closely at why we can use the correlation coefficient to estimate the y value for a
given x value. If there is a perfect correlation (that is, r = 1 or r = -1) then we can perfectly
predict y from x.
r=1
r = .8
r = .5
r = -.3
If x goes up 1 SD from average, then y goes up 1 SD from average. All points lie
on the SD line.
If x increases 1 SD, then y increases only .8 SD on the average.
If x increases 1 SD, then y increases only .5 SD on the average.
If x increases 1 SD, then y decreases only .3 SD on the average.
The regression line estimates the mean value for the dependent variable (y) corresponding to
each value of the independent variable (x).
1 SD increase in x means only an r SD increase in y.
sy
So, the slope of the regression line is βˆ1 = r * .
sx
The following points are always true for simple linear regression:
• The least squares regression line goes through the center of the point cloud.
• The regression line passes through the intersection of x and y .
• On average, the distance between yˆ and y is shorter than the distance between y and y.
Example: A large class took two exams with the following results:
AvgExam1 = 80,
AvgExam2 = 70,
SDExam1 = 10
SDExam2 = 15
r = .6
Find the regression equation for predicting Exam 2 score using Exam 1 score.
Residuals (Error)
The regression line estimates the average value of y for each value of x. But unless the
correlation is perfect, the actual y values differ from the predicted values. These differences are
called prediction errors or residuals.
Residual = Actual value – Predicted value
(ei = εˆi = yi − yˆi )
For any regression line, the average (and the sum) of the residuals is zero.
2
1
0
-3
-2
-1
Residual
0
-3
-2
-1
y
1
2
3
Plot of Residuals
3
Scatterplot with Regression Line
-3
-2
-1
0
1
2
3
-3
-2
x
-1
0
1
2
3
x
Calculating “spread” for regression
The standard deviation of the residuals, also called the residual standard error or Root Mean
Square Error (RMSE), is a measure of the typical spread of the data around the regression line.
Rather than finding all the residuals, then finding their variance, then finding their standard
deviation, it's much easier to use this formula:
SDresiduals = se = 1 − r 2 * s y
If r = ±1, we can perfectly predict y from x, so there is no prediction error.
If r = 0, then our best prediction is the average of y and the likely size of the prediction error is
just the SD of y.
Example: The HANES height-weight study on parents and children gives the following
summary statistics for men aged 18-24:
Height
Weight
Avg
70 inches
162 pounds
SD
3 inches
30 pounds
Correlation: r = 0.5
Predicting weight from height
a)
Find the regression line of interest.
b)
What is the residual standard error for predicting weight from height?
c)
Predict the weight for someone who is 73" tall.
d)
This prediction is likely to be off by _________lbs. or so.
Predicting height from weight
e)
Find the regression line of interest.
f)
What is the residual standard error for predicting height from weight?
g)
Predict the height of someone who weighs 132 lbs.
i)
This prediction is likely to be off by _______inches or so.
j)
What is the residual for someone who is 132 lbs. but only 65”?
Example in R
Let’s look at those identical twins again. Suppose we want to regress the variable Foster onto the
variable Biological for the purposes of prediction.
>
>
>
>
library(UsingR)
attach(twins)
plot(Foster~Biological)
cor(Biological, Foster)
# previously we used plot(Biological, Foster)
Do the plot and correlation coefficient support a simple linear regression?
The linear model function in R is simply lm(y~x) where y is your dependent variable and x is
your independent variable. What is the regression equation for Foster given Biological?
> lm(Foster~Biological)
We can store this regression, and see that there is actually much more in the model results. (The
text often stores linear models as res which is too much like “residual” and thus deceiving.)
What other things does the lm( ) function generate?
> model1 = lm(Foster~Biological)
> names(model1)
Remember that before we sell ourselves on this model, we want to look at the plots once more.
First, the original data scatterplot with the regression line.
> par(mfrow=c(1,2))
> plot(Foster~Biological, main=’Biological vs Foster raised twins’)
> abline(model1)
Second, the fitted and residual values. What two things are we looking for here? Does this plot
support the linear regression model?
> res = model1$residuals; fit = model1$fitted
> plot(res~fit, xlab=’Fitted values’, ylab=’Residuals’,
main=’Fitted vs Residual values’)
> abline(h=0, lty=2)
Now that we have the linear regression equation, maybe we’d like to try some prediction. Say
that a child raised by his Biological parents has an IQ of 128. What is the predicted IQ of his
identical twin raised by Foster parents? How much are we likely to be off by?
> predict(model1, data.frame(Biological=128))
> sd(res)
Example in R
Let’s look at another dataset. Load the data for kid.weights (UsingR), and then attach the
variables. Answer each question and write down any R commands you used.
1) Look at the scatterplot with height on the x-axis and weight on the y-axis. Compute the
correlation coefficient of height and weight. What does this exploratory analysis tell you about
the validity of a linear regression?
2) Find the simple linear regression equation of weight on height and save it as model.h.
3) The shortest child has a height of 12 inches and weight of 10 pounds. What is his predicted
weight? What do you think about the predicted value? What does it say about the regression
equation?
4) Look at the scatterplot with fitted values on the x-axis and residuals on the y-axis. Add a line
at y=0. What does this residual plot tell you about the validity of a linear regression?
5) Look at the scatterplot with age on the x-axis and weight on the y-axis. Find the simple linear
regression equation of weight on height and save it as model.a.
6) Compute the correlation coefficient of age and weight, and the residual standard error of
weight on age. Does age or height seem to be a better linear predictor of weight?
7) Look at the scatterplot with fitted values on the x-axis and residuals on the y-axis. Add a line
at y=0. What does this residual plot tell you about the validity of a linear regression?