Multilevel Modeling Break

Multilevel Modeling
1.
Overview
2.
Application #1:
Growth Modeling
Break
3.
Application # 2:
4.
Questions?
Individuals Nested
Within Groups
Overview
1.
2.
3.
4.
5.
6.
7.
8.
What is multilevel modeling?
Examples of multilevel data structures
Brief history
Current applications
Why multilevel modeling?
What types of studies use multilevel
modeling?
Computer Programs (HLM 6
SAS Mixed
Resources
Multilevel Question

What effects do the following variables
have on 3rd grade reading achievement?
School Size
Classroom Climate
Student Gender
What is Multilevel or Hierarchical
Linear Modeling?
Nested Data Structures
Several Types of Nesting

1. Individuals Nested Within Groups
Individuals Undivided
Unit of Analysis = Individuals
Individuals Nested Within
Groups
Unit of Analysis = Individuals + Classes
… and Further Nested
Unit of Analysis = Individuals + Classes + Schools
Examples of Multilevel Data
Structures



Neighborhoods are nested within
communities
Families are nested within
neighborhoods
Children are nested within families
Examples of Multilevel Data
Structures

Schools are nested within districts

Classes are nested within schools

Students are nested within classes
Multilevel Data Structures
Level 4 District (l)
Level 3 School (k)
Level 2 Class (j)
Level 1 Student (i)
2nd Type of Nesting

Repeated Measures Nested Within
Individuals
Focus = Change or Growth
Time Points Nested Within
Individuals
Repeated Measures Nested
Within Individuals
Carlos
Day
Monday = 0
Tuesday = 1
Wednes. = 2
Thursday = 3
Friday
=4
Energy Level
98
90
85
72
70
Repeated Measures Nested
Within Individuals
100
90
80
ENERG Y
70
60
0
DAY
1
2
3
4
5
Repeated Measures Nested
Within Individuals
100
90
80
ENERGY
70
60
Rsq = 0.9641
0
DAY
1
2
3
4
5
Changes for 5 Individuals
Changes in Energy Level Over the Week
100.00
Energy Level
75.00
50.00
25.00
0
0
1.00
2.00
Time
3.00
4.00
3rd Type of Nesting
(similar to the 2nd)

Repeated Measures Nested Within
Individuals
Focus is not on change
Focus in on relationships between
variables within an individual
Repeated Measures Nested
Within Individuals
Carlos
Day
Hours of Sleep Energy Level
Monday
9
98
Tuesday
8
90
Wednesday
8
85
Thursday
6
72
Friday
7
70
Repeated Measures Nested Within
Individuals (Not Change)
100
90
80
ENERGY
70
60
5.5
6.0
HOURS
6.5
7.0
7.5
8.0
8.5
9.0
9.5
Repeated Measures Nested Within
Individuals (Not Change)
100
90
80
ENERGY
70
60
5.5
6.0
HOURS
6.5
7.0
7.5
8.0
8.5
9.0
9.5
Repeated Measures Nested
Within Individuals
Repeated Measures Nested Within Individuals (3 Individuals)
100.00
Energy Level
75.00
50.00
25.00
0
2.00
4.50
7.00
Hours of Sleep
9.50
12.00
Repeated Measures Within
Persons
Level 2 Student (i)
Level 1 Repeated Measures
Over Time (t)
Nested Data


Data nested within a group tend to
be more alike than data from
individuals selected at random.
Nature of group dynamics will tend
to exert an effect on individuals.
Nested Data

Intraclass correlation (ICC) provides
a measure of the clustering and
dependence of the data
0 (very independent) to 1.0 (very
dependent)
Details discussed later
Brief History
of Multilevel Modeling
Robinson, W. S. (1950). Ecological
correlations and the behavior of
individuals. Sociological Review, 15, 351357.
Burstein, Leigh (1976). The use of data from
groups for inferences about individuals in
educational research. Doctoral
Dissertation, Stanford University.
Table 1
Frequency of HLM application evidenced in Scholarly Journals
Journal
1999
2000
2001
2002
2003
Total by journal
American Educational Research Journal
3
5
4
3
?
~15
Child Development
3
2
6
5
13
29
Cognition and Instruction
1
0
0
0
0
1
Contemporary Educational Psychology
0
0
0
0
0
0
Developmental Psychology
2
1
2
5
7
17
Educational Evaluation and Policy Analysis
2
1
5
2
2
12
Educational Technology, Research and Development
0
0
0
0
0
0
Journal of Applied Psychology
1
1
5
7
6
20
Journal of Counseling Psychology
0
2
1
0
0
3
Journal of Educational Computing Research
0
0
0
0
0
0
Journal of Educational Psychology
1
2
3
6
1
13
Journal of Educational Research
2
0
3
3
5
13
Journal of Experimental Child Psychology
0
0
0
0
0
0
Journal of Experimental Education
0
0
0
0
1
1
Journal of Personality and Social Psychology
4
4
6
5
13
32
Journal of Reading Behavior/Literacy Research
0
0
0
0
0
0
Journal of Research in Mathematics Education
0
0
0
0
0
0
Reading Research Quarterly
0
0
0
1
0
1
Sociology of Education
1
2
5
2
1
11
Total by Year
20
20
40
39
49
~168
Multilevel Articles
Frequency of Studies Employing HLM in Education or Related Journals
50
Total for 19 Journals Reviewed
Journal of Personality and Social Psychology
Child Development
Frequency
Journal of Educational Research
25
0
1999
2000
2001
Year
2002
2003
Some Current Applications
of Multilevel Modeling



Growth Curve Analysis
Value Added Modeling of Teacher
and School Effects
Meta-Analysis
Multilevel Modeling Seems
New But….
Extension of General Linear Modeling
Simple Linear Regression
Multiple Linear Regression
ANOVA
ANCOVA
Repeated Measures ANOVA
Multilevel Modeling

Our focus will be on observed
variables (not Latent Variables as in
Structural Equation Modeling)
Why Multilevel Modeling
vs.
Traditional Approaches?
Traditional Approaches – 1-Level
1.
2.
Individual level analysis (ignore
group)
Group level analysis (aggregate
data and ignore individuals)
Problems with
Traditional Approaches
1.
Individual level analysis (ignore group)
Violation of independence of data
assumption leading to misestimated
standard errors (standard errors are
smaller than they should be).
Problems with
Traditional Approaches
1.
Group level analysis
(aggregate data and ignore individuals)
Aggregation bias = the meaning of a
variable at Level-1 (e.g., individual level
SES) may not be the same as the
meaning at Level-2 (e.g., school level
SES)
Multilevel Approach


2 or more levels can be considered
simultaneously
Can analyze within- and betweengroup variability
What Types of Studies Use
Multilevel Modeling?
Quantitative
Experimental
*Nonexperimental
(Survey, Observational)
How Many Levels Are
Usually Examined?
2 or 3 levels very common
15 students x 10 classes x 10 schools
= 1,500
Types of Outcomes



Continuous Scale (Achievement,
Attitudes)
Binary (pass/fail)
Categorical with 3 + categories
Software to do Multilevel
Modeling
SPSS Users
2 SAV Files:
Level 1
Level 2
HLM 6 (Menu Driven)
(Raudenbush, Bryk, Cheong, & Congdon,
2004)
HLM 6
Software to do Multilevel
Modeling
SAS Users
Proc Mixed
Resources
(Sample…see handouts for more complete list)

Books

Hierarchical Linear Models: Applications and
Data Analysis Methods, 2nd ed.
Raudenbush & Bryk, 2002.

Introducing Multilevel Modeling.
Kreft & DeLeeum, 1998.

Journals
 Educational and Psychological Measurement
 Journal of Educational and Behavioral Sciences
 Multilevel Modeling Newsletter
Resources (cont)
(Sample…see handouts for more complete list)


Software
 HLM6
 SAS (NLMIXED and PROC MIXED)
 MLwiN
Journal Articles


See Handouts for various methodological and
applied articles
Data Sets
 NAEP Data
 NELS:88; High School and Beyond
Self-Check 1

A teacher with 1 classroom of 24
students used weekly curriculumbased measurements to monitor
reading over a 14 week period. The
teacher was interested in individual
students’ rates of change and
differences in change by male and
female students.
Self-Check 1

How would you classify this
situation?
(a) not multilevel
(b) 2-level
(c) 3-level
Self-Check 2

A researcher randomly selected 50
elementary schools and randomly
selected 30 teachers within each
school. The researcher was
interested in the relationships
between 2 predictors (school size
and teachers’ years experience at
their current school) and teachers’
job satisfaction.
Self-Check 2

How would you classify this
situation?
(a) not multilevel
(b) 2-level
(c) 3-level
Self-Check 3

60 undergraduates from the research participant
pool volunteered for a study that used written
vignettes to manipulate the interactional style
(warm, not warm) of a professor interacting with
a student. 30 randomly assigned students read
the vignette depicting warmth and 30 randomly
assigned students read the vignette depicting a
lack of warmth. After reading the vignette
students used a questionnaire to rate the
likeability of the professor.
Self-Check 3

How would you classify this
situation?
(Select ONLY one)
(a) not multilevel
(b) 2-level
(c) 3-level
Growth Curve Modeling


Studying the growth in reading
achievement over a two year period
Studying changes in student
attitudes over the middle school
years
Research Questions

What is the form of change for an
individual during the study?
Research Questions

What is an individual’s initial status
on the outcome of interest?
Research Questions

How much does an individual change
during the course of the study?
Rise
Run
b
Rise
Run
Research Questions

What is the average initial status of
the participants?
Research Questions

What is the average change of the
participants?
Research Questions

To what extent do participants vary
in their initial status?
Research Questions

To what extent do participants vary
in their growth?
Research Questions

To what extent does initial status
relate to growth?
Research Questions

To what extent is initial status
related to predictors of interest?
Research Questions

To what extent is growth related to
predictors of interest?
Design Issues

How many waves a data collection
are needed?


>2
Depends on complexity of growth curve
Design Issues

Can there be different numbers of
observations for different
participants?
Examples


Missing data
Planned missingness
Design Issues

Can the time between observations
vary from participant to participant?
Example: Students observed



1, 3, 5, & 7 months
1, 2, 4, & 8 months
2, 4, 6, & 8 months
Design Issues

How many participants are needed?



More is better
Power analyses
> 30 rule of thumb
Design Issues

How should participants be sampled?

What you have learned about sampling
still applies
Design Issues

What is the value of random
assignment?

What you have leaned about random
assignment still applies
Design Issues

How should the outcome be
measured?

What you have learned about
measurement still applies
Example

Context description
A researcher was interested in changes in
verbal fluency of 4th grade students,
and differences in the changes between
boys and girls.
ID
1
2
3
4
5
6
7
8
9
Gender
0
0
0
0
0
1
1
1
1
Time______
t0
t4
20
40
45
50
42
45
39
46
44
30
44
40
55
48
52
55
58
49
t7
30
49
60
59
53
61
63
68
59
Example

Level-1 model specification
Yfluency   0   1 * (Time)  error1
Example

Level-2 model specification
 0  G00  G01 * (Gender )  error2
 1  G10  G11 * (Gender )
Example

Combined Model
Yfluency  G00  G01 * (Gender )  G10 * (Time ) 
G11 * (Gender ) * (Time)  error2  error1
Example

SAS program
proc mixed covtest;
class gender;
model score = time gender time*gender/s;
random intercept / sub=student s;
Example

SAS output – variance estimates
Covariance Parameter Estimates
Cov Parm
Subject
Estimate
Standard
Error
Intercept
Residual
Student
62.5125
14.1173
35.9682
4.9912
Z
Value
Pr Z
1.74
2.83
0.0411
0.0023
Example

SAS output – fixed effects
Solution for Fixed Effects
Effect
Gender
Intercept
time
Gender
Gender
time*Gender
time*Gender
F
M
F
M
Estimate
39.8103
1.5077
5.7090
0
1.0692
0
Standard
Error
DF t Value
3.7975
0.3295
5.6962
.
0.4943
.
7
16
16
.
16
.
10.48
4.58
1.00
.
2.16
.
Pr > |t|
<.0001
0.0003
0.3311
.
0.0460
.
Example

Graph – fixed effects
100.00
GENDER = 0
GENDER = 1
SCORE
75.00
50.00
25.00
0
0
2.50
5.00
TIME
7.50
10.00
Example

Conclusions

Fourth grade girl’s verbal fluency is
increasing at a faster rate than boy’s.
Persons Nested in Contexts


Studying attitudes of teachers who
are nested in schools
Studying achievement for students
who are nested in classrooms that
are nested in schools
Research Questions

How much variation occurs within and
among groups?


To what extent do teacher attitudes vary
within schools?
To what extent does the average teacher
attitude vary among schools?
Research Questions

What is the relationship among selected
within group factors and an outcome?


To what extent do teacher attitudes vary
within schools as function of years
experience?
To what extent does student achievement vary
within schools as a function of SES?
Research Questions

What is the relationship among selected
between group factors and an outcome?


To what extent do teacher attitudes vary
across schools as function of principal
leadership style?
To what extent does student math
achievement vary across schools as a function
of the school adopted curriculum?
Research Questions

To what extent is the relationship among
selected within group factors and an
outcome moderated by a between group
factor?

To what extent does the within schools
relationship between student achievement and
SES depend on the school adopted
curriculum?
Design Issues

Consider a design where students are
nested in schools


How should schools should be sampled?
How should students be sampled within
schools?
Design Issues

Consider a design where students are
nested in schools


How many schools should be sampled?
How many students should be sampled per
school?
Design Issues

What kind of outcomes can be
considered?




Continuous
Binary
Count
Ordinal
Design Issues

How will level-1 variables be
conceptualized and measured?


SES
How will level-2 variables be
conceptualized and measured?

SES
Terminology



Individual growth trajectory – individual growth
curve model
 A model describing the change process for an
individual
Intercept
 Predicted value of an individual’s status at
some fixed point
 The intercept cold represent the status at the
beginning of a study
Slope
 The average amount of change in the
outcome for every 1 unit change in time
Intercept & Slope Illustration
25
20
Score
15
10
b
Rise
5
Rise
Run
Run
0
0
intercept
1
2
3
4
5
Time
6
7
8
9
10
Curvature =Acceleration=Quadratic Component
35
30
Score
25
20
15
10
5
0
0
1
2
Time
3
4
HLM

Hierarchical Linear Model


The hierarchical or nested structure of
the data
For growth curve models, the repeated
measures are nested within each
individual
Levels in Multilevel Models

Level 1 = time-series data nested
within an individual
Y  0  1 *(Time)  error
Levels in Multilevel Models

Level 2 = model that attempts to
explain the variation in the level 1
parameters
 0  G00  G01 * (Sessions)  error
 1  G10  G11 * (Sessions)  error
More terminology

Fixed coefficient


A regression coefficient that does not
vary across individuals
Random coefficient

A regression coefficient that does vary
across individuals
More terminology

Balanced design


Unbalanced design


Unequal number of observation per unit
Unconditional model


Equal number of observations per unit
Simplest level 2 model; no predictors of the
level 1 parameters (e.g., intercept and slope)
Conditional model

Level 2 model contains predictors of level 1
parameters
Estimation Methods

Empirical Bayes (EB) estimate

“optimal composite of an estimate
based on the data from that individual
and an estimate based on data from
other similar individuals” (Bryk,
Raudenbush, & Condon, 1994, p.4)
Estimation Methods

Expectation-maximization (EM)
algorithm

An iterative numerical algorithm for
producing maximum likelihood
estimates of variance covariance
components for unbalanced data.