Pathway to Proficiency: Linking the STAR Reading

March 30, 2016
Pathway to Proficiency: Linking the
STAR Reading™ and STAR Math™ Scales with
Performance Levels on the Georgia Milestones
Quick reference guide to the STAR Assessments™
STAR Reading™—used for screening and progress-monitoring assessment—is a reliable, valid,
and efficient computer-adaptive assessment of general reading achievement and
comprehension for grades 1–12. STAR Reading provides nationally norm-referenced reading
scores and criterion-referenced scores. A STAR Reading assessment can be completed without
teacher assistance in about 10 minutes and repeated as often as weekly for progress
monitoring.
STAR Math™—used for screening, progress-monitoring, and diagnostic assessment—is a
reliable, valid, and efficient computer-adaptive assessment of general math achievement for
grades 1–12. STAR Math provides nationally norm-referenced math scores and criterionreferenced evaluations of skill levels. A STAR Math assessment can be completed without
teacher assistance in less than 15 minutes and repeated as often as weekly for progress
monitoring.
STAR Reading and STAR Math received the highest possible ratings for screening and progress monitoring by
the National Center on Response to Intervention, are highly rated for progress monitoring by the National
Center on Intensive Intervention, and meet all criteria for scientifically based progress-monitoring tools set
by the National Center on Student Progress Monitoring.
© 2016 by Renaissance Learning, Inc. All rights reserved. Printed in the United States of America. All logos, designs, and
brand names for Renaissance Learning’s products and services, including but not limited to Renaissance Learning,
Renaissance Place, STAR, STAR Assessments, STAR Math, and STAR Reading are trademarks of Renaissance Learning,
Inc., and its subsidiaries, registered, common law, or pending registration in the United States and other countries. All
other product and company names should be considered the property of their respective companies and organizations.
This publication is protected by U.S. and international copyright laws. It is unlawful to duplicate or reproduce any
copyrighted material without authorization from the copyright holder. For more information, contact:
RENAISSANCE LEARNING
P.O. Box 8036
Wisconsin Rapids, WI 54495-8036
(800) 338-4204
www.renaissance.com
Page 2 of 22
Introduction
Educators face many challenges; chief among them is making decisions regarding how to allocate limited resources to
best serve diverse student needs. A good assessment system supports teachers by providing timely, relevant information
that can help address key questions such as which students are on track to meet important performance standards? And
which students are not on track and thus need additional help? Different educational assessments serve different
purposes, but those that can identify students early in the school year as being at-risk to miss academic standards can
be especially useful because they can help inform instructional decisions that can improve student performance and
reduce gaps in achievement. Assessments that can do that while taking little time away from instruction are particularly
valuable.
Indicating which students are on track to meet later expectations is one of the potential capabilities of a category of
educational assessments called “interim” (Perie, Marian, Gong, & Wurtzel, 2007). They are one of three broad categories
of assessment:
1.
Summative – typically annual tests that evaluate the extent to which students have met a set of standards. Most
common are state-mandated tests such as the Georgia Milestones assessments.
2.
Formative – short and frequent processes embedded in the instructional program that support learning by
providing feedback on student performance and identifying specific things students know and can do as well as
gaps in their knowledge.
3.
Interim – assessments that fall in between formative and summative in terms of their duration and frequency.
Some interim tests can serve one or more purposes, including informing instruction, evaluating curriculum and
student responsiveness to intervention, and forecasting likely performance on a high-stakes summative test later
in the year.
This study focuses on the application of interim test results, notably their power to inform educators about which
students are on track to succeed on the year-end summative state test and which students might need additional
assistance to reach proficiency. Specifically, it involves linking the Georgia Milestones performance levels for ELA and
Mathematics with scales from two interim tests, STAR Reading and STAR Math. The Georgia Milestones assessments are
used statewide as summative assessments in Georgia. The STAR tests are the most widely used assessments in the U.S.,
are computer adaptive and use item response theory, require very little time (on average, less than 10–15 minutes group
administration time), and may be given repeatedly throughout the school year.1
An outcome of this study is that educators using the STAR Reading Enterprise or STAR Math Enterprise assessments can
access the Reading Dashboard and STAR Performance Reports (see Sample Reports, pp. 12–16) that indicate whether
individual students or groups of students (by class, grade, or demographic characteristics) are on track to meet
proficiency as measured by the Georgia Milestones. The dashboard and reports allow instructors to evaluate student
progress toward proficiency and make instructional decisions based on data well in advance of annual state tests. They
also help educators screen for later difficulties and progress monitor students’ responsiveness to interventions.
1
For an overview of the STAR tests and how they work, please see the References section for a link to download The research foundation for STAR
Assessments report. For additional information, full technical manuals are available for each STAR assessment by contacting Renaissance Learning at
[email protected]
Page 3 of 22
Sources of data
STAR Reading and STAR Math data were gathered from schools that use those assessments on the Renaissance Place
hosted platform2. Performance-level distributions from the Georgia Milestones English Language Arts (ELA) End-ofGrade assessments (GA Milestones ELA) and the GA Milestones Mathematics End-of-Grade assessments (GA Milestones
Mathematics) were retrieved from Georgia Milestones school summary data available on the Georgia Department of
Education website.
The Georgia Milestones use four performance levels: Level
1: Beginning Learning, Level 2: Developing Learner, Level 3:
Proficient Learner, Level 4: Distinguished Learner. Students
scoring in the Level 3 or Level 4 categories would be
counted as meeting proficiency standards for state and
federal performance-level reporting.
This study uses STAR Reading, STAR Math, and Georgia
Milestones end-of-grade data from the 2014–15 school
year.
Georgia Milestones Performance Levels:
Level 1: Beginning Learner
Level 2: Developing Learner
Level 3: Proficient Learner
Level 4: Distinguished Learner
Methodology
Many of the ways to link scores between two tests require that the scores from each test be available at a student level.
Obtaining a sufficient sample of student-level data can be a lengthy and difficult process. However, there is an
alternative technique that produces similar results without requiring us to know each individual student’s Georgia
Milestones score and STAR scaled score. The alternative involves using school-level data to determine the STAR scaled
scores that correspond to each Georgia Milestones performance level cutscore. School level Georgia Milestones end-ofgrade data are publically available, allowing us to streamline the linking process and complete linking studies more
rapidly.
The STAR scores used in this analysis were “projected” scaled scores using STAR Reading and STAR Math decile-based
growth norms. The growth norms are both grade- and subject-specific and are based on the growth patterns of more
than one million students using STAR Assessments over a three-year period. They provide typical growth rates for
students based on their starting STAR test score, making predictions much more accurate than a “one-size-fits-all”
growth rate.
For each observed score, the number of weeks between the STAR test administration date and the middle of the Georgia
Milestones window was calculated. Then, the number of weeks between the two dates was multiplied by the student’s
expected weekly scaled score growth (from our decile-based growth norms, which take into account grade and starting
observed score). Expected growth was added to the observed scaled score to determine each student’s projected STAR
score. For students with multiple STAR tests within a school year, the average of their projected scores was used.
This method used to link our STAR scale to Georgia Milestones proficiency levels is equivalent groups equipercentile
equating. This method looks at the distribution of Georgia Milestones performance levels in the sample and compares
that to the distribution of projected STAR scores for the sample; the STAR scaled score that cuts off the same percentage
2
Renaissance Place is a service that involves “hosting” schools’ data from the STAR tests and other products. For more information about Renaissance
Place, see http://www.renaissance.com/products/renaissance-place
Page 4 of 22
of students as each Georgia Milestones performance level is taken to be the cutscore for each respective proficiency
level.
For several different states, we compared the results from the equivalent groups equipercentile equating to results from
student-level data and found the accuracy of the two methods to be nearly identical (Renaissance Learning, 2015a,
2015b). McLaughlin and Bandeira de Mello (2002) employed a similar method in their comparison of NAEP scores and
state assessment results, and this method has been used multiple times since 2002 (Bandeira de Mello, Blankenship, &
McLaughlin, 2009; McLaughlin & Bandeira de Mello, 2003; McLaughlin & Bandeira de Mello, 2006; McLaughlin, Bandeira
de Mello, Blankenship, Chaney, Esra, Hikawa, Rojas, William, & Wolman, 2008). Additionally, Cronin et al. (2007) found
this method was able to determine performance level cutscore estimates very similar to the cutscores generated by
statistical methods requiring student-level data.
Sample selection
To find a sample of students who were assessed by both the Georgia Milestones and STAR, we began by gathering all
hosted STAR Reading and STAR Math test records for the year 2014-2015. Then, each school’s STAR Reading and STAR
Math data were aggregated by grade and subject area. The next step was to match STAR data with the Georgia
Milestones end-of-grade data. To do this, performance level distribution data from the Georgia Milestones was obtained
from the Georgia Department of Education website. The file included the total number of students tested in each grade
and the percentage of students in each performance level. The STAR Reading and STAR Math data were matched to the
Georgia Milestones end-of-grade ELA and mathematics data by system and school name.
Once we determined how many students in each grade at a school were tested on the Georgia Milestones for English
Language Arts and took a STAR Reading assessment, we calculated the percentage of enrolled students assessed on both
tests. Then we repeated this exercise for the mathematics assessments. In each grade at each school, if between 95%
and 105% of the students who tested on the Georgia Milestones had taken a STAR assessment, that grade was included
in the sample. The process was conducted separately for the English Language Arts and mathematics assessments. This
method of sample selection ensured that our sample consisted of schools in which all or nearly all of the enrolled
students who took the Georgia Milestones also took STAR within the specified window of time. If a total of approximately
1,000 or more students per grade met the sample criteria, that grade’s sample was considered sufficiently large for
analysis.
Demographic information for the schools in our sample was available from the student enrollment by grade level report
available on the Georgia Department of Education website. We aggregated this data to the grade level to create Table 1
on the following page and Table 3 on page 7.
Page 5 of 22
Sample description: Reading
A total of 354 unique schools across grades 3 through 8 met the sample requirements (explained previously in Sample
Selection). Racial/ethnic characteristics for each grade of the sample are presented along with statewide averages in
Table 1. White students were slightly over-represented and Black students were slightly under-represented in the
reading sample.
Table 2 displays by-grade test summaries for the reading sample. It includes counts of students taking STAR Reading and
the Georgia Milestones ELA. It also includes percentages of students in each performance level, both for the sample and
statewide. Students in the reading sample had similar performance to students statewide.
Table 1. Characteristics of reading sample: Racial/ethnic statistics
Percent of students by racial/ethnic category
Grade
Number of
Schools
3
Native
American
Asian & Pac.
Islander
Hispanic
Black
White
Multiple
Races
114
0.2%
3.1%
13.4%
30.2%
49.6%
3.5%
4
142
0.2%
3.4%
13.6%
31.1%
48.4%
3.4%
5
175
0.2%
3.4%
12.8%
30.9%
49.4%
3.4%
6
50
0.2%
3.5%
10.6%
33.2%
49.2%
3.3%
7
69
0.2%
3.3%
10.6%
33.8%
48.9%
3.3%
8
53
0.2%
4.2%
9.6%
34.2%
48.9%
3.0%
Statewide
0.0%
4.0%
14.0%
37.0%
42.0%
3.0%
Table 2. Characteristics of reading sample: Performance on STAR Reading ™ and Georgia Milestones ELA
Level 1:
Level 2:
Level 3:
Level 4:
Students
Students
Beginning
Developing
Proficient
Distinguished
taking GA
Grade taking STAR
Learner
Learner
Learner
Learner
Milestones
Reading™
ELA
Sample State Sample
State Sample State Sample
State
3
12,658
12,374
25%
33%
29%
30%
31%
27%
15%
10%
4
14,885
14,494
23%
29%
33%
34%
32%
28%
12%
9%
5
18,222
17,804
22%
27%
31%
34%
36%
31%
11%
8%
6
10,161
9,964
26%
31%
29%
30%
35%
31%
10%
8%
7
15,796
15,481
27%
31%
32%
33%
33%
30%
8%
6%
8
12,446
12,267
22%
24%
33%
37%
34%
31%
11%
8%
Page 6 of 22
Sample description: Math
Georgia Milestones mathematics grade counts were matched to counts of students who took STAR Math the same year.
A total of 206 unique schools across grades 3 through 8 met the sample requirements for math (explained in Sample
selection). Racial/ethnic characteristics for each of the math samples were calculated in the same manner as for reading
and are presented along with statewide averages in Table 3. White students were slightly over-represented and Black
students were slightly under-represented in the math sample.
Table 4 displays by-grade test summaries for the math sample. It includes counts of students taking STAR Math and a
Georgia Milestones Mathematics assessment. It also includes percentages of students in each performance level, both
for the sample and statewide. Students in the math sample had similar Georgia Milestones Mathematics performance to
the statewide population.
Table 3. Characteristics of math sample: Racial/ethnic statistics
Percent of students by racial/ethnic category
Grade
Number of Schools
Native
American
Asian &
Pac.
Islander
Hispanic
Black
White
Multiple Races
3
85
0.2%
3.4%
13.8%
30.6%
48.5%
3.5%
4
77
0.2%
3.6%
14.3%
31.4%
47.2%
3.3%
5
103
0.2%
3.7%
12.8%
30.4%
49.5%
3.4%
6
33
0.2%
5.6%
13.1%
29.8%
48.1%
3.2%
7
35
0.2%
4.2%
10.6%
33.0%
49.0%
3.1%
8
34
0.2%
4.6%
10.9%
33.1%
48.4%
2.9%
Statewide
0.0%
4.0%
14.0%
37.0%
42.0%
3.0%
Table 4. Characteristics of math sample: Performance on STAR Math™ and Georgia Milestones Mathematics
Level 1:
Level 2:
Level 3:
Level 4:
Students
Students taking
Beginning
Developing
Proficient
Distinguished
taking
Grade
GA Milestones
Learner
Learner
Learner
Learner
STAR
Mathematics
Math™
Sample State Sample State Sample State Sample State
3
9,466
9,271
16%
21%
38%
41%
34%
30%
12%
8%
4
7,828
7,641
17%
20%
35%
40%
34%
31%
14%
9%
5
11,179
10,911
20%
25%
34%
37%
30%
27%
16%
11%
6
7,288
7,110
21%
26%
33%
39%
30%
26%
16%
9%
7
8,999
8,789
19%
25%
34%
38%
28%
25%
19%
12%
8
8,227
8,080
19%
25%
35%
38%
27%
25%
19%
12%
Page 7 of 22
Analysis
First, we aggregated the sample of schools for each subject to grade level. Next, we calculated the percentage of
students scoring in each Georgia Milestones performance level for each grade. Finally, we ordered STAR scores and
analyzed the distribution to determine the scaled score at the same percentile as the Georgia Milestones proficiency
level. For example, in our third grade reading sample, 25% of students were at Level 1: Beginning Learner, 29% at Level 2:
Developing Learner, 31% at Level 3: Proficient Learner, and 15% at Level 4: Distinguished Learner. Therefore, the cutscores
for achievement levels in the third grade are at the 25th percentile for Level 2: Developing Learner, the 54th percentile for
Level 3: Proficient Learner, and the 85th percentile for Level 4: Distinguished Learner.
Figure 1. Illustration of linked STAR Reading™ and Georgia Milestones ELA scales for 3rd grade.
STAR Reading
1400
15%
GA Milestones ELA
830
15% Level 4
581
525
475
618
31% Level 3
29% Level 2
31%
469
29%
25% Level 1
180
349
25%
0
Page 8 of 22
Results and reporting
Table 5 presents estimates of equivalent scores on STAR Reading and Georgia Milestones ELA. Table 6 presents
estimates of equivalent scores on STAR Math and Georgia Milestones Mathematics. These results will be incorporated
into the Renaissance Learning’s Reading Dashboard and STAR Performance Reports (see Sample dashboard and reports,
pp. 12–16) that can be used to help educators determine early and periodically which students are on track to reach the
Proficient Learner status or higher in order to make instructional decisions accordingly.
Table 5. Estimated STAR Reading™ cut scores for Georgia Milestones ELA performance levels
Grade
Level 1: Beginning
Learner
Level 2: Developing
Learner
Level 3: Proficient
Learner
Level 4: Distinguished
Learner
3
< 349
349
469
618
4
< 416
416
564
797
5
< 479
479
667
960
6
< 544
544
743
1087
7
< 582
582
843
1231
8
< 615
615
908
1274
Table 6. Estimated STAR Math™ cut scores for Georgia Milestones Mathematics performance levels
Grade
Level 1: Beginning
Learner
Level 2: Developing
Learner
Level 3: Proficient
Learner
Level 4: Distinguished
Learner
3
< 542
542
634
707
4
< 604
604
701
779
5
< 675
675
768
832
6
< 706
706
800
867
7
< 716
716
820
880
8
< 743
743
843
897
Page 9 of 22
Accuracy analyses
Following the linking study summarized earlier in this report, 3 Georgia school districts shared student level 2015
Georgia Milestones scores in order to explore the accuracy of using STAR Reading and STAR Math for forecasting Georgia
Milestones performance.
Classification diagnostics were derived from counts of correct and incorrect classifications that could be made when
using STAR scores to predict whether or not a student would achieve proficiency on the GA Milestones.3 The results are
summarized in the following tables and figures, and indicate that the statistical linkages provide an effective means of
estimating end-of-year achievement on the GA Milestones based on STAR scores obtained earlier in the year.
Table 7. Fourfold schema of proficiency classification diagnostic data
GA Milestones Result
STAR
Estimate
Proficient
Not
Total
Total
Proficient
Not
True Positive
(TP)
False Negative
(FN)
Observed Proficient
(TP + FN)
False Positive
(FP)
True Negative
(TN)
Observed Not
(FP + TN)
Projected Proficient
(TP + FP)
Projected Not
(FN + TN)
N = TP+FP+FN+TN
Table 8a. Proficiency classification frequencies: ELA
Classification
Grade
3
4
5
6
7
8
True Positive (TP)
170
201
132
90
80
99
True Negative (TN)
448
424
500
318
325
293
False Positive (FP)
83
62
71
21
28
38
False Negative (FN)
33
31
44
37
22
31
Total (N)
734
718
747
466
455
461
Table 8b. Proficiency classification frequencies: Mathematics
Classification
Grade
3
4
5
6
7
8
True Positive (TP)
250
220
235
80
90
89
True Negative (TN)
444
406
491
338
309
309
False Positive (FP)
77
68
65
32
37
26
False Negative (FN)
43
27
54
21
17
39
Total (N)
814
721
845
471
453
463
3
Classifications were based upon STAR scores projected to April 15, 2015, the midpoint of the 2014-2015 GA Milestones end-of-grade main
administration window (March 30, 2015 – May 1, 2015).
Page 10 of 22
Table 9a. Accuracy of projected STAR™ scores for predicting GA Milestones proficiency: ELA
Measure
Formula
Interpretation
Grade
3
4
5
6
7
8
Percentage of correct classifications
84%
87%
85%
88%
89%
85%
Percentage of proficient students identified as such using STAR
84%
87%
75%
71%
78%
76%
Percentage of not proficient students identified as such using
STAR
84%
87%
88%
94%
92%
89%
Percentage of students STAR finds proficient who actually are
proficient
67%
76%
65%
81%
74%
72%
Percentage of students STAR finds not proficient who actually
are not proficient
93%
93%
92%
90%
94%
90%
Percentage of students who achieve proficient
28%
32%
24%
27%
22%
28%
Percentage of students STAR finds proficient
34%
37%
27%
24%
24%
30%
Difference between projected and observed proficiency rates
6%
5%
3%
-3%
2%
2%
Area Under the ROC Curve
Summary measure of diagnostic accuracy
0.93
0.95
0.91
0.93
0.94
0.92
Correlation (r)
Relationship between STAR and GA Milestones ELA scores
0.84
0.85
0.81
0.83
0.84
0.84
Overall classification
accuracy
Sensitivity
Specificity
Positive predictive value
(PPV)
Negative predictive value
(NPV)
Observed proficiency rate
(OPR)
Projected proficiency rate
(PPR)
Proficiency status
projection error
TP + TN
N
TP
TP + FN
TN
TN + FP
TP
TP + FP
TN
FN + TN
TP + FN
N
TP + FP
N
PPR - OPR
For reading, STAR correctly classified students as proficient or not between 84% and 89% of the time depending on grade. Sensitivity (i.e., the percentage
of students correctly forecasted as proficient) ranged from 71% to 87%. Specificity (i.e., the percentage of students correctly forecasted as not proficient)
values ranged from 84% to 94%. Positive predictive values ranged from 65% to 81%, meaning that when STAR forecasted students to be proficient, they
actually were 65%-81% of the time depending on grade. Negative predictive values ranged from 90% to 94%. When STAR forecasted that students were
not proficient, they actually were not 90%-94% of the time depending on grade.
Proficiency status projection errors ranged from -3% to 6% across grades. The area under the ROC curve (AUC) ranged from 0.91 to 0.95, providing a strong
indication that STAR did a good job of discriminating between which students reached the GA Milestones ELA proficiency benchmarks and which did not.
Finally, the correlations between STAR and GA Milestones ELA were between 0.81 and 0.85, reflecting a strong positive relationship between projected
STAR Reading scores and GA Milestones ELA scores.
Page 11 of 22
Table 9b. Accuracy of projected STAR™ scores for predicting GA Milestones proficiency: Mathematics
Measure
Formula
Interpretation
Grade
3
4
5
6
7
8
Percentage of correct classifications
85%
87%
86%
89%
88%
86%
Percentage of proficient students identified as such using STAR
85%
89%
81%
79%
84%
70%
Percentage of not proficient students identified as such using
STAR
85%
86%
88%
91%
89%
92%
Percentage of students STAR finds proficient who actually are
proficient
76%
76%
78%
71%
71%
77%
Percentage of students STAR finds not proficient who actually
are not proficient
91%
94%
90%
94%
95%
89%
Percentage of students who achieve proficient
36%
34%
34%
21%
24%
28%
Percentage of students STAR finds proficient
40%
40%
36%
24%
28%
25%
Difference between projected and observed proficiency rates
4%
6%
2%
3%
4%
-3%
Area Under the ROC Curve
Summary measure of diagnostic accuracy
0.94
0.94
0.93
0.96
0.95
0.93
Correlation (r)
Relationship between STAR and GA Milestones Mathematics
scores
0.86
0.84
0.83
0.80
0.80
0.77
Overall classification
accuracy
Sensitivity
Specificity
Positive predictive value
(PPV)
Negative predictive value
(NPV)
Observed proficiency rate
(OPR)
Projected proficiency rate
(PPR)
Proficiency status
projection error
TP + TN
N
TP
TP + FN
TN
TN + FP
TP
TP + FP
TN
FN + TN
TP + FN
N
TP + FP
N
PPR - OPR
For mathematics, STAR correctly classified students as proficient or not 85% to 89% of the time depending on grade. Sensitivity ranged from 70% to 89%
and specificity ranged from 85% to 92%. Positive predictive values ranged from 71% to 78%, meaning that when STAR forecasted students to be proficient,
they actually were 71%-78% of the time depending on grade. Negative predictive values ranged from 89% to 95%. When STAR forecasted that students
were not proficient, they actually were not 89%-95% of the time depending on grade.
Proficiency status projection errors ranged from -3% to 6% across grades and the area under the ROC curve (AUC) ranged from 0.93 to 0.96, providing a
strong indication that STAR did a good job of discriminating between which students reached the GA Milestones mathematics proficiency benchmarks and
which did not. Finally, the correlations between STAR and GA Milestones mathematics were between 0.77 and 0.86, reflecting a positive relationship
between projected STAR Math scores and GA Milestones Mathematics scores.
Page 12 of 22
Scatter plots: ELA
800
800
700
700
600
600
GA Milestones ELA
GA Milestones ELA
For each student, projected STAR Reading™ scores and GA Milestones ELA scores were plotted across grades.
500
400
300
200
500
400
300
200
0
200
400
600
800
1000
1200
1400
0
200
400
800
800
700
700
600
600
500
400
300
200
1000
1200
1400
500
400
300
200
0
200
400
600
800
1000
1200
1400
0
200
400
STAR Reading (Grade 5)
600
800
1000
1200
1400
STAR Reading (Grade 6)
800
800
700
700
600
600
GA Milestones ELA
GA Milestones ELA
800
STAR Reading (Grade 4)
GA Milestones ELA
GA Milestones ELA
STAR Reading (Grade 3)
600
500
400
300
200
500
400
300
200
0
200
400
600
800
1000
1200
STAR Reading (Grade 7)
1400
0
200
400
600
800
1000
1200
1400
STAR Reading (Grade 8)
Page 13 of 22
Scatter plots: Mathematics
800
800
700
700
GA MilestonesMathematics
GA Milestones Mathematics
For each student, projected STAR Math™ scores and GA Milestones Mathematics scores were plotted across grades.
600
500
400
300
200
600
500
400
300
200
0
200
400
600
800
1000
1200
1400
0
200
400
STAR Math (Grade 3)
1000
1200
1400
1200
1400
800
GA MilestonesMathematics
GA Milestones Mathematics
800
STAR Math (Grade 4)
800
700
600
500
400
300
200
700
600
500
400
300
200
0
200
400
600
800
1000
1200
1400
0
200
400
STAR Math (Grade 5)
600
800
1000
STAR Math (Grade 6)
800
800
700
700
GA Milestones Mathematics
GA Milestones Mathematics
600
600
500
400
300
200
600
500
400
300
200
0
200
400
600
800
1000
STAR Math (Grade 7)
1200
1400
0
200
400
600
800
1000
1200
1400
STAR Math (Grade 8)
Page 14 of 22
References and additional information
Bandeira de Mello, V., Blankenship, C., & McLaughlin, D. H. (2009). Mapping state proficiency standards onto NAEP scales:
2005–2007 (NCES 2010-456). Washington, DC: U.S. Department of Education, Institute of Education Sciences,
National Center for Education Statistics. Retrieved from
http://www.schooldata.org/Portals/0/uploads/Reports/LSAC_2003_handout.ppt
Cronin, J., Kingsbury, G. G., Dahlin, M., & Bowe, B. (2007, April). Alternate methodologies for estimating state standards on
a widely used computer adaptive test. Paper presented at the American Educational Research Association,
Chicago, IL.
McGlaughlin, D. H., & V. Bandeira de Mello. (2002, April). Comparison of state elementary school mathematics achievement
standards using NAEP 2000. Paper presented at the annual meeting of the American Educational Research
Association, New Orleans, LA.
McLaughlin, D., & Bandeira de Mello, V. (2003, June). Comparing state reading and math performance standards using
NAEP. Paper presented at the National Conference on Large-Scale Assessment, San Antonio, TX.
McLaughlin, D., & Bandeira de Mello, V. (2006). How to compare NAEP and state assessment results. Paper presented at
the annual National Conference on Large-Scale Assessment. Retrieved from http://www.naepreports.org/task
1.1/LSAC_20050618.ppt
McLaughlin, D. H., Bandeira de Mello, V., Blankenship, C., Chaney, K., Esra, P., Hikawa, H., et al. (2008a). Comparison
between NAEP and state reading assessment results: 2003 (NCES 2008-474). Washington, DC: U.S. Department of
Education, Institute of Education Sciences, National Center for Education Statistics.
McLaughlin, D.H., Bandeira de Mello, V., Blankenship, C., Chaney, K., Esra, P., Hikawa, H., et al. (2008b). Comparison
between NAEP and state mathematics assessment results: 2003 (NCES 2008-475). Washington, DC: U.S. Department
of Education, Institute of Education Sciences, National Center for Education Statistics.
Perie, M., Marion, S., Gong, B., & Wurtzel, J. (2007). The role of interim assessments in a comprehensive assessment system.
Aspen, CO: Aspen Institute.
Renaissance Learning. (2014). The research foundation for STAR Assessments: The science of STAR. Wisconsin Rapids, WI:
Author. Available online from http://doc.renlearn.com/KMNet/R003957507GG2170.pdf
Renaissance Learning. (2015a). STAR Math: Technical manual. Wisconsin Rapids, WI: Author. Available from Renaissance
Learning by request to [email protected]
Renaissance Learning. (2015b). STAR Reading: Technical manual. Wisconsin Rapids, WI: Author. Available from
Renaissance Learning by request [email protected]
Page 15 of 22
Independent technical reviews of STAR Reading™ and STAR Math™
U.S. Department of Education: National Center on Intensive Intervention. (2015a). Review of progress monitoring tools
[Review of STAR Math]. Washington, DC: Author. Available online from
http://www.intensiveintervention.org/chart/progress-monitoring
U.S. Department of Education: National Center on Intensive Intervention. (2015b). Review of progress monitoring tools
[Review of STAR Reading]. Washington, DC: Author. Available online from
http://www.intensiveintervention.org/chart/progress-monitoring
U.S. Department of Education: National Center on Response to Intervention. (2010a). Review of progress monitoring tools
[Review of STAR Math]. Washington, DC: Author. Available online from
https://web.archive.org/web/20120813035500/http://www.rti4success.org/pdf/progressMonitoringGOM.pdf
U.S. Department of Education: National Center on Response to Intervention. (2010b). Review of progress monitoring tools
[Review of STAR Reading]. Washington, DC: Author. Available online from
https://web.archive.org/web/20120813035500/http://www.rti4success.org/pdf/progressMonitoringGOM.pdf
U.S. Department of Education: National Center on Response to Intervention. (2011a). Review of screening tools [Review of
STAR Math]. Washington, DC: Author. Available online from http://www.rti4success.org/resources/toolscharts/screening-tools-chart
U.S. Department of Education: National Center on Response to Intervention. (2011b). Review of screening tools [Review of
STAR Reading]. Washington, DC: Author. Available online from http://www.rti4success.org/resources/toolscharts/screening-tools-chart
Page 16 of 22
Sample dashboard and reports4
Reading Dashboard. The Reading Dashboard combines key data from across Renaissance’s products to provide a 360
degree view of student performance and progress toward goals. At-a-glance data paired with actionable insight helps
teachers drive student growth to meet standards and ensure college and career readiness. Educators can review data at
the student, group, or class level and various timeframes to determine what’s working, what isn’t, and most importantly,
where to focus attention to drive growth.
4
Reports are regularly reviewed and may vary from those shown as enhancements are made.
Page 17 of 22
Sample STAR™ Performance Reports focusing on the Pathway to Proficiency. This report will be available to schools
using STAR Reading Enterprise or STAR Math Enterprise. The report graphs the student’s STAR Reading or STAR Math
scores and trend line (projected growth) for easy comparison with the pathway to proficiency on the PAWS.
Page 18 of 22
Sample Group Performance Report. For the groups and for the STAR test date ranges identified by educators, the Group Performance Report compares your
students' performance on the STAR Assessments to the pathway to proficiency for your annual state tests and summarizes the results. It helps you see how groups
of your students (whole class, for example) are progressing toward proficiency. The report displays the most current data as well as historical data as bar charts so
that you can see patterns in the percentages of students on the pathway to proficiency and below the pathway—at a glance.
Page 19 of 22
Sample Growth Proficiency Chart. Using the classroom GPC, school administrators and teachers can better identify best practices that are having a significant
educational impact on student growth. Displayed on an interactive, web-based growth proficiency chart, STAR Assessments’ Student Growth Percentiles and
expected State Assessment performance are viewable by district, school, grade, or class. In addition to Student Growth Percentiles, the Growth Report displays
other key growth indicators such as grade equivalency, percentile rank, and instructional reading level.
Page 20 of 22
Sample Performance Reports. This report is for administrators using STAR Reading and STAR Math assessments. It provides users with periodic, high level
forecasts of student performance on your state's reading and math tests. It includes a performance outlook for each performance level of your annual state tests.
The report includes options for how to group and list information. These reports are adapted to each state to indicate the appropriate number and names of
performance levels.
Page 21 of 22
Acknowledgments
The following experts have advised Renaissance Learning in the development of the STAR Assessments.
Thomas P. Hogan, Ph.D., is a professor of psychology and a Distinguished University
Fellow at the University of Scranton. He has more than 40 years of experience conducting
reviews of mathematics curricular content, principally in connection with the preparation of
a wide variety of educational tests, including the Stanford Diagnostic Mathematics Test,
Stanford Modern Mathematics Test, and the Metropolitan Achievement Test. Hogan has
published articles in the Journal for Research in Mathematics Education and Mathematical Thinking and Learning,
and he has authored two textbooks and more than 100 scholarly publications in the areas of measurement and
evaluation. He has also served as consultant to a wide variety of school systems, states, and other organizations
on matters of educational assessment, program evaluation, and research design.
James R. McBride, Ph.D., is vice president and chief psychometrician for Renaissance
Learning. He was a leader of the pioneering work related to computerized adaptive testing
(CAT) conducted by the Department of Defense. McBride has been instrumental in the
practical application of item response theory (IRT) and since 1976 has conducted test
development and personnel research for a variety of organizations. At Renaissance Learning,
he has contributed to the psychometric research and development of STAR Math, STAR
Reading, and STAR Early Literacy. McBride is co-editor of a leading book on the development of CAT and has
authored numerous journal articles, professional papers, book chapters, and technical reports.
Michael Milone, Ph.D., is a research psychologist and award-winning educational writer
and consultant to publishers and school districts. He earned a Ph.D. in 1978 from The Ohio
State University and has served in an adjunct capacity at Ohio State, the University of
Arizona, Gallaudet University, and Minnesota State University. He has taught in regular and
special education programs at all levels, holds a Master of Arts degree from Gallaudet
University, and is fluent in American Sign Language. Milone served on the board of directors
of the Association of Educational Publishers and was a member of the Literacy Assessment Committee and a
past chair of the Technology and Literacy Committee of the International Reading Association. He has
contributed to both readingonline.org and Technology & Learning magazine on a regular basis. Over the past 30
years, he has been involved in a broad range of publishing projects, including the SRA reading series,
assessments developed for Academic Therapy Publications, and software published by The Learning Company
and LeapFrog. He has completed 34 marathons and 2 Ironman races.
Page 22 of 22
R45876.160330