Plano ISD Formative Assessment Process Progress Monitoring

Plano ISD Formative Assessment Process
Progress Monitoring
What students have
learned – What they
are ready to learn
Measuring Against
State Standards
STAAR
MAP
Assessment
Curriculum
Plan
TEKS
TPRI/TLee
How students learn
TELPAS
CogAT
DesCartes
Instructional
Adaptation:
Strategies, Scaffolding
Grouping
Differentiation
Classroom
Assessment
College Readiness
Standards
EXPLORE
PLAN
Measuring Growth
Meeting/Exceeding
Verbal
Battery
CogAT
Nonverbal Battery
CogAT
STAAR
Lesson
NWEA-MAP
Development and
Delivery
Quantitative
Battery
Percent of Schools
Distinction for Top 25% in Student Growth
COMPETING
AGAINST LIKE
CAMPUSES
69%
67%
41%
27%
Elementary
(18/44)
Middle Schools
(9/13)
PLANO
STATE
PLANO
STATE
20%
PLANO
STATE
24%
High Schools
(4/6)
2
Assessments Given in Plano
NWEA MAP
Measure growth
Probability of STAAR Level II
Probability of STAAR Level III
Common Semester
Exams in Grades 6-12
in all Core Subjects
End of Unit
Assessments for
Content Mastery
Ability Testing – CogAT
Ability Profile to understand
how a student learns
Differentiation and grouping
strategy
Tier II and Tier III
iStation
District Common Assessments
3
Index 2: Student Progress
Change Score to Progress Expectation
Level I/II: Met
(Current Level II Final Standard)
− (Prior Level II Final Standard)
Level III: Met
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Change Score Points Needed (Current Level III Standard)
− (Prior Level III Standard)
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Change Score Points Needed
Level I/II/III: Exceeded
(Current Level III Standard)
− (Prior Level II Final Standard)
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Change Score Points Needed
1551
1583
Grade 7 2012
Grade 8 2013
Current Year:
Level I
Chance Range
0 – 13 Items
1798
Current Year:
Level II
Chance Range
0 – 12 Items
Prior Year:
Level II
Prior Year:
Level III
Current Year:
Level III
Prior Year:
Level I
Understanding STAAR Growth – Gr 7 Math
2233
2189
1863
1505
1448
2012 – 2013 STAAR Student Progress Report
Elementary School Math Grade 4 to 5
Mutually Exclusive
Groups Based on Prior
Year STAAR Scale Score
Elementary STAAR Growth Roster
Your School - 99
STAAR Growth Analysis Guide
• Purpose is to guide campus
staff through analysis and
provide framework for team
discussions about student
growth as shown on STAAR
• STAAR Student Progress
Report
– Performance of groups of
students – by prior year STAAR
Performance Levels
• STAAR Growth Roster
– Performance of students
9
STAAR Student Progress: Focused Data Analysis Guide
Goals of STAAR Student Growth Analysis
The Focused Data Analysis provides a structured process for analyzing and discussing student growth on
STAAR. Students’ 3-8 STAAR scale scores in Reading and Math are reported on a vertical scale and should
increase from one year to the next. Student Progress Targets are specific to grade and subject. The Met
Progress targets are determined by comparing the differences between the prior year Level II Final standard
and the current year Level II Final standard. In addition, maintaining a Level III performance from the prior
year to the current year is classified as having Met or Exceeded Progress. The Exceeded Progress targets are
determined by comparing the differences between the prior year Level II Final standard and the current year
Level III standard. Scoring one of the three highest raw scores in the current year is classified as having
Exceeded Progress. Writing STAAR and all EOCs are reported on a horizontal scale. For these tests, the
progress targets may be zero or negative but they still represent growth from one year to the next because
of the increased difficulty in content.
Data Sources

STAAR Student Progress Report

STAAR Growth Roster
Analysis of STAAR Student Progress Report
The purpose of this analysis is to identify student groups who are not meeting STAAR progress
standards as outlined in Index 2 of the Accountability Report.
1. Determine how many students on your campus were in each Prior Year Level





Prior Year Level divides students who took STAAR into four groups based on what level of
performance they achieved on STAAR in the year previous.
Level I = Students who achieved Unsatisfactory Performance on STAAR
Level II Phase-in-1 = Students who achieved Satisfactory Performance on STAAR
Level II Final = Students who achieved Satisfactory Performance on STAAR AND achieved a scale
score greater than or equal to the Final Recommended standard on STAAR
Level III = Students who achieved Advanced Performance on STAAR
2. Determine the number of students from each Prior Year Level that Met or Exceeded Progress in the
Current Year.


Current Year Level divides students who took STAAR into three groups based on what level of
performance they achieved on STAAR in the spring of the current year.
D=District Percentage
3. Determine number of students who fall within shaded regions on the STAAR Student Progress Report.

In order to be considered sufficiently prepared for the next grade level or course, a student must
achieve at least a Level II performance in the current year. Students starting at Level II (Phase-in
1) must at least achieve “Met Progress” to maintain the same grade level standard. Students in
the shaded regions include those who maintained a Level I performance in the current year but
did not exceed progress targets, and those who achieved a Level II but were classified as “Not
Met Progress.”
4. Compare the percentages of students in each column with district percentages.



Not Met Progress = Goal is 0%; campus should aim to be close to or below district %. (D= %)
Met/Exceeded Progress = Goal is 100%; campus should aim to be close to or above district %.
If a student group does not demonstrate growth similar to, or better than the district, further
analysis is needed to find out why this is the case and what action needs to be taken.
Assessment and Accountability Plano ISD
S:\Groups\Research\2013-2014\Projects\STAAR_Growth\STAAR Student Progress Focused Data Analysis.docx
Page 1 of 2
STAAR Student Progress: Focused Data Analysis Guide
Analysis of STAAR Growth Roster
The purpose of this analysis is to identify students that are not meeting or exceeding
progress and to identify the instructional programs and strategies that have shown
success with students.
Prior Year Level I
Goal: For ALL Prior Year Level I students to achieve a Level II or Exceed Progress in the Current Year
1. Identify all students in Prior Year Level I who did not achieve a Level II in the Current Year.
2. Evaluate the programs and instructional strategies used for these students.
3. Analyze whether the performance of students not meeting progress is consistent with classroom
performance and other assessments (CogAT, MAP).
4. Complete a Needs Analysis and create an instructional plan to address the needs of these students.
5. Identify Prior Year Level I students who met or exceeded progress. Determine the successful
strategies or interventions used with these students.
Prior Year Level II (Phase-in-1 and Final)
Goal: For ALL Prior Year Level II students to Meet or Exceed Progress in the Current Year
1. Identify all Prior Year Level II students who did not meet progress and/or achieved a Level I in the
Current Year.
2. Evaluate the programs and instructional strategies used for these students.
3. Analyze whether the performance of students not meeting progress is consistent with classroom
performance and other assessments (CogAT, MAP).
4. Complete a Needs Analysis and create an instructional plan to address the needs of these students.
5. Identify Prior Year Level II students who met or exceeded progress. Determine the successful
strategies or interventions used with these students.
Prior Year Level III
Goal: For ALL Prior Year Level III students to Meet or Exceed Progress in the Current Year
1. Identify all Prior Year Level III students who did not meet or exceed progress in the Current Year.
2. Evaluate the programs and instructional strategies used for these students.
3. Analyze whether the performance of students not meeting progress is consistent with classroom
performance and other assessments (CogAT, MAP).
4. Identify Prior Year Level III students who met or exceeded progress. Determine the successful
strategies used with these students.
Evaluation
1. Identify and discuss the various instructional programs, interventions, and strategies used with
different student groups on your campus.
2. Evaluate the effectiveness of these programs for each student group.
3. Identify and discuss ways that other campuses or grades plan and implement instruction,
interventions, and strategies for similar student groups.
Assessment and Accountability Plano ISD
S:\Groups\Research\2013-2014\Projects\STAAR_Growth\STAAR Student Progress Focused Data Analysis.docx
Page 2 of 2
Plano Effect Score
MAP Normative Growth
Score
10
MAP Adaptive Item Progression
260
Grade 4 Mathematics
255
250
RIT Level of Item
245
RIT Score - 229
240
235
230
225
220
215
210
205
200
1
4
7
10 13 16 19 22 25 28 31 34 37 40 43 46 49 52
Item Sequence
MAP Initial Scores and Quintile Bands
140
Quintile 1
Q2
Q3 Q 4
Quintile 5
100
80
60
40
20
0
171
178
185
190
195
200
205
210
215
220
225
230
235
240
245
250
255
260
265
270
275
280
285
295
306
Number of Students
120
Starting Grade 7 Mathematics MAP Score
MAP Normative Growth Report
13
Analysis of PES - Normative Growth Report
Determine the
distribution of
students among
quintiles on
your campus.
Identify quintile
groups that fall
below the Lower
Bounds OR do
not demonstrate
growth similar to
other quintiles.
14
Identify specific
students whose results
contributed to PES
below Lower Bound
Identify the
Analyze
performance
instructional
in
other assessments
strategies
that were
(STAAR/CogAT)
used for these for
patterns
students/evaluate
effectiveness
15
Identify ALL students in
quintiles 1 and 2
Complete Needs
Assessments for these
students
Identify students for
whom a higher goal
might be appropriate
16
For further analysis…
 Assess the performance of
each quintile
 Look for patterns between
and among quintiles
 Identify ways other
campuses or grades plan
and implement intervention
and instruction successfully
17
18
MAP Normative Growth Focused Data Analysis Guide
Grade Level
Subject
Goals of Student Growth Analysis
The Focused Data Analysis provides a structured process for analyzing and discussing student growth.
Almost all students’ scores increase from one test to the next, but some students do not progress as
much as is typical of students who had the same starting score on the RIT scale. Typical growth is the
average growth obtained by Plano students in prior years who shared the same starting point on the RIT
scale. Students scoring within plus or minus three points should be considered as achieving typical
growth.
Data Sources

Plano Effect Score Report

PES Roster Analysis Report
Analysis of Plano Effect Score – Normative Growth Report
The purpose of this analysis is to identify quintile groups that are falling below typical
growth.
1. Determine how many students are in each quintile and the distribution of students among
quintiles on your campus.
Quintiles divide students in a grade level into five groups, based on initial RIT score.
Quintile 1 represents MAP scores that are the lowest 20% for that grade level in the district.
Quintile 5 represents MAP scores that are the highest 20% for that grade level in the district.
2. Identify Plano Effect Scores for any quintiles that do not fall within the Lower and Upper Bounds
of the typical growth for that grade or subject.
3. Identify any quintiles that performed above the lower bound but that are lower than other
quintiles for this grade and subject.
In summary, if your campus or grade has identified a quintile group that falls below the Lower Bound
or does not demonstrate growth similar to other quintiles, further analysis is needed to find out why
this is the case and what action needs to be taken.
Analysis of Plano Effect Score – Normative Growth Roster
The purpose of this analysis is to identify students that are not meeting typical growth
and to identify the instructional programs and strategies that have shown success with
students.
1. If your campus or grade has identified a quintile group that fell below the Lower Bound, locate
that quintile group on the PES Roster and identify specific students whose results may have
contributed to this pattern.
a. Identify instructional strategies used for these students and the effectiveness of these
strategies.
b. Identify other students within the quintile who did show growth. Determine the
successful strategies or interventions used with these students.
Assessment and Accountability Plano ISD S:\Groups\Research\2013-2014\Presentations\2014.02.16 TexasAssessmentConference\MAP
Focused Data Analysis 2013-2014.docx
Page 1 of 2
MAP Normative Growth Focused Data Analysis Guide
2. If your campus or grade has identified a quintile group that did not demonstrate growth similar
to other quintiles in the same subject, locate that quintile group on the PES Roster and identify
specific students whose results may have contributed to this pattern.
a. Identify strategies, programs, or services provided to these students and analyze the
results.
b. Identify similar quintile groups in other grade levels in your building that did show
growth. Identify the strategies, programs, or services of successful quintile groups.
3. Assess the performance of each quintile.
a. Identify 2 or 3 RIT bands within each quintile in which small percentages of students met
their typical growth score.
b. Identify any patterns that are revealed by the CogAT Profiles and by other academic
characteristics of students not meeting their typical growth score.
c. Analyze whether the performance of students with less than typical growth is consistent
with classroom performance and other assessments.
4. Identify students whose RIT score showed a less than 50% probability of success on STAAR.
Take note of the quintile ranges in which these students appear.
5. Identify the specific students in quintiles 1 and 2.
a. Even when students in quintiles 1 and 2 make typical growth, they will not move up to
the next quintile. Identify those students in quintiles 1 and 2 for whom it would be
appropriate to set a goal of more than typical growth.
b. Complete a Needs Analysis on each of the students in quintiles 1 and 2 to identify
specific student needs.
c. Create an instructional plan to address the needs of students in quintiles 1 and 2.
Evaluation
1. Identify various instructional programs and strategies used for the different groups of students
on your campus.
2. Evaluate the effectiveness of these programs and strategies for different groups of students.
3.
Identify ways that other campuses or grades plan and implement these interventions.
Assessment and Accountability Plano ISD S:\Groups\Research\2013-2014\Presentations\2014.02.16 TexasAssessmentConference\MAP
Focused Data Analysis 2013-2014.docx
Page 2 of 2
Predicting a Student’s
STAAR Performance
19
100
1.0
90
0.9
80
0.8
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
10
0.1
0
0.0
170
175
180
185
190
195
200
205 209
210
216 220
215
225
Current Spring RIT Reading
230
235
240
245
250
255
260
Probability of Meeting Level II Standards at Each RIT Score
Percent Achieving STAAR Level II
MAP to STAAR Grade 4 Reading Level II 2012
Probability of Meeting STAAR Standard
262
264
259
253
70% Chance of Level III
Well Prepared
90% or higher of Level II
Sufficiently Prepared
70-89% Chance of Level II
50-69% Chance of Level II
246
239
Less than 50% Chance of
Meeting Level II on STAAR
229
217
203
Mathematics
Less than 50% Chance of
Meeting Level II on STAAR
184
KN
1
2
3
4
5
6
7
8
9
21
Student A Probability Range
Greater than 70%
chance of Meeting
Level III Standard
• Level III in Grade 3
43/46 Items Correct
• Level III in Grade 4
44/48 Items Correct
Student B Probability Range
Chance of Meeting
Phase-In-2
Standard = 50-69%
• Level II in Grade 3 (Phase-In 1)
Met Phase-In-2 Standard
by 0 items
• Level II in Grade 4 (Phase-In 1)
Missed Phase-In-2 Standard by 2
items
23
Ability Performance Charts
STAAR and MAP
CogAT
24
Ability-Performance Charts - MAP
Ask The Right Questions
≥70% Level III
≥ 90% Level II
70-89% Level II
50-69% Level II
≤50% Level II
25
Ability-Performance Charts - MAP
Look for Patterns
≥70% Level III
≥ 90% Level II
70-89% Level II
50-69% Level II
≤50% Level II
26
STAAR Math Performance vs Quantitative Ability
27
Percent of Schools
Distinction for Top 25% in Student Growth
COMPETING
AGAINST LIKE
CAMPUSES
69%
67%
41%
Middle Schools
(9/13)
PLANO
STATE
20%
PLANO
PLANO
STATE
Elementary
(18/44)
STATE
27%
24%
High Schools
(4/6)
28
Plano ISD Formative Assessment Process
What students have
learned – What they
are ready to learn
Measuring Against
State Standards
STAAR
MAP
Assessment
Curriculum
Plan
TEKS
TPRI/TLee
How students learn
TELPAS
CogAT
DesCartes
Instructional
Adaptation:
Strategies, Scaffolding
Grouping
Differentiation
Classroom
Assessment
College Readiness
Standards
EXPLORE
PLAN
Measuring Growth
Meeting/Exceeding
Verbal
Battery
CogAT
Nonverbal Battery
CogAT
STAAR
Lesson
NWEA-MAP
Development and
Delivery
Quantitative
Battery