Improving District Problem Solving with Data Dashboards

Your Turn
Data in Your Everyday Life
h"p://miblsi.cenmi.org2
Improving District
Problem Solving with
Data Dashboards
•  Take 20 seconds to write down at
least 1 type of data that you use in
your everyday life
•  Share with two to three people
around you
APBS 2015
Jennifer Rollenhagen
Catherine Claes
2
Session Objectives
Agenda
•  Understand the function of a
dashboard
•  Understand how district implementation
teams can use a dashboard within a
school improvement framework for
data-based decision making
•  Develop or redefine MTSS
implementation plans based on the
data analysis
I.  Understanding the function of a
Dashboard
II.  Using a dashboard for Data-Based
Decision Making
III.  Developing an Implementation Plan
3
The Context
4
MiBLSi Mission Statement
Michigan’s Integrated Behavior and
Learning Support Initiative (MiBLSi) is
funded by the Michigan Department of
Education, Office of Special Education
and Michigan’s State Personnel
Development Grant from the U.S.
Department of Education, Office of
Special Education Programs.
To develop capacity to scale-up an
integrated behavior and academic MTSS
model that can be implemented with
fidelity, endures over time and utilizes
data-based decision making at all levels
of implementation support
5
6
Understanding the Function
of a Dashboard
8
7
Why a Dashboard?
Effective Dashboard Displays
1.  Filter out what is not relevant
“A dashboard is a visual display of the
most important information needed to
achieve one or more objectives
consolidated on a single screen so it can
be monitored and understood at a glance”
2.  Sort the data to see it in order of
magnitude
3.  Provide high-level (big picture) and
low-level (detailed) views of the data
Stephen Few
4.  Offer varying views and perspectives
of the data using different display types
9
What are District Dashboards
10
District Dashboard
District dashboards
were built to align with
a specific set of data
analysis questions and
action planning steps to
help districts
continuously improve
implementation of
MTSS to maximize
positive student
outcomes within a
School Improvement
Framework.
•  When we put it all on one screen it can
be really exciting and maybe a little
much all at once
•  The good news… we have a wealth of
data to inform our problem solving and
action planning and you won’t see it all
at once
11
•  And we will walk you through each tab
as it relates to the data review
questions!
12
The MiBLSi Database
MiData
Unique Features of MiData
•  Dashboards and reports are designed for alignment
with MiBLSi’s data review process at the school,
district, and ISD levels
The MiBLSi Database (MiData) includes
district dashboards that aggregate school-level
information for use during problem-solving
sessions.
•  A place to enter and analyze reach, capacity, and
fidelity data not hosted in any other system
•  A single point of data entry (school-level) that
aggregates data up to the district, ISD, and state
levels
•  Based on principles for effective display of data and
information processing, combined with repeated
cycles of usability testing
13
Types of Data
14
Reading Student Outcome Data
•  DIBELS
•  AIMSweb
Student Outcome Data:
Meeting educational goals and objectives
Program Quality/Fidelity Data:
Extent that implementation occurs as
intended
Engagement Outcome Data
•  EWS
Capacity Data:
The ability or power to do
Behavior Outcome Data
•  SWIS
•  SRSS
Reach Data:
Extend the range of contact and
influence
15
16
Behavior Implementation
Fidelity Measures
Reading Implementation
Fidelity Measures
•  Planning and Evaluation Tool for
Effective School-wide Reading
Programs (PET-R)
•  School-wide Evaluation Tool for
Secondary Literacy (SWEPT)
•  School-Wide Evaluation Tool for
Reading-Self-Assessment (SWETR-S)
• 
• 
• 
• 
• 
17
PBIS Tiered Fidelity Inventory (TFI)
Benchmarks of Quality (BoQ)
Benchmarks of Advanced Tiers (BAT)
Self Assessment Survey (SAS)
School-wide Evaluation Tool (SET)
18
The Process and Logistics of
Developing a Dashboard
Capacity and Reach Measures
System Capacity Measures:
•  District Capacity Assessment (DCA)
•  Regional Capacity Assessment (RCA)
Content
Knowledge/
Expertise
Technical
Design
Expertise
Capacity &
Competent
Usage
Usability Testing
19
20
Reflection
What is your current experience using
data dashboards in your district?
How can dashboards improve your
district’s data analysis?
Using a Dashboard for
Data-based Decision Making
21
District Data Review Process
22
Timeline of Data Review Process
ISD
Systems
Review
To determine our district supports for
implementing MTSS with fidelity at the
district and school level with a focus on
efficiency, effectiveness and durability that
will address the needs of all levels of
learners
ISD
Data
Review
District
Data
Review
School
Level
Data
Review
School
Level
Coaches
Meeting
23
ISD
Data
Review
ISD
Data
Review
District
Data
Review
District
Data
Review
District
Check-In
School
Level
Data
Review
School
Level
Coaches
Meeting
School
Level
Data
Review
School
Level
Coaches
Meeting
24
Examining the Dashboard Data
Key Question to Consider:
Given where you are heading with
MTSS implementation (your
objectives), how close are you to
getting there?
26
25
Considerations When Examining
Reach Data
Recall
Student Outcome Data:
Meeting educational goals and objectives
•  Do you need to update the stage of
implementation for any of the schools
in your district?
Program Quality/Fidelity Data:
Extent that implementation occurs as
intended
•  If yes, please go to the Data Entry
section of MiData and make any
necessary changes
Capacity Data:
The ability or power to do
Reach Data:
Extend the range of contact and
influence
•  What has been done to systematically
support schools in pre-exploration and/
or exploration to move into installation?
27
28
Considerations While Studying
Outcome Data
Reach Data
•  Student Outcome Data:
•  How do these data compare to our
objective(s)?
•  Do we see improvement/growth from
Fall to Winter? From year to year?
•  Do the data indicate a district-wide
trend?
29
30
Student Outcome Data
Student Outcome Data
31
32
Program Quality/Fidelity Data
Considerations While Studying
Fidelity Data
•  Fidelity Data:
•  Do we have any new Program Quality/
Fidelity data since the last data review?
•  Implementation Plan/Action Plan
•  The focus of your plan is on the District
working to make it happen, not just
letting it happen
33
Program Quality/Fidelity Data
34
Considerations When Addressing
Capacity
•  Current Capacity to
support MTSS
Implementation
•  Needs related to Capacity given the layering
on of additional focus area(s) and/or schools
35
36
Capacity Data
Bottom Line
Our Data Review process will only be as
strong as the data we have available for
analysis
If we work to ensure we have access to
high quality data we will enhance the
quality of our data review and planning
37
38
Reflection
What is the current process for databased decision making in your
district?
Developing an
Implementation Plan
How can information presented today
improve that process in your district?
39
40
MTSS Implementation Plan:
School Improvement Framework
MTSS Implementation Plan:
Big Picture
•  You need a plan to guide your work
•  That plan should have target(s) for
what you are trying to achieve (goals
and/or objectives)
•  Your plan should spell out the steps
needed to get your target including
who will do what and by when
•  You’ll need to monitor the progress
on your plan
•  Michigan’s Continuous School
Improvement Framework has the
components for building a strong plan
•  Provides a framework for creating a
plan
•  Allows for aligning the work into one
plan and attaching funding to MTSS
work
41
42
The dashboard organizes your district’s
data to increase the efficiency of plan
development
MTSS Implementation Plan
Goal: Broad statement focused on an
academic or an organizational goal.
Objective: What will happen, how will it
be measured, and by when
43
MTSS Implementation Plan
44
Strategies & Activities
Strategy: an action that is specific,
planned, and evidence based – can be
thought of as the “how” to achieve the
objective and/or goal
•  Should begin with “The District will…”
or “The District Implementation Team
will…”
•  Should not be focused on what the
buildings are doing but rather what the
district is doing make MTSS
implementation with fidelity happen
Activity: an individual task or function
performed as part of an overall strategy
for instruction or improvement – these are
specific and include a comprehensive
description of the steps to be taken
45
Maintaining Your
ISD MTSS Implementation Plan
46
Refining Your Plan
•  Likely going to need to revise/refine
your strategies and activities
•  Plan is the focus of data review work
•  Action items are regularly reviewed and
updated when progress is made
•  Celebrations are collected
•  Barriers are addressed
•  Additional actions are added as needed
•  Timelines are revisited and monitored
•  May need to add additional strategies
and activities
•  Our goal is to support your work so we
have some sample strategies and
activities related to the example
objectives from earlier today
47
48
Partner Share
Addressing Barriers
If you have a dashboard in your district,
share how the dashboard improves the
efficiency of data analysis, decision
making and planning at district,
building, and grade level.
•  Schools identified specific barriers to
their implementation efforts
•  As you work to develop your plan, be
sure that you are systematically
addressing barriers (as appropriate) in
order to support school-level
implementation efforts
If you do not have a data dashboard in
your district, what talking points could
you take back around the benefits of
dashboards?
49
50