ITBS Consortium Workbook 2013

ITBS Consortium Workbook - 2013
The purpose of this workbook is to give directions for analyzing ITBS data at the school level. The first three parts are a
guide for comparing your school’s scores to others.
Part A: Comparing your school with the national norms. (Use iRM reports)
Part B: Comparing your school with the consortium results. (Use OCS Forum)
Part C: Comparing your school with other schools in the consortium. (Use My Desk> ITBS Annual Test Data)
Part D: Comparing data within your school by using Disaggregation (Use iRM)
Part E: Discusses the use of ITBS scores for annual ISL accreditation requirements. (Use My Desk)
At the classroom level, the iRM tool allows for greater depth of formative data analysis for individual student needs. To
learn more about using iRM in the classroom: http://www.riversidepublishing.com/products/irm/userresources.html.
Part A. Comparing your school with the national norms. (Five good reports in iRM.)
Creating reports is done by choosing settings in the left column of the Interactive Results Manager (iRM). Be warned
that they are often slow in loading, so be patient. The sample Reports shown are created setting Grade Level=3, and
they report the cumulative scores for the Archdiocese, not a specific school. Your reports will look similar but have your
school name instead of Seattle Archdiocese. Remember to only set the Grade/Level to a grade you have tested,
otherwise you will get a blank screen with the message: There are no data available for your selection. Reports can be
exported as a PDF or Excel spreadsheet. There are five key reports of particular importance. If you need more help to
generating the five reports in this part, see Appendix: Sample steps in generating an iRM Report at the end of this
document.
1. Group Display – National Percentile Rank
This is the most common report and shows the battery of test scores, for a given grade level. Get this report by
selecting Display Type=Group Display and Score Type= National Percentile. Your school norms are graphed
against national percentiles, 0 to 100. Depending upon your screen size, you may need to scroll to see the whole
list.
Group Display choice
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 1
2. Group Display – Grade Equivalent
This is like the previous report but reports by grade equivalents for the battery of scores. Change Score Type to
Grade Equivalent. These reports are effective when communicating to parents and lay audiences.
Grade Equivalent choice
3. Group Common Core – Percent Correct on core skills
The core subjects, Reading, Language and Math have been correlated with the common core standards. Skills
are reported by percent correct; these numbers should not be confused with National Percentile. For each skill,
your grade level is compared to the national grade level of percent correct.
Select the Display Type=Group Common Core Domain Display and a Grade/Level.
Display Type: Group Common Core
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 2
4. Proficiency Level – Above and below ranks
Proficiency scores are displayed by both a percentage and the number of students who are above or below
proficiency. Select the Display Type=Proficiency Level Display and a Grade/Level.
Display Type: Proficiency Level
5. Group Population Display – Quartile, Gender and Ethnicity detail
Group Population Displays provides a detailed breakdown of gender, ethnicity and quartile divisions. In general,
this report can be cumbersome since it gives the detail for only a single skill in the battery of tests. This detail
can be helpful when completing the ISL accreditation templates.
Display Type=Group Population Display with Content Area=Vocabulary.
Note single content area
on this type of report.
Display Type: Group Population
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 3
Part B. Comparing your school with the consortium results. (Use OCS Forum)
Using the five key reports from Part A, you can also compare your school to the composite Archdiocese scores for the
participating schools. The comprehensive Archdiocesan results are available from OCSWW.org in the ITBS Consortium
Forum. http://www.ocsww.org/site/forumdisplay.php?f=64. Or, they can be accessed by going to OCSWW.org and
clicking on Forums and then, under the heading Data Management, selecting ITBS/iRM Consortium as shown below.
Forums
This form
This thread
Choose the Archdiocese results for Comparison and Ranking or by Subject.
Comparison thread
Subject Areas thread
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 4
The Comparison files are listed by grade for each of the five types of files described in Part A.
The sample screen below relates to Group Display NPR, the first of the five.
The sample screen below shows a Subject (Language Totals) by grade for the consortium.
There are many more subject scores posted each year.
Note: The 2013 files will be posted when processed
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 5
Part C. Comparing your school with other schools in the consortium.
Use My Desk> ITBS Annual Test Data (Consortium)
By grade level, compare your school across subjects using either the NPR, SS, or Core NPR rankings. Scores of individual
consortium schools are available without identification of the school. This data is not public and is only available by
logging on to OCSWW.org, clicking on My Desk, and choosing ITBS Annual Test Data (Consortium). The yellow values
are your scores. Note that multiple yellow scores in a column means that you share the same score with other schools.
Sub-menu choice for
ranking by grade.
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 6
Consortium Averages: This option on the Test Results Analysis menu provides you with average scores of Core Subjects
for each grade. The comparison can be made using either the Grade Equivalent scores or Standard Scores.
Choose Consortium Averages
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 7
Part D. Comparing data within your school by using Disaggregation in iRM.
You can see a report of virtually any subset of your data. On the screen below you see that Disaggregation allows you to
choose a report by Gender, Race/Ethnicity, Programs, and Admin Codes. In the screen below we see a report of
Program-Other 1. Starting in 2011, Program: Other 1 was coded for Fulcrum Tuition Assistance Recipients. After
selecting a criteria, you must click Apply Criteria for the report to run and show properly. If there is only one criteria, it
will be labeled in the heading. In many cases you want to have multiple criteria, such as Race and Gender, this is possible
but the report labels it as Multiple and you must be careful to keep a record of the multiple criteria.
Use Reset to revert to All Students. Finally, only the first four of the Part A reports can be generated with Disaggregation.
The fifth report, Group Population Display is essentially already a disaggregation on Race and Gender.
Single criteria label
2. Click Apply Criteria after selection
1. Disaggregation: Program: Other 1 - Fulcrum Tuition
For this disaggregated report to be valid, the student answer sheets must be coded correctly with Race/Ethnicity data.
In addition, under the heading TEST ADMINISTRATOR USE ONLY are the Programs data, and Administrator Codes.
In the PROGRAM(S) box, our Consortium uses the following codes, (select all that apply):
• SE – (Special Education) – Students who have an Individual Education Program (IEP), as identified through the
local school district.
• TI L – Title I – Reading/Language
• TI M – Title I – Math
• ELL – English Language Learner
• Other 1 – Fulcrum Foundation (Current Recipient)
Complete the Code column (types of accommodations) and column A (Grade the student ENTERED).
Note: Do not enter and code in column Z without written consent of the OCS Consortium Director. We include all
students in testing and then can filter select populations per our supplemental coding.
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 8
Part E. Completing ISL Accreditation Appendix Templates
Use My Desk> ITBS Annual Test Data (Consortium)
Great News: Many ISL accreditation templates from Appendix B are fully populated in the ITBS database.
In the future, we will be populating some of the Appendix A forms, in the meantime you can find this information in
School Data Annual Reports or in the annual printed booklet of NCEA data. Log in to OCS website and go to My Desk >
Iowa/ITBS Annual Test Data (Consortia). Click on ISL Data to view the six menu options shown below.
Appendix B-1 Enrollment: This template is populated for you as shown below. The recommended way of using this in
your report is to copy and paste it into an Excel spreadsheet. It can then be formatted as you desire and either linked or
copied to your document. (Note: when copying a table, click at the start of the heading and be sure the highlighting
includes all the data before the copy command. Paste into any single cell and the table will be expanded properly.) A
second method is to take a screen shot and then crop and insert into your document. The most important task is for you
to address the questions based on your data.
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 9
Appendix B-2 Finances: This option as shown below; it provides three of the four tables requested for ISL. The fourth
table which you needed to construct on your own is your Annual Budget Surplus/Deficit for the past five years.
Appendix B-3 Quartiles: This option provides over twenty tables of data and meets the needs of the ISL template named
ISL Appendix B-3. You can copy the complete set and paste them into an Excel spreadsheet all at once. These templates
would take hours to construct, spend your time doing analysis not compilation.
Use slider to
see all tables
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 10
Appendix B-4: This template is being revised in the 2012 WCEA ISL update. During this transition we recommend that
you use chose some subtests from the Group Common Core Domain Display data (see page 2) and create a longitudinal
table from corresponding data found in this same display for previous years. Guiding questions could include:
• What is/could be the cause of the differences?
• What are the trends?
• What can be done to improve student learning?
• What impact will this have on the budget?
Appendix B-5 (5 year – Gender): This option provides over twenty tables of disaggregate data (i.e. gender) and meets
the needs of the ISL template B-5. You can copy the complete set and paste them into an Excel spreadsheet all at once
or you may want to select only a few tables. You have a choice to display the scores as GE or NPR. You can use the
consortium supplemental coding guide to disaggregate by ethnicity, ELL, Special Ed, etc.
Choice of
score type
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 11
Appendix B-5 (1 year Gender): This option is an alternative to the previous B-5 template. It provides a bar graph of
disaggregate data (i.e. gender) for a single grade across subjects. This graph cannot be copied and pasted to a
spreadsheet. We recommend that you use the screen-shot method to place it in a document. You can use the
consortium supplemental coding guide to disaggregate by ethnicity, ELL, Special Ed, etc..
Single grade
Other Links
Consortium Tab Link: http://www.ocsww.org/consortiums.php
New WCEA templates: www.westwcea.org (Click on “Team Forms” on the left-hand side)
Interactive Results Manager (iRM): http://irm30.rpclearning.com/
Contact for logins and passwords?: [email protected]
iRM Resources: http://www.riversidepublishing.com/products/irm/userresources.html
iRM technical Support: 800-323-9540 ext. 6070, or emailing: [email protected].
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 12
iRM Appendix: Sample steps in generating an iRM Report
Note that each new selection or criteria change automatically starts generating a report.
1. Select your Display Type
2. Select your Grade / Level
3. Select a Score Type
4. Group Display Report (Note the Export Tools: Print, PDF & Excel)
Have patience: these reports take several minutes to generate!
Rev 2013-11-30
ITBS Consortium Workbook – 2013
Page 13