HISTORY of the MEAP SCIENCE ASSESSMENT

Draft Two – Not for circulation
HISTORY of the MEAP SCIENCE ASSESSMENT
An Overview of MEAP
The Michigan Educational Assessment Program (MEAP) has been in existence since 1969. It was
initially established by the State Board of Education as a branch of the Department of Education
assigned to measure the extent to which Michigan students mastered specified essential skills in
education. MEAP tests were initially given each autumn to all fourth, seventh, and tenth grade students
are assessed on their achievement of essential skills in reading and mathematics. In addition, some of the
students were assessed in other subject areas (science, health, etc.) in conjunction with the basic MEAP
assessment. Science was first measured in 1974 and again in 1980 by administering an instrument
jointly developed by the Michigan Science Teachers Association and MEAP, to a small sample of
fourth, seventh, and tenth grade students. In all cases, the assessment results were to used by the State
Board of Education, school administrators, teachers, parents and other interested persons to chart the
status and progress of Michigan students in these content areas.
The MEAP assessment instruments were originally, and still are, criterion referenced. Initially, three
multiple-choice test items were used to measure each general objective. If the student correctly
answered two of the three items correctly, the objective is defined as being attained (mastered) by that
student. Since 1995 the methods used to establish proficiency are a little more complex, but the general
goal of establishing a level of proficiency across clearly stated learning goals still exists.
All MEAP exams were developed to measure pre-existing, state-wide, learning objectives. These
objectives were developed through consultation with Michigan professional associations, teachers, and
university educators. The resulting objective documents were published and distributed to schools from
1 to 3 years prior to any student assessment. At no time was any objective document declared
mandatory curriculum for the schools and teachers of Michigan. But, since the published objectives
formed the basis of the MEAP exams, schools that adapted their curriculum to meet the intent of the
objectives could possibly perform better on the exams.
The 1980 Science Assessment
The 1980 MEAP science assessment was administered to a representative sample of 5294 Michigan
students in the fourth, seventh, and tenth grades. Items for the
assessment were based on objectives contained in the 1980
Michigan State Board of Education document, Minimal
Performance Objectives for Science. While there was barely 9
months between the release of the objectives and the
administration of the test, this test was merely meant to be a
demonstration of the possibility of producing a broadly
administered test based upon mutually agreed upon science
objectives.
The objectives and resulting assessment items were strongly
influenced by the behaviorist theory of learning. Objectives were
clustered around two of Bloom’s three domains of learning,
affective and cognitive. The affective domain objectives were
called attitudes. This included things like “open-mindedness” and
-1-
Draft Two – Not for circulation
“questioning”. The cognitive domain
objectives were limited strictly to
processes, since this was all that the
behaviorist theory would recognize as
knowledge. Thus the focus was on
identified “science skills” such as
observing, classifying, and inferring
rather than any specific science
knowledge (as we understand knowledge
today).
Twenty of the objectives were measured
at the fourth grade level with 82% of the
students achieving attainment rates of
75% or better (Table 1). Nineteen
objectives were measured at the seventh
grade level. Sixty-six percent of the
seventh grade students in the sample
achieved attainment rates of 75% or
better. The tenth grade assessment
instrument measured 43 objectives across
all of the specified process skills. The
75% attainment rate was much lower
with 50% of the students reaching that
level. These results do not include
scoring on open ended items also used on
the assessment..
The interpretive report from the 1980
Figure 1 Page from the 1980 MPOS showing both
assessment notes that, "there is a
attitude and process objectives.
considerable body of scientific
knowledge appropriate to the school curriculum", that was not dealt with in an assessment on scientific
processes ( 1980-81 Science Education Interpretive Report, Michigan Educational Assessment Program,
1981, p 9). It was recommended that any future assessment include subject matter content as well as
scientific process skills. In other words, MEAP was encouraged to drop objectives and measurement
based solely on behaviorist theory of learning. This proved hard to do since the theory was well
ingrained in educational practice in Michigan.
Table 1
Percent of Michigan Students Achieving at the
75% Attainment Rate on the 1980 Science MEAP.
Process Skill
Overall
Percent of Students
4th
7th
Grade
Grade
82
66
-2-
10th
Grade
50
Draft Two – Not for circulation
The 1986 Science Assessment
Six years passed before the Michigan Department of Education was ready to attempt another all-student
MEAP assessment in science. Recognizing suggestions from the
1980 pilot assessment, the 1986 science assessment was designed
to have a balance between both process and content approaches to
measurement. The objectives, which were written prior to the
assessment, were less behaviorist influenced and stated more like
Intended Learning Outcomes (ILO) than objectives (see Figure 2).
However, emphasis was still placed on performance consistent
with behaviorist learning principles.
The new objectives were written by teachers and university
educators stating in 1983 and finally approved by the Michigan
State Board of Education in 1986. These were titled the Essential
Performance Objectives for Science Education, Grades K-9 (1986).
The objectives included subject matter content from life science,
physical science, earth and space science, and science, technology,
and society, as well as the science processes. Each content/skill
area was represented by between 4 and 15 general objectives that
were recommended for measurement at their respective grade
level. As in past assessments, not all the recommended objectives were measured in the 1986 MEAP
science assessment.
A MDE/MSTA sponsored test writing team of first grade through university level teachers wrote items
keyed to each objective during 1984 and 1985. These items were first tried out in a number of schools
that volunteered to give the tests. Then the items were pilot tested during the 1985 MEAP testing
period. Approximately 30,000 students participated in the pilot study. As a result of the pilot study,
some items were removed or rewritten in order to further clarify the assessment instrument.
The resulting MEAP science assessment instrument was administered to all fourth, seventh, and tenth
grade students as a part of the every-pupil MEAP given in the autumn of 1986. Over 300,000 students
participated in the assessment. The results, however, were less encouraging than the results of the 1980
sample assessment (Table 2).
Fourth Grade Results. A total of 30 objectives were measured at the fourth grade level. The minimum
passing score was again 75% of the objectives tested. This was achieved by just under 40% of the
105,00 students measured. Fourth grade students performed lowest on physical and earth and space
science items.
Seventh Grade Results. A total of 31 objectives were measured at the seventh grade level.
Approximately 27% of the 108,000 students measured passed the test. Again, student performance was
lowest on the physical science objectives. Other content areas also showed low performance rates.
Tenth Grade Results. A total of 32 objectives were measured at the tenth grade level. Just over 22% of
the 119,000 students measured passed the test. Physical and earth and space science objectives had the
lower attainment rates as was the case with the other grade levels assessed.
-3-
Draft Two – Not for circulation
The results of the 1986 MEAP
science assessment were announced
by the Superintendent of Public
Instruction in a news conference on
February 10, 1987. The poor score
results were coupled with a long
range plan for Michigan schools to
improve their science assessment
performance.
The third part of this Science
Curriculum Action Plan called for the
offering of the MEAP science
assessment instrument to schools on a
voluntary basis. For the following
three years, schools and school
districts that wanted to use the MEAP
science assessment did so by applying
to the Michigan Department of
Education. However, a random
sample of schools was selected and
administered the assessment
instrument on a non-voluntary basis.
These results were used to monitor
science achievement statewide.
Throughout this time, and until the
1995 assessment, the items on the
science MEAP remained constant.
Figure 2 Page from the 1986 EPOSE showing the
grade levels when each objective was to be
introduced (I), developed (D), and reinforced (R ).
The * indicated the level at which MEAP
assessment would occur.
Table 2
Percent of Michigan Students Achieving
the 75% Attainment Rate on the 1986 Science MEAP
Content Area
Overall
Percent of Students
4th
7th
Grade
Grade
39
27
-4-
10th
Grade
22
Draft Two – Not for circulation
Science Assessment Based on MEGOSE (1995-2001)
As soon as the 1986 MEAP science assessment was in place, the Michigan Department of Education
began setting the stage for the development of the next version of
the assessment (estimated to be in place within 5 years). Since
this new assessment was envisioned to be based less on the
behaviorist view of learning science and to be more in tune with
current research in science education and cognitive learning
theory, a new objectives document had to be produced.
Development of the objectives document alone took four years
before final approval by the State Board of Education in 1991.
Hundreds of educators from throughout Michigan were involved
in its direct development and/or review. The new document was
titled Michigan Essential Goals and Objectives in Science
Education (MEGOSE). While it described what all students
should be able to do by the end of
elementary, middle, and high school in
science, the subtitle for this document was,
New Directions for Science Education in
Michigan. MEGOSE was a significant
enough departure from previous practice,
that it was made available to teachers and
schools for three years prior to any
assessment based on it.
The MEGOSE writers used AAAS's
Project 2061: Science For All Americans
as their basis for development. In addition,
they divided the objectives (intended
learning outcomes) into three broad
categories of activities (not processes) that
were considered characteristic of
scientifically literate individuals. These
categories were USING scientific
knowledge, CONSTRUCTING new
scientific knowledge, and REFLECTING
on scientific knowledge.
The primary science content objectives
were organized under the USING category.
This was knowledge to be employed in
Figure 3 MEGOSE paragraphs describing
learning research for each objective.
-5-
Draft Two – Not for circulation
describing, predicting, explaining, and
controlling the environment about the
literate individual, rather than just
regurgitated facts for the sake of passing
tests. All life, physical, and earth/space
science objectives fell under the USING
category in MEGOSE.
The MEAP science assessment based on
MEGOSE took three years to develop.
The assessment model and item
development was entrusted to the
Michigan Science Teachers Association
and an advisory board composed of MDE
specialists, university educators and
teachers.
The test model evolved into something
quite different from past tests. Four test
sub-environments were proposed as a
way to best meet MEGOSE learning
vision, 1) Cluster problems–item groups
that presented a real-world situation
coupled to a series of four or five using,
reflecting and constructing questions
about it; 2) Investigation
problems–students were asked to
conduct an investigation ahead of time.
During the test period, they were to
answer a series of reflecting/constructing
Figure 4 MEGOSE objective tables showing
questions about the investigation; 3) Text
important concepts related to each objective and
criticism problems–students were to read
a passage from the popular press and then examples of real-world applications.
answer reflecting and constructing
questions about it; 4) Independent items–These were individual items similar to those on previous
MEAP science tests, except for the specific using knowledge in a real-world context each was set in.
While this was all happening, a change was made in the 10th grade MEAP science so that it became part
of the High School Proficiency Test, science testing was moved to the 5th, 8th, and 11th grades, and
enough items were written so that the test could change from year to year and past items be released.
Percent of Michigan Students Achieving
Proficiency on the 1995/96 Science MEAP
Content Area
Overall
5th
Grade
26.9
Percent of Students
8th
Grade
21.5
-6-
11th
Grade
32.0
Draft Two – Not for circulation
MEGOSE to Frameworks and Benchmarks
Starting in 1998, the MDE began a program to consolidate the objective documents for mathematics,
science, reading, and writing into a consistent format. The result
was the Michigan Curriculum Frameworks, a single document that
specified learning (and assessment) goals across all of the content
areas measured by MEAP. The objective tables in MEGOSE were
translated into Framework tables with little loss of language. But
while consistency was achieved, it was done at the expense of
much clarifying information about the intent of the objectives that
existed in paragraph form in MEGOSE. For many years, the
missing information was only available to those who possessed a
rare copy of MEGOSE. The MiCLIMB CD, released in 2000 by
the MDE, has helped to partially correct this deficiency.
As part of the translation of the MEGOSE objectives to the
Frameworks tables, the objectives were renamed benchmarks.
This was purely a renaming process, not a rewrite or a change of
intent. There are several reasons why benchmarks is actually a
better term for what were originally called objectives. First, at no
time were Michigan objectives written in correct behavioral
objective format. At best the objectives were written as Intended Learning Outcomes. At other times
they were just lists of content ideas to be covered in the curriculum. In addition, Michigan objectives
were never easily translatable to classroom practice. Teachers always had to subdivide and secondguess the intent of the objectives. Since the objectives were written to cover a 3-4 year period of
instruction, it is best to refer to them as benchmarks. Benchmarks represent goals to attained at the end
of that 3-4 year period. Achieving the benchmarks should be synonymous with attaining proficiency on
the MEAP tests.
Other changes have occurred since the publication of the Curriculum Framework. Starting in 2000,
Executive Order No. 1999-12 transferred MEAP from the Department of Education to the Department
of Treasury. This essentially separated the organization that generated the standards from the
organization that measured the standards. For the time being, this has made no difference since the
Michgan Curriculum Frameworks are still the standard upon which all assessment is performed.
More recently at the federal level, the Elementary and Secondary Education Act of 2001( No Child Left
Behind) was passed and signed into law. States were required to establish learning benchmarks in math
and English and begin testing students in grades 3-8 annually in reading and mathematics. Since the
tests must be aligned with state academic standards, grade level benchmarks were written for these
content areas. They were also written for science (Lang). It remains to be seen how these science grade
level benchmarks will be employed since they are not mandated by federal law.
Percent of Michigan Students Achieving
Proficiency on the 2001/02 Science MEAP
Content Area
Overall
Percent of Students
5th
8th
Grade
Grade
77.7
70.0
-7-
11th
Grade
49.5
Draft Two – Not for circulation
References
American Association for the Advancement of Science (1989) Science For All Americans, Project 2061.
Washington, D.C.: AAAS, 1989
Burns, M. (January 22, 1998). Interpreting the Reliability and Validity of the Michigan Educational
Assessment Program. Fact Finding on the Michigan Educational Assessment Program: Standing
Committee of the Michigan Association of School Psychologists.
Coleman, G. J., Ed. (1983). Questions and Answers about the Michigan Educational Assessment
Program. Lansing, Michigan: Michigan State Board of Education.
Education, M. S. B. o. (1980). Minimal Performance Objectives for Science. Lansing, Michigan:
Michigan Department of Education.
Education, M. S. B. o. (1986). Essential Performance Objectives for Science Education, Grades K-9.
Lansing, Michigan: Michigan Department of Education.
Education, M. S. B. o. (1991). Michigan Essential Goals and Objectives for Science Education (K-12):
New Directions for Science Education in Michigan. Lansing, Michigan: Michigan Department of
Education.
Epp, R. H. (2002). Student Performance Standards for Michigan's Science Test. MSTA Journal, 47(2).
General, M. O. o. t. A. (1999). Performance Audit of the Michigan Educational Assessment Program.
Lansing, Michigan: Michigan Office of the Auditor General.
Johnson, K. A. (2002, May 28, 2002). POLICY BRIEF: Which Educational Achievement Test is Best for
Michigan? A Comparison of the MEAP, SAT-9, and ITBS. Retrieved August 15, 2003, from
http://www.mackinac.org/print.asp?ID=4382
Laboratory, N. C. R. E. (1994, 1994). Building Collaborative Education Systems: New Roles for State
Education and Higher Education Agencies. Retrieved August 12, 2003, from
http://www.ncrel.org/sdrs/pbriefs/94/94-1mi.htm
Lang, M. P. (2002). Grade-Level Academic Achievement Benchmarks in Science (GLAABS). MSTA
Journal, 47(2).
Roberts, D. B. e. a. (1986). Michigan Educational Assessment Handbook. Lansing, Michigan: Michigan
State Board of Education.
Smith, E. (1998). Fact Sheet: Science Education Reform in Michigan–Did you know? Retrieved August
20, 2003, from http://ed-web3.educ.msu.edu/cvsme/
Yarroch, W. L. (1988, February 19-20, 1988). Future MEAP Science Assessments: A Proposal. Paper
presented at the thirty-fifth annual conference of the Michigan Science Teachers Association,
Lansing, Michigan.
-8-