An Evaluation of the Second Edition of UCSMP Algebra

Transcription

An Evaluation
of the Second Edition of
UCSMP Algebra
University of Chicago School Mathematics Project
An Evaluation
of the Second Edition of
UCSMP Algebra
Denisse R. Thompson
University of South Florida
Sharon L. Senk
Michigan State University
David Witonsky
Zalman Usiskin
University of Chicago
Gurcharn Kaeley
2006 by University of Chicago School Mathematics Project
Chicago, IL
ii
Table of Contents
Overview of the Evaluation Report…………………………………………………… ..1
Chapter 1
Background for the Study………………………………………………. 3
Calls for Curricular Reform……………………………………………………………... ..3
A Brief Overview of the University of Chicago School Mathematics Project ………….. .6
A Brief Overview of the Secondary Component of UCSMP…………………………….. 7
A Brief Description of UCSMP Algebra………………………………. ..........................11
Chapter 2
Design of the Study…………………………………………………….. 15
Research Questions……………………………………………………………………… 15
Procedures……………………………………………………………………………….. 17
Instructional Materials……………………………………………………………………20
Instruments………………………………………………………………………………. 24
Description of the Samples……………………………………………………………… 28
Chapter 3
The Implemented Curriculum and Instruction……………….……... 33
Content Coverage………………………………………………………………………...33
Instructional Practices and Issues……………………………………………………….. 37
Summary………………………………………………………………………………… 49
Chapter 4
The Achieved Curriculum…………………………………………….. .51
Achievement on the High School Subject Tests: Algebra………………………………. 51
Achievement on the UCSMP Algebra Test………………………………. ......................57
Achievement on the Problem-Solving and Understanding Test…………………………74
Summary………………………………………………………………………………… 84
Chapter 5
Attitudes………………………………………………………………… 87
Students’ Attitudes………………………………………………………………………. 87
Teachers’ Attitudes…………………………………………………………………….. 106
Summary……………………………………………………………………………….. 109
Chapter 6
Summary and Conclusions…………………………...……………….113
The Implemented Curriculum …………………………………………………………. 115
The Achieved Curriculum………………………………………………………………117
Attitudes………………………………………………………………………………... 119
Changes Made for Commercial Publication…………………………………………… 121
Conclusions and Discussion…………………………………………………………….122
References…………………………………………………………………………… ...125
Appendix A Participation Information and Guidelines .......................................... A-1
Description of Requirements for Participation………………………………………… A-3
School Information Form……………………………………………………………….A-4
iii
Appendix B Textbook Tables of Contents ................................................................B-1
UCSMP Algebra (Second Edition, Field Trial Version) .................................................B-3
UCSMP Algebra (Second Edition, Commercially Published Version).........................B-13
UCSMP Algebra (First Edition) ....................................................................................B-21
Appendix C Instruments .............................................................................................C-1
UCSMP Algebra Test ......................................................................................................C-3
UCSMP Problem-Solving and Understanding Test (Odd Form) ..................................C-15
UCSMP Problem-Solving and Understanding Test (Even Form).................................C-20
Fall Survey of Opinions About Mathematics ................................................................C-25
Spring Student Opinion Survey .....................................................................................C-26
Teacher Survey ..............................................................................................................C-29
Chapter Evaluation Form ...............................................................................................C-30
Appendix D Rubrics and Sample Student Responses ............................................. D-1
Problem-Solving and Understanding Test: Rubrics ....................................................... D-3
Problem-Solving and Understanding Test: Sample Student Responses and Scores .... D-12
Appendix E Classroom Observation Report Form and Interview Schedule ........ E-1
Rationale and Suggested Strategy for Site Visits ............................................................ E-3
Classroom Observation Report Form .............................................................................. E-5
Teacher Interview Schedule............................................................................................. E-9
iv
List of Tables
Chapter 2
Table 1
Chapter Titles for Each of the Textbooks Used in the Study
21
Table 2
General Scoring Rubric: Problem-Solving and Understanding Test
26
Table 3
Number (Percent) of Students in the Second Edition and First
Edition Sample
29
Table 4
Pretest Means, by Matched Pair: Second Edition and First Edition
30
Table 5
Number (Percent) of Students in the Second Edition and
non-UCSMP Sample
31
Pretest Means, by Matched Pair: Second Edition and non-UCSMP
32
Days Spent on Each Chapter of the Second Edition, Including
Testing, by Teachers in the Second Edition and First Edition
Sample
34
Days Spent on Each Chapter of the Second Edition, Including
Testing, by Teachers in the Second Edition and non-UCSMP
Sample
36
Percent of Students Reporting Levels of Use of Calculators:
Second Edition and First Edition
39
Percent of Students Reporting Levels of Use of Calculators:
Second Edition and non-UCSMP
40
Percent of Students Indicating Frequency of Reading Textbook
Explanations: Second Edition and First Edition
42
Percent of Students Indicating Frequency of Reading Textbook
Explanations: Second Edition and non-UCSMP
43
Percent of Students Spending Various Amounts of Time on
Homework and Needing Various Levels of Help With Their
Homework: Second Edition and First Edition
45
Table 6
Chapter 3
Table 7
Table 8
Table 9
Table 10
Table 11
Table 12
Table 13
v
Table 14
Percent of Students Spending Various Amounts of Time on
Homework and Needing Various Levels of Help With Their
Homework: Second Edition and non-UCSMP
47
Mean Percent Correct and Teachers’ Reported OTL on the Content
on the High School Subject Tests: Algebra
52
Mean Percent Correct on the Fair Tests from the High School
Subject Tests: Algebra
55
Mean Percent Correct on the Two Conservative Subtests of the
High School Subject Tests: Algebra
56
Mean Percent Correct and Teachers' Reported OTL on the
Content of the UCSMP Algebra Test
58
Mean Percent Correct on the Fair Tests from the UCSMP Algebra
Test
60
Mean Percent Correct on the Two Conservative Tests from the
UCSMP Algebra Test
61
Percent Successful on the UCSMP Algebra Test by
Item and Content Strand: Second Edition and First Edition
68
Percent Successful on the UCSMP Algebra Test by
Item and Content Strand: Second Edition and non-UCSMP
72
Mean Score on the Odd Form of the Problem-Solving and
Understanding Test
75
Mean Score on the Even Form of the Problem-Solving and
Understanding Test
76
Item Means (Standard Deviations) for the Odd Form of the
Problem-Solving and Understanding Test: Second Edition and First
Edition
78
Item Means (Standard Deviations) for the Odd Form of the
Problem-Solving and Understanding Test: Second Edition and nonUCSMP
80
Item Means (Standard Deviations) for the Even Form of the
Problem-Solving and Understanding Test: Second Edition and First
Edition
81
Chapter 4
Table 15
Table 16
Table 17
Table 18
Table 19
Table 20
Table 21
Table 22
Table 23
Table 24
Table 25
Table 26
Table 27
vi
Table 28
Item Means (Standard Deviations) for the Even Form of the
Problem-Solving and Understanding Test: Second Edition and nonUCSMP
83
Percentages of Students Agreeing or Disagreeing on the
Student Opinion Survey to Items Dealing with Attitudes Toward
Mathematics as a Discipline: Second Edition and First Edition
89
Student Opinion Survey to Items Dealing with Attitudes Toward
Mathematics as a Discipline: Second Edition and non-UCSMP
91
Student Opinion Survey to Items Dealing with Confidence Toward
Mathematics: Second Edition and First Edition
93
Student Opinion Survey to Items Dealing with Confidence Toward
Mathematics: Second Edition and non-UCSMP
94
Student Opinion Survey to Items Dealing with Calculators: Second
Edition and First Edition
96
Student Opinion Survey to Items Dealing with Calculators: Second
Edition and non-UCSMP
98
Chapter 5
Table 29
Table 30
Table 31
Table 32
Table 33
Table 34
Table 35
Table 36
Table 37
Table 38
Student Opinion Survey to Items Dealing with Their
Mathematics Course: Second Edition and First Edition
100
Student Opinion Survey to Items Dealing with Their
Mathematics Course: Second Edition and non-UCSMP
101
Student Opinion Survey to Items Dealing with Their Mathematics
Textbook: Second Edition and First Edition
103
Student Opinion Survey to Items Dealing with Their Mathematics
Textbook: Second Edition and non-UCSMP
105
vii
List of Figures
Figure 1
Stems of UCSMP Algebra Test Items by Content Strand
viii
64
OVERVIEW OF THE EVALUATION REPORT
This report describes the Field Test of the Second Edition of Algebra, published
by the University of Chicago School Mathematics Project (UCSMP), and reports its
results, including the effects of changes made for the Second Edition on students'
achievement and attitudes. The evaluation report consists of six chapters.
Chapter 1 provides some background to the study, including information about
major reform initiatives and recommendations in mathematics education that influenced
the development of the curriculum. In addition, this chapter describes the University of
Chicago School Mathematics Project in general and gives details about the major aims of
Algebra. Chapter 2 describes the design of the study, including the two study samples: (1)
UCSMP Second Edition and UCSMP First Edition; and (2) UCSMP Second Edition and
non-UCSMP comparison classes.
The major results of the study are discussed in Chapters 3 - 5, with each chapter
focusing on a particular aspect of curriculum or instruction. Chapter 3 focuses on the
implemented curriculum and instruction, Chapter 4 focuses on the achieved curriculum,
and Chapter 5 focuses on attitudes and opinions of students and teachers. Results from
both samples are discussed in each chapter. Chapter 6 summarizes Chapters 1 - 5,
contrasts results from the two samples, and draws conclusions about the effectiveness of
the materials.
Instruments used in the study, rubrics for scoring open-ended items, and tables of
contents for the textbooks are provided in the Appendices.
1
2
CHAPTER 1
BACKGROUND FOR THE STUDY
In the years 1975-1983, numerous publications, in both the professional arena and
the popular press, focused on the state of education in the United States and the need for
mathematics curricular reform. It was in this context, in 1983, that the University of
Chicago School Mathematics Project (UCSMP) began its curricular work. As UCSMP's
work continued, other national groups continued to make recommendations regarding
school mathematics. This chapter provides background for understanding the curricular
materials developed by UCSMP, and in particular, the development of the algebra course
that is the subject of this report.
The first section of this chapter contains a brief overview of some of the major
reports and recommendations for mathematics curricular reform from the latter part of the
20th century and describes the educational climate in which UCSMP conducted its work.
The second section provides a brief overview of the entire University of Chicago School
Mathematics Project. The third section describes the Secondary Component of UCSMP,
including features that are common to all the secondary curriculum materials. The final
section discusses problems and issues specifically addressed by Algebra.
Calls for Curricular Reform
Reports recommending reform in mathematics education in the final quarter of
the 20th century came from both within and outside the mathematics education
community. Those from within tended to focus solely on mathematics curriculum and
instruction. Those from outside focused on broad issues of educational reform and made
recommendations about the mathematics curriculum within the context of broader
educational concerns.
Among the earliest reports from this era were Overview and Analysis of School
Mathematics: Grades K-12 (National Advisory Committee on Mathematical Education
1975) and An Agenda for Action: Recommendations for School Mathematics of the 1980s
(National Council of Teachers of Mathematics [NCTM] 1980). Both reports were
developed by the mathematics education community and recommended major changes in
the curriculum, including an emphasis on applications of the mathematics being studied,
appropriate use of available technology, and an updating of content to reflect important
new areas of mathematics. They also reflected a concern for the mathematical preparation
of all students.
Reports prepared by broad-based educational commissions, such as A Nation at
Risk (The National Commission on Excellence in Education 1984) and Educating
Americans for the 21st Century (The National Science Board Commission on Precollege
Education in Mathematics, Science and Technology 1983), reiterated these
recommendations and focused on the importance of mathematical literacy for the
continued well-being of the country. For instance, Educating Americans for the 21st
3
Century states “... America’s security, economic health and quality of life are directly
related to the mathematics, science and technology literacy of all its citizens” (p. 12).
In addition to broad reflections on mathematics education, these reports made
specific recommendations about the mathematics curriculum. For instance, A Nation at
Risk states:
The teaching of mathematics in high school should equip graduates to:
(a) understand geometric and algebraic concepts;
(b) understand elementary probability and statistics;
(c) apply mathematics in everyday situations; and
(d) estimate, approximate, measure, and test the accuracy of their calculations.
In addition to the traditional sequence of studies available for college-bound
students, new, equally demanding mathematics curricula need to be developed for
those who do not plan to continue their formal education immediately. (p. 25)
The importance of preparing all students was echoed in Educating Americans for the 21st
Century as this report expressed the “need to expand the focus of mathematics, science
and technology education from only the pre-professional to all students. ... Discrete
mathematics, elementary statistics and probability should now be considered fundamental
for all high school students” (pp. 41-43).
The College Board, in its 1983 report Academic Preparation for College,
reinforced this emphasis on a broad curriculum in mathematics by recommending that all
students, college-bound or not, should possess the following skills:
•
•
•
•
•
The ability to apply mathematical techniques in the solution of real-life
problems and to recognize when to apply those techniques;
Familiarity with the language, notation, and deductive nature of mathematics
and the ability to express quantitative ideas with precision;
The ability to use computers and calculators;
Familiarity with the basic concepts of statistics and statistical reasoning;
Knowledge in considerable depth and detail of algebra, geometry, and
functions. (p. 20)
These reports from the late 1970s through the mid-1980s provided a relatively
consistent message about the nature of the changes needed in the school mathematics
curriculum in order for educators to prepare students for the workplace of the 21st
century. The various recommendations were embedded in the guidelines for a redesign of
the mathematics curriculum published in the Curriculum and Evaluation Standards for
School Mathematics (National Council of Teachers of Mathematics 1989).
Educational goals for students must reflect the importance of mathematical
literacy. Toward this end, the K-12 standards articulate five general goals for all
students:
• that they learn to value mathematics;
• that they become confident in their abilities to do mathematics;
• that they become mathematical problem solvers;
• that they learn to communicate mathematically; and
• that they learn to reason mathematically. (p. 5)
4
The document provided specific recommendations for four process standards (problemsolving, reasoning, communication, and connections) and eight or nine content standards
at each of three grade-level ranges: K-4, 5-8, and 9-12.
The broader mathematics community also published two reports at roughly the
same time. Everybody Counts (National Research Council 1989) discussed the critical
role that mathematics plays in the future career options of students and noted the poor
course-taking habits of many students.
More than any other subject, mathematics filters students out of programs leading
to scientific and professional careers. From high school through graduate school,
the half-life of students in the mathematics pipeline is about one year; on average,
we lose half the students from mathematics each year. ... (p. 7)
Perhaps in response to the declining enrollment in mathematics, Reshaping
School Mathematics: A Philosophy and Framework for Curriculum (Mathematical
Sciences Education Board 1990) reiterated the importance of redesigning the curriculum
in the following statement:
[T]he United States must restructure the mathematics curriculum - both what is
taught and the way it is taught - if our children are to develop the mathematical
knowledge (and the confidence to use that knowledge) that they will need to be
personally and professionally competent in the twenty-first century. ... What is
required is a complete redesign of the content of school mathematics and the way
it is taught. (p. 1)
Recognizing that change in curriculum is just one piece of reform, the National
Council of Teachers of Mathematics also recommended changes in the nature of
mathematics instruction in the classroom. Among the recommendations in the
Professional Standards for Teaching Mathematics (NCTM 1991) were that teachers
engage students in worthwhile mathematical tasks and rich classroom discourse and that
they use tools such as manipulatives as well as computers and calculators whenever
appropriate.
The third and final document in the trilogy of standards recommendations from
NCTM, the Assessment Standards for School Mathematics (1995), focused on the
importance of broadening the assessment of learning beyond information gained solely
from timed, on-demand tests. This document recommended aligning assessment with
instruction and encouraged the use of open-ended tasks, projects, etc., particularly when
students used curriculum materials emphasizing these approaches and strategies.
Taken together, these reports, published between 1975 and the mid-1990s,
provided a rather consistent message about the state of mathematics education and
recommendations for change: update the content of school mathematics; emphasize
realistic uses of mathematics; use technology to support learning; and make classrooms
learning environments in which students work collaboratively to explore mathematics. It
was in this educational climate of mathematics education reform that the University of
Chicago School Mathematics Project was founded and began its work. The overall nature
of UCSMP is the subject of the next section.
5
A Brief Overview of The University of Chicago School Mathematics Project
The University of Chicago School Mathematics Project (UCSMP) developed
from conversations between Izaak Wirszup of the University of Chicago and Keith
McHenry of the Amoco Corporation. In 1983 the Departments of Mathematics and
Education received a generous six-year grant from the Amoco Foundation for a multifaceted project to improve mathematics education for the vast majority of students in
grades K-12. Paul Sally of the Department of Mathematics was named Director of
UCSMP.
Given the overall consensus on the problems facing mathematics education and
the recommendations for change in pre-college mathematics instruction, the leaders of
the UCSMP decided not to attempt to create another set of recommendations for
curriculum. Rather, the project undertook the task of trying to translate the existing
recommendations into the reality of curriculum materials for classrooms and schools.
In addition to grants from the Amoco Foundation, the UCSMP also received
funding from the National Science Foundation, the Ford Motor Company, the Carnegie
Corporation of New York, the General Electric Foundation, the GTE Corporation, and
Citicorp/Citibank. Since 1990, royalties from the commercial publications of the UCSMP
materials have enabled the project to provide inservice opportunities for teachers using
the materials and to fund additional research in mathematics education.
Since its inception, the work of the UCSMP has been conducted by several
independent, yet interconnected, components, each with its own director(s) and staff.
(See the annual University of Chicago School Mathematics Project brochure for more
details on the various components of the project. See also Usiskin (1986/87, 2003) for
more about the history of the UCSMP.)
The Resource Development Component, directed by Izaak Wirszup, has
translated school mathematics publications from around the world, offering educators a
first-hand look at expectations, approaches, and methodologies differing from those in
the United States. This component has organized four international conferences on
mathematics education, the latest in August 1998. The work of the Resource Component
has enabled UCSMP to learn of achievement standards in other countries, encouraging
the project to expect more of students and influencing the development of curriculum
materials for all grades.
The Elementary Component, directed by Max Bell, has developed a sequenced set
of curriculum materials from Kindergarten through Grade 6. These materials are rich in
content, using results from international studies to build a curriculum that takes
advantage of early experiences and capabilities of young children. The curriculum, titled
Everyday Mathematics, aims to help children make the gradual transition from intuition
and concrete operations to abstractions and symbol-processing skills.
The Teacher Development Component, a part of the Elementary Component
directed by Sheila Sconiers, designed several professional development programs to
support mathematics reform in Grades K-6. First, a package of monthly workshops,
called Math Tools for Teachers, was created to enable classroom teachers to conduct staff
development workshops for their colleagues. Second, the UCSMP Mathematics
6
Specialist Program was designed and conducted for several years to prepare specialist
teachers for mathematics in Grades 4-6 and to reduce the number of teachers needing
staff development.
The Secondary Component, co-directed by Zalman Usiskin and Sharon Senk, has
developed a six-year mathematics curriculum for students in Grades 7-12. First editions
of these materials were developed, tested, and refined from 1983 through 1991.
Beginning in 1992, the component worked on the second editions of the materials. The
UCSMP secondary curriculum transforms high school mathematics into a mathematical
sciences curriculum, covering a broad range of material important for life in a
technological society. This curriculum targets the general high school population —
students who will graduate from high school — and conveys the essential role of
mathematics in everyday life by teaching students to use mathematics effectively.
Finally, during the initial development of the curriculum materials, the Evaluation
Component, co-directed by Larry Hedges and Susan Stodolsky, conducted field-tests of
the materials to aid in their development and to determine their effectiveness in the realworld reality of the mathematics classroom. Those evaluations included formative
evaluations in the first year or two of development and national summative evaluations in
later years of development. In addition to student achievement, evaluation has examined
actual classroom use of materials.
Since the commercial publication of the materials, tens of millions of students
have used the UCSMP elementary and secondary materials. Teachers of many other
students, along with teacher educators, have participated in UCSMP teacher development
programs or attended UCSMP conferences.
The next section describes the Secondary Component, responsible for developing
all of the materials for Grades 7-12, including Algebra, the course that is the subject of
this report.
A Brief Overview of the Secondary Component of UCSMP
When the project began, three general problems in mathematics education in the
United States led to three major goals of the UCSMP secondary mathematics curriculum.
Although progress on these problems has been made over the last two decades of the 20th
century, they remain of concern to mathematics educators at the beginning of the 21st
century.
First, students do not learn enough mathematics by the time they leave school.
Specifically, many students lack the background to succeed in college, on the job, or in
daily affairs. They often are not introduced to applications of mathematics or to problems
requiring thought before answering. They terminate their study of mathematics too early
and do not learn to become independent learners capable of acquiring mathematics
outside of school when the need arises. Hence, one goal of UCSMP has been to upgrade
students’ achievement.
Second, the school mathematics curriculum has not kept up with changes in
mathematics and the ways in which mathematics is used. For instance, many curricula
have not taken advantage of present calculator and computer technology. Although
7
students may be prepared for calculus, they are often unprepared for other mathematics
they may encounter in college. Statistics and discrete mathematics are often missing from
the curriculum, as are applications and estimation/approximation techniques. Hence, a
second goal of UCSMP has been to update the mathematics curriculum.
Third, too many students have not taken the mathematics needed for employment
and further schooling. Tracking has made it easy for students to go down levels, but not
up; remedial programs cause students to get further behind. Enrichment classes often
contain important topics from statistics or discrete mathematics that are useful for all
students. Preset standards and numbers may limit some students’ opportunities to explore
higher levels of mathematics. Hence, a third goal of UCSMP has been to increase the
number of students who take mathematics beyond algebra and geometry.
A number of basic elements are common throughout all of the secondary
materials. Materials have wide scope, with some geometry, algebra, and discrete
mathematics in all courses and with statistics/probability integrated into the study of
algebra and functions. Reading and problem solving are evident as students are expected
to read the lessons and answer questions pertaining to the reading and are exposed to a
variety of problem-solving methods throughout the text. Applications of the mathematics
being studied are embedded throughout, providing opportunities for the development of
skills and an understanding of the importance of mathematics in everyday life. The
presence of technology is assumed, with scientific calculators expected in all courses and
graphing calculators assumed in the last three courses. Automatic drawing tools are
expected in the study of Geometry and statistical software is expected in Functions,
Statistics, and Trigonometry.
In addition to the content elements, several instructional elements are also
common throughout. The curriculum materials emphasize a multidimensional approach
to understanding, with a balanced view of skills, properties, uses (applications), and
representations (or picturing) of concepts. A modified mastery approach features
learning by continual review, with review questions from previous lessons included in
each problem set and in subsequent chapters and with end-of-chapter materials including
summary, self-test, and review questions keyed to objectives. Projects in the last two
courses of the first-edition materials and in all courses of the second-edition materials
offer students an opportunity to work on an extended topic over a period of time and to
explore that topic in some depth.
The UCSMP Secondary curriculum consists of six courses: Transition
Mathematics; Algebra; Geometry; Advanced Algebra; Functions, Statistics, and
Trigonometry; and Precalculus and Discrete Mathematics. These six courses are
appropriate for average to above-average students beginning in the seventh grade and
proceeding one course a year through twelfth grade. By starting earlier or later, or taking
the first two courses at a slower pace, the curriculum sequence has accommodated a
range of students. A goal of the developers of the UCSMP Secondary curriculum has
been to have every high school graduate take the first four courses (i.e., through
Advanced Algebra), all college-bound students take the first five, and all students who
may study technical subjects take all six courses, or their equivalents. The six courses are
briefly described below.
8
Transition Mathematics (Year 1) weaves together three themes — applied
arithmetic, prealgebra, and pregeometry — by focusing on arithmetic operations in
mathematics and the real world. Variables are used as pattern generalizers, abbreviations
in formulas, and unknowns in problems, and are represented on the number line and
graphed in the coordinate plane. Basic arithmetic and algebraic skills are connected to
corresponding geometry and measurement topics.
Algebra (Year 2) has a scope far wider than most traditional algebra books.
Applications motivate all topics. Exponential growth and compound interest are covered.
Statistics and geometry are settings for work with linear expressions and sentences.
Probability provides a context for algebraic fractions, functions, and set ideas.
Considerable attention is given to graphing. Manipulation with rational algebraic
expressions is, however, delayed until later courses.
Geometry (Year 3) significantly diverges from the order of topics in most
geometry texts, presenting coordinates, transformations, measurement formulas, and
three-dimensional figures earlier in the year. Work with proof-writing follows a carefully
sequenced development of the logical and conceptual precursors to proof.
Advanced Algebra (Year 4) emphasizes facility with algebraic expressions and
forms, especially linear and quadratic forms, powers and roots, and functions based on
these concepts. Students study logarithmic, trigonometric, polynomial, and other special
functions both for their abstract properties and as tools for modeling real-world
situations. The course applies geometrical ideas learned in the previous years, including
transformations and measurement formulas.
Functions, Statistics, and Trigonometry (Year 5) integrates statistical and
algebraic concepts and previews calculus in work with functions and intuitive notions of
limits. Students study both descriptive and inferential statistics, combinatorics, and
probability; they also do further work with polynomial, exponential, logarithmic, and
trigonometric functions. Enough trigonometry is available to constitute a standard
precalculus background in trigonometry and circular functions. Throughout the course,
students use computers or calculators to study functions, explore relationships between
equations and their graphs, analyze data, and develop limit concepts.
Precalculus and Discrete Mathematics (Year 6) integrates the background
students must have to be successful in calculus with the discrete mathematics helpful in
computer science. Precalculus topics include a review of the elementary functions,
advanced properties of functions (including special attention to polynomial and rational
functions), polar coordinates, complex numbers, and introductions to the derivative and
integral. Discrete mathematics topics include recursion, mathematical induction,
combinatorics, vectors, graphs, and circuits. Manipulation of rational expressions is
studied. Mathematical thinking, including specific attention to formal logic and proof and
comparing structures, is a unifying theme throughout.
The first edition of each text was developed in stages spanning four or five years.
In the planning stage, overall goals for the courses were developed through consultation
with a national advisory board of distinguished mathematics educators and through
discussion with classroom teachers, school administrators, and district and state
supervisors.
9
At the pilot stage, the co-directors of the Secondary Component selected authors
to write drafts of the course. Half of all UCSMP authors were teaching mathematics in
secondary schools when they wrote the materials, and all of the authors and editors for
the first five courses had secondary school teaching experience. At the pilot stage,
authors or teachers they knew taught the first draft of the materials.
After revision by the authors and editors, the materials entered a formative stage
of development in which more classes used the materials, teachers met periodically at the
university to provide feedback to authors for revisions, and independent evaluators
monitored achievement and attitudes. For the first three books, national field tests were
conducted comparing performance with UCSMP materials to performance with
traditional materials. The last three books underwent formative evaluations.
The national field trials showed that students using early editions of UCSMP
middle school materials performed at least as well as their counterparts on traditional
skill content and outperformed them on new content and on applications of mathematics
(Hedges, Stodolsky, Mathison, & Flores 1986; Mathison, Hedges, Stodolsky, Flores, &
Sarther 1989). A longitudinal study of students who had completed four years of early
versions of UCSMP materials found that UCSMP students outperformed comparison
students at two sites, on both traditional content and applications. However, at the third
site, comparison students outperformed UCSMP students on a standardized test
(Hirschhorn 1993).
In 1992, the UCSMP Secondary Component began planning for the second
editions of the curriculum materials, using the results of the research conducted on the
first editions and information gathered from the many users of the commercially
published materials. The second editions were developed by a combination of firstedition authors, experienced users, and new authors. Revisions for the first four books
underwent a field test. Initially, only minor changes were planned for the last two books,
so no field tests were planned. However, because of the influence of technology and data
updates, a decision was made to review all lessons. Experienced users were polled for
information to guide revisions.
The second editions were revised to include new emphases on student writing and
projects as part of broader assessment measures as outlined in the Curriculum and
Evaluation Standards, the Professional Standards for Teaching Mathematics, and the
Assessment Standards (NCTM 1989, 1991, 1995). The increased use of technology,
particularly the widespread use of graphing calculators, had a major influence on the last
three courses with the assumption that students in all three courses would have continual
access to a graphing utility.
10
A Brief Description of UCSMP Algebra
Algebra is the second of the six courses in the secondary curriculum developed by
the UCSMP. The First Edition of Algebra was developed in response to seven problems
that UCSMP did not believe could be resolved by minor changes in traditional content or
approach. First, large numbers of students do not see why they need algebra. In response,
applications of algebra are used throughout the curriculum to motivate the development
of concepts and skills. Word problems with little use in the real world (e.g., coin
problems and age problems) are replaced by meaningful problem types. In addition,
algebra is connected to the arithmetic students already know and to geometry that
students will study in the future.
Second, the mathematics curriculum has been lagging behind today's widely
available and inexpensive technology. In response, Algebra assumes the availability of a
scientific calculator from the first chapter. The use of the calculator permits teachers to
address more content because students are not bogged down in difficult calculations.
Further, use of the calculator permits the use of realistic applications in which numbers
and answers are not integers. Some new content important in a computer age is included
in the algebra curriculum, such as discrete and continuous domains as well as the
interpretation of graphs.
Third, too many students fail algebra. The UCSMP response is to spread out
important algebra concepts, with some ideas such as variables as unknowns and as
pattern generalizers introduced in Transition Mathematics and others such as
concentrated work with polynomial and rational expressions delayed until later UCSMP
courses. Evidence from UCSMP studies has shown that students studying from UCSMP
Transition Mathematics knew more algebra at the end of the school year than students in
comparison classes. Although students using UCSMP Algebra will not necessarily have
studied from Transition Mathematics, we expect better performance from those who have
had the rich experiences provided by that course.
Fourth, even students who succeed in algebra often do poorly in geometry.
Because one of the best predictors of success with geometry and proof is the amount of
geometry knowledge students possess at the beginning of a geometry course, students in
Algebra continue the study of geometry concepts that was begun in Transition
Mathematics. In particular, students study numerical relationships with lines, angles, and
polygons.
Fifth, students don't read. In response, Algebra contains material in each lesson
that students are expected to read, with careful attention paid to explanations, examples,
and questions so that students learn to use their textbook as a resource for information. In
addition, each lesson contains questions about the reading.
Sixth, high school students know very little statistics and probability. UCSMP
includes a considerable amount of statistics in two courses: Algebra and Functions,
Statistics, and Trigonometry. Because statistics begins with data, the Algebra course
includes data throughout the text, with students regularly expected to graph, organize,
and interpret data. Probability concepts are also included throughout as appropriate.
11
Seventh, students are not skillful enough, regardless of what they are taught. To
help students become skillful at problems other than non-routine problems, there are
many problems with complicated numbers, various wordings, and a variety of contexts so
that students learn to apply their skills in many situations.
In 1987-88, the Evaluation Component of UCSMP conducted a national study of
Algebra with 40 matched pairs of classes in 9 states, half of which used the UCSMP
Algebra curriculum and half of which used traditional algebra texts. Roughly 2400
students participated in the study. Classes were matched on the basis of arithmetic,
algebra, and geometry readiness.
At the end of the school year, three tests were given to each student: (1) the
American Testronics High School Subjects Test: Algebra, a 40-item multiple-choice
standardized test on which calculators were not permitted; (2 and 3) Algebra Part I and
Algebra Part II, a 70-item test constructed by the UCSMP to assess the wide range of
content in UCSMP Algebra as well as topics considered important to all algebra classes,
regardless of the curriculum being studied.
Overall, there was no significant difference between UCSMP Algebra students
and comparison students on the standardized test, even though the UCSMP students spent
less time on factoring or work with rational expressions. In general, UCSMP students
performed 10% better than comparison students on justifying properties, selecting
equations for a line given points or a graph, finding slope, and identifying expressions for
word problems. Comparison students performed 10% better than UCSMP students on
skill items, such as multiplying binomials, simplifying rational expressions with powers,
factoring trinomials, and subtracting radicals.
On the UCSMP Algebra Part I and II tests, the UCSMP Algebra students
significantly outperformed comparison students. In particular, UCSMP students
performed at least 25% better than comparison students on items such as applying a
formula, finding the area between two rectangles, calculating compound interest, and
finding the third angle in a triangle. (For more information about the First Edition study,
see the Professional Sourcebook of the Teacher's Edition of Algebra (McConnell, Brown,
Eddins, Hackworth, Sachs, Woodward, Flanders, Hirschhorn, Hynes, Polonsky, &
Usiskin 1990).)
Based on the results of the national study, minor revisions were made for the first
commercially available edition of Algebra (McConnell, Brown, Eddins, Hackworth,
Sachs, Woodward, Flanders, Hirschhorn, Hynes, Polonsky, & Usiskin 1990), published
by ScottForesman.
Prior to the preparation of the second edition, the publisher surveyed users from
all regions of the country and engaged other users in focus group discussions. In addition,
UCSMP benefited from user reports completed by many of those who had used the first
commercial version of Algebra. So, in developing the Second Edition, the authors
benefited from the earlier field studies as well as from the comments from a large number
of users.
Some of the changes made for the Second Edition include a reorganization of the
first seven chapters to incorporate equation-solving much earlier in the course.
12
Spreadsheets and automatic graphers (ie., graphing calculators) are incorporated
throughout the course. More emphasis is placed on pattern generalizing and properties
with variables; an entire chapter on factoring is included.
Other discussions in the 1990s about instructional strategies also influenced the
authors' thinking as they prepared the Second Edition, specifically recommendations
about more active learning by students such as explorations and group activities, the use
of alternative assessment options to include the use of open-ended questions and projects,
and the importance of writing about mathematics to aid learning (Countryman 1992;
Stenmark 1991). As a result, the Second Edition includes a number of new features. To
broaden teacher assessment strategies, each chapter includes a set of projects to enable
students to explore concepts in more depth and over a longer period of time. To
encourage writing, more questions ask students to write about mathematics, to explain or
justify their reasoning, and to describe representations and procedures. This focus on
writing addresses an eighth problem that authors of UCSMP Algebra identified prior to
the Second Edition, Students are not very good at communicating mathematics in writing.
Solutions to examples are printed in a special font to help model what students should
write when they do mathematics.
The remainder of this report describes the study of the Field-Trial Version of the
Second Edition of Algebra and its effect on students' achievement and attitudes.
13
14
CHAPTER 2
DESIGN OF THE STUDY
With input from the co-directors of the Secondary Component, Zalman Usiskin
and Sharon Senk, an outside evaluator, Gurcharn Kaeley, designed, monitored, and
oversaw the Second Edition Evaluation Study, which combined aspects of both a
formative evaluation and a summative evaluation. 1 The aim of the formative evaluation
was to obtain feedback on the Second Edition materials from both the students and
teachers as soon as possible in order to guide further revisions being made during that
school year. The aim of the summative evaluation was to compare the effectiveness of
Algebra (Second Edition, Field Trial Version) with Algebra (First Edition) or with the
current curriculum materials being used in comparison classes in the study schools. Thus,
this part of the study focused on achievement of students, attitudes and opinions of both
students and teachers, and issues of instructional practice that would provide evidence of
the effectiveness of the changes made to the Second Edition of Algebra. The results of
both the formative and summative evaluations influenced the authors and editors as they
made changes in the Field Trial Version in preparation for commercial publication.
This chapter describes the overall design of the study in five main sections. The
first identifies the research questions that guided the study. The second discusses the
procedures used in the study, including the selection of schools, the structure of the
matched-pair design, and the types of data collected during the school year. The third
describes the instructional materials used by the schools participating in the study: the
First Edition of UCSMP Algebra; the Second Edition, Field Trial Version, of UCSMP
Algebra; and the non-UCSMP, or so-called traditional materials currently in use in the
schools. The fourth describes the various instruments used to collect data to answer the
research questions. (The actual instruments are included in the Appendices.) The fifth and
final section describes the demographic information about the samples, including student
performance on pretests used to measure the comparability of the classes in the study.
Research Questions
The evaluation covered instructional practice, student achievement, and attitudes
and opinions about the materials and the course.
Research on Instructional Practice
In order to understand the nature of achievement with UCSMP Algebra (Second
Edition) or with the comparison materials, it is essential to understand the extent to which
the curriculum was implemented, including the extent to which technology or other
instructional practices recommended by the Curriculum and Evaluation Standards and
embedded in the materials were incorporated into the course. Hence, one of the central
1
The selection of the schools to participate in the study occurred in the Spring and Summer prior to the
school year in which the outside evaluator came on board.
15
questions was How do teachers' instructional practices when using UCSMP Algebra
(Second Edition, Field Trial Version) compare to teachers' instructional practices
when using UCSMP Algebra (First Edition) or the non-UCSMP materials currently
being used in the schools? In particular, several sub-questions were asked to obtain
more detailed information about this area:
•
To what extent do students in the three groups have an equal opportunity to
learn various mathematical concepts and skills?
•
What types of technology access and use are available to students in the three
groups?
•
How do teachers implement the curriculum defined by their textbooks?
Research on Student Achievement
A second major issue dealt with the extent to which students achieve with the
curriculum materials. Hence, another central question addressed by the study was How
does the achievement of students in classes using UCSMP Algebra (Second Edition,
Field Trial Version) compare to that of students using UCSMP Algebra (First
Edition) or to students using non-UCSMP materials? In particular, the study
examined the relation between curriculum and achievement on three measures: a
standardized multiple-choice algebra test; a UCSMP-constructed test containing items
related to content emphasized in the UCSMP curriculum as well as content that should be
important to all classes, regardless of the curriculum studied; and a UCSMP ProblemSolving and Understanding Test on which solutions to the problems require multiple
steps or constructed responses. For each type of measure, students' performance was
examined in relation to information provided by the teachers on the opportunity-to-learn
the mathematics tested.
With respect to performance on multiple-choice items, two additional questions
were investigated.
•
How proficient is each group of students in the following content areas:
translating from verbal representations to symbolic representations; linear
relationships, including equations, inequalities, and systems; quadratic
relationships; geometric relationships; statistics and probability; and
arithmetic applications?
•
How is achievement related to the four dimensions of understanding: skills,
properties, uses, and representations?
Research on Attitudes
In addition to issues of implementation and achievement, UCSMP was also
interested in the attitudes of both teachers and students to the materials. Thus, the third
central question was How do attitudes of students and teachers using UCSMP
Algebra (Second Edition, Field Trial Version) compare to those of students and
16
teachers using UCSMP Algebra (First Edition) or non-UCSMP materials? In
particular, one sub-question was investigated.
•
What are the attitudes of the three groups of students toward mathematics,
homework, their textbook, reading, explanations, and use of technology?
Procedures
The evaluation study was conducted during the 1992-93 school year. This section
outlines the procedures used in designing the study and collecting data.
Selection of Participating Schools
Random selection of schools was not possible. Rather, schools were recruited by
advertising in UCSMP and NCTM publications. Schools were not recruited looking for
any particular non-UCSMP texts in use. Instead, among those who volunteered, the
project staff attempted to find schools that might represent a broad range of educational
conditions in the United States in terms of curriculum and demographic characteristics.
To the knowledge of UCSMP staff, no studies had ever been conducted comparing a
second edition of a text to the first edition of that text. Hence, project staff decided to put
more resources into that aspect of the evaluation. Thus, more schools were chosen which
were using the First Edition of Algebra than which were using non-UCSMP comparison
materials.
To participate in the study, a school needed at least four sections of the equivalent
of an algebra class, whether at middle school or high school, and had to promise to keep
classes intact for a full year. Individuals from interested schools who answered the Call
for Study Participation completed a follow-up application (see Appendix A). From
among the forms submitted by schools interested in participating in the study, UCSMP
personnel determined whether or not schools had students and teachers in the target
groups. The study's target groups, described below, were based on the grade level of
students taking algebra.
•
Eighth graders in the 50th to 90th percentiles
•
Ninth graders in the 30th to 70th percentiles.
In terms of prerequisites, students in UCSMP Algebra should have UCSMP Transition
Mathematics or a strong prealgebra course in the preceding year, a willingness and
maturity to complete daily homework, and a plan to study geometry in the subsequent
year (McConnell et al, 1996, T31).
In each school, the district mathematics supervisor, department chair, or a teacher
provided the names of at least two teachers willing to participate in the study. Where
possible, teachers were randomly assigned to UCSMP Second Edition classes and to the
comparison classes using First Edition materials or the non-UCSMP comparison textbook
currently in place at that school. In some situations, local conditions did not permit
random assignment.
17
Initially, classes were selected in 13 sites. From 28 Second Edition classes and 26
comparison classes (both First Edition and non-UCSMP), 26 pairs were formed; two
Second Edition classes in two schools did not have a match. Nineteen of these pairs
involved students studying from the Second Edition or First Edition of Algebra. Seven
pairs involved students studying from the Second Edition of Algebra or the comparison
texts already in use in the school.
School-Year Procedures
No direct inservice was provided to the teachers using UCSMP Algebra (Second
Edition, Field Trial Version), either before or during the school year. Although teachers
had a tentative Table of Contents for the entire book when school began, they received
the actual text in three spiral-bound parts: Chapters 1-4 at the beginning of the school
year; Chapters 5-8 around November; and Chapters 9-13 in early winter. Additionally,
teachers received lesson notes and answers to questions, one chapter at a time, throughout
the school year.
To assist with the Formative Evaluation, UCSMP Second Edition teachers
completed a Chapter Evaluation form (see Appendix C) after completing each chapter.
These teachers also met in Chicago once in the fall and again in the spring to give
feedback to the developers about the materials. During these meetings, there were brief
opportunities to raise issues related to the use of technology, the use of reading and group
problem-solving in class, and to discuss other instructional concerns. Also, these
meetings provided an opportunity for the developers to learn about any unusual
circumstances in the schools that could influence the results.
At the beginning of the school year, students completed a survey and a pretest.
The survey collected demographic information and queried students about their attitudes
toward mathematics. The pretest was a standardized test (Iowa Algebra Aptitude Test)
and was used to determine whether pairs of classes were comparable in terms of
prerequisite knowledge at the beginning of the year.
During the second semester, each school in the field study was visited for one or
two days. At least one class taught by each UCSMP Second Edition teacher and at least
one class taught by each comparison teacher (either First Edition or non-UCSMP) was
observed. In addition, teachers were interviewed about the content covered and their
pedagogical practices. Classroom observation notes and interviews were transcribed. (See
Appendix E for the Classroom Observation Report Form and the Interview Protocol.)
The site visits were conducted by the Director of Evaluation and by graduate
students from the University of Chicago. None of the observers was directly involved in
the writing of the Algebra (Second Edition) text.
Shortly before the end of the school year, teachers administered several
instruments: a standardized multiple-choice posttest to assess achievement with algebraic
skills and applications; a UCSMP-constructed multiple-choice posttest to assess
achievement on content emphasized in UCSMP Algebra as well as achievement on other
content deemed important in algebra, regardless of the curriculum studied; one of two
forms of a Problem-Solving and Understanding Test to assess ability to solve problems
18
requiring multiple steps or constructed responses (students were randomly assigned to
receive one of the two forms); and a survey of students' attitudes and opinions about
mathematics, their course, and their text. Neither pretest nor posttest scores had any
influence on students' grades in the course. However, students were encouraged to do
their best.
At the end of the year, teachers also completed a short survey about their
academic preparation and their years of teaching experience. They also completed an
opportunity-to-learn form for each of the posttests.
Matched-Pair Design
Factors such as student ability, amount of time allocated to mathematics
instruction, socioeconomic status of the school population, size of the community, and
location can influence student learning and achievement. To control for these factors, the
study employed a matched-pair design in which classes were matched in the same school
on the basis of students' mathematics ability. Generally, both teachers participating in the
study agreed to teach the Second Edition or the comparison materials depending on the
outcome of the random selection; hence, the classes should not have differed on the basis
of one teacher being particularly enthusiastic when compared to the other.
With this design, each matched pair is a mini-study replicated many times. This
enables the evaluation to take local contexts into account but it still permits overall
generalizations through aggregation, particularly if the results across the mini-studies are
consistent.
As indicated previously, among the 13 school sites, 26 pairs were formed initially.
Nineteen pairs consisted of one class using UCSMP Algebra (Second Edition, Field Trial
Version) and one class using UCSMP Algebra (First Edition). Seven pairs consisted of
one class using UCSMP Algebra (Second Edition, Field Trial Version) and one class
using the non-UCSMP text currently in use for the course in the participating schools.
For each pair of classes, the differences in the pretest means were examined and
two-tailed t-tests were used for comparison of the extent to which the classes were good
matches. At the end of the school year, the matches were checked again using pretest
results only from students present for the pretest, all three posttests, and both
administrations of the student survey. Thus, a pair was discarded, either at the beginning
or the end of the school year, if any of the following criteria were satisfied:
1. On the pretest, the difference in the means is significant (p ≤ 0.025).
2. On the pretest, the variance is significantly different (p ≤ 0.05).
3. On the pretest, the difference in the means is significant (p ≤ 0.025) when only
students who took the pretest, all posttests, and both student surveys are
considered.
4. On the pretest, the variance is significantly different (p ≤ 0.05) when
considering only those students who took the pretest, all posttests, and both
student surveys are considered.
19
5. A class in either pair dropped out of the study.
6. One class in the pair had more than twice the number of students in the other
class in the pair when only students who took the pretest, all posttests, and
both student surveys are considered.
7. Additional information suggests that the students in the classes were mostly
different.
Although the comparison of pretest means for only those students taking the
pretest, all posttests, and both student surveys (that is, students in the final sample)
weighed most heavily in making decisions on the viability of the matches, information
suggesting that the classes were indeed different in some way could override any other
criteria. In cases where two Second Edition classes were initially matched with the same
comparison class, the single best-matched pair was selected for inclusion in the final
analysis, based on examining means, standard deviations, range, and shape of the pretest
distributions.
One site was dropped from the study during the year as the First Edition teacher
failed to return the pretest or any of the posttests. At a second site, neither pair matched at
the end of the school year, because only 3 students in one First Edition class and 4
students in a second First Edition class completed all instruments. Four other pairs at
various sites failed to match at the end of the school year when pretest results were
checked again using results from only those students who completed all instruments.
For the purposes of the remaining analyses, there are nineteen well-matched pairs.
Of these, thirteen are pairs in which one class of students used UCSMP Algebra (Second
Edition, Field Trial Version) and the other class used UCSMP Algebra (First Edition);
these pairs are in eight schools in seven states. Six are pairs in which one class used
UCSMP Algebra (Second Edition, Field Trial Version) and the other class used a nonUCSMP text currently in use for the course; these pairs are in three schools in three
states. 2
Instructional Materials
This section describes the three types of instructional materials used in the study
schools: UCSMP Algebra (First Edition); UCSMP Algebra (Second Edition); and the
non-UCSMP algebra materials in use in the participating schools at the time of the study.
Complete Tables of Contents for the UCSMP texts are found in Appendix B. For
discussion purposes within this section, Table 1 contains chapter titles for each of the
texts used in the study.
2
Throughout the remainder of this report, the use of Algebra (Second Edition) is understood to mean
Algebra (Second Edition, Field Trial Version). Based on the formative and summative aspects of the study,
the Field Trial Version was modified slightly prior to commercial publication. Some of the changes made
between the two versions are discussed in Chapter 6 of this report.
20
Table 1. Chapter Titles for Each of the Textbooks Used in the Study
Chapter
UCSMP Algebra
UCSMP Algebra
(Second Edition, Field Trial
(First Edition)
Version)
1
Uses of Variables
Basic Concepts
Algebra I
(Fair & Bragg 1990)
Real Numbers
Algebra: Structure and
Method (Dolciani, Brown,
Ebos, & Cole 1984)
Introduction to Algebra
2
Multiplication in Algebra
Addition in Algebra
Algebraic Expressions
Working with Real
Numbers
3
Linear Expressions Involving
Addition
Subtraction in Algebra
Equations in One Variable
Solving Equations and
Problems
4
Linear Expressions Involving
Subtraction
Multiplication in Algebra
More Equations in One
Variable
Polynomials
5
Linear Sentences
Division in Algebra
Inequalities in One
Variable
Factoring Polynomials
6
Division in Algebra
Linear Sentences
Polynomials
Fractions
7
Slopes and Lines
Lines and Distance
Factoring Polynomials
Applying Fractions
8
Exponents and Powers
Slopes and Lines
Rational Expressions
Linear Equations and
Systems
9
Quadratic Equations and Square
Roots
Exponents and Powers
Linear Equations
Introduction to Functions
10
Products, Factors, and Quadratics
Polynomials
Relations, Functions, and
Variation
Inequalities
11
Systems
Systems
Systems of Linear
Equations
Rational and Irrational
Numbers
12
Polynomials and Sets
Parabolas and Quadratic
Equations
Radicals
Quadratic Functions
13
Functions
Functions
Quadratic Equations and
Functions
14
Statistics and Probability
15
Right Triangle
Relationships
21
Algebra 1: An
Incremental Development
(Saxon)
The Saxon textbook is not
divided into chapters.
Rather it simply has daily
lessons.
UCSMP Algebra: Second Edition and First Edition
As indicated by the chapter titles in Table 1, the First and Second Editions of
UCSMP Algebra are quite similar. In general, most of the changes between the two
editions were focused on fine-tuning of lessons and organization based on information
from users of the First Edition. For instance, the content of the first seven chapters was
re-ordered from the First Edition to the Second Edition, resulting in a change in the order
in which different types of linear equations are introduced and solved. In the Second
Edition, the first equation to be solved is of the form ax = b in Chapter 2. This permits
multi-step linear equations of the form ax + b = c to be solved in Chapter 3 and the
general linear equation, ax + b = cx + d, to be solved in Chapter 5. In the First Edition,
equation solving began with equations of the form x + a = b. To help facilitate this
reordering of content, refresher activities precede each of the four chapters dealing with
the operations of addition, subtraction, multiplication, and division in algebra.
In addition, quadratic equations are introduced earlier in the Second Edition, in
Chapter 9 rather than in Chapter 12 as in the First Edition. Also, Chapter 10 now focuses
on polynomial products and factoring, thereby placing a greater emphasis in the Second
Edition on factoring of various types of polynomials. More emphasis is also placed in the
Second Edition on using variables to generalize from patterns and on properties of
variables.
Certain technology is more prevalent in the Second Edition. Although both
editions assume the use of scientific calculators, the Second Edition incorporates the use
of spreadsheets and automatic graphers, with spreadsheets introduced in Chapter 1 and
automatic graphers (either graphing calculators or computer graphing software) in
Chapter 4.
Like all UCSMP textbooks, both editions of UCSMP Algebra emphasize four
dimensions of understanding: skills, properties, uses, and representations. Skills deal with
procedures to get answers and include both rote memorization of basic facts and the
development of new algorithms for solving problems. Properties relate to the principles
behind the mathematics and include identification of properties as well as the
development of new proofs. Uses deal with applications of mathematics in real situations.
Representations deal with pictures, graphs, or objects used to represent concepts
(McConnell et al., 1996, T52). The majority of lessons begin with either a realistic
context or some graphical representation, and most lessons and chapters include work
with each of these four dimensions, with roughly equal emphasis to each of the four
dimensions across the text. Both editions encourage students to approach problems from
multiple perspectives, often showing multiple solutions in worked examples.
However, the Second Edition encourages more writing in mathematics. Teachers
using the Second Edition are encouraged to use a variety of assessment practices through
the inclusion of extended projects at the end of each chapter and through more openended questions in which students explain their thinking or justify their reasoning to
describe representations and procedures. A special font is used in the Second Edition to
help model what students should write when they do mathematics.
22
UCSMP Algebra (Second Edition) and non-UCSMP Texts
The texts used by the non-UCSMP classes were the algebra texts in use in those
three schools at the time of the study. They were Algebra I: An Incremental Development
(Saxon) used in School X, Houghton Mifflin Algebra: Structure and Method Book I
(Dolciani, Brown, Ebos, & Cole 1984) used in School Y, and Prentice Hall Algebra 1
(Fair & Bragg 1990) used in School Z. According to Weiss, Matti, and Smith (1994), the
Houghton Mifflin Algebra 1 text, as well as the UCSMP Algebra (First Edition) text,
were among the most widely used algebra texts at the time the study was conducted.
The mathematical topics in these three texts and the UCSMP Algebra (Second
Edition) overlap considerably. Each includes work with variables, solving linear
equations and inequalities, solving systems of equations and inequalities, introduction to
polynomials and factoring, and quadratic functions. However, the order in which the
topics are presented and the relative emphasis given to each topic vary across texts, as
does the pedagogical approach taken to that content, and the kinds of exercises and
problems students are expected to solve. For example, although quadratic equations are
studied in Chapter 9 (of 13 chapters) in the UCSMP text, they are studied in Chapter 13
(of 15) in the Fair and Bragg text, Chapter 12 (of 12) in the Dolciani et al. text, and in the
final third of the Saxon text.
The non-UCSMP texts contain some topics not covered in the UCSMP Algebra.
For example, each of the non-UCSMP texts includes work with simplifying and
operating with rational expressions, solving rational equations, adding and subtracting
radicals, solving radical equations, completing the square, dividing polynomials, and
direct and inverse variation. The Fair and Bragg text includes a chapter on statistics and
probability.
Although the texts used in the comparison classes each treat all the dimensions of
understanding to some extent, they emphasize skills far more than the other three
dimensions. There are fewer applications or problems set in real contexts than in the
UCSMP text, even though each problem set in the Fair and Bragg text contains a few
applications at the end of the set of exercises. In the non-UCSMP texts, there are often
separate lessons that focus on problem solving of particular types of problems.
Every exercise section of UCSMP Algebra contains review questions covering
topics earlier in the specific chapter as well as topics from earlier chapters. Likewise, the
Saxon text is designed with continual review; in fact, each exercise set contains just a few
problems from the current lesson and many more problems from previous lessons. The
Fair and Bragg text includes separate pages at the end of each chapter on preparation for
standardized tests and maintaining skills. The Dolciani et al. text contains a cumulative
review at the end of each chapter, beginning with Chapter 3. Only the UCSMP Algebra
text includes extended projects at the end of each chapter.
There are some similarities and differences in the ways that the UCSMP Algebra
text and the non-UCSMP texts treat technology. As previously mentioned, the UCSMP
Algebra text assumes that students have continual access to a scientific calculator and
occasional access to spreadsheets and automatic graphers. The Fair and Bragg text has
some calculator problems marked throughout the text and includes some BASIC
computer programming applications. The Dolciani et al. text contains separate sections
23
entitled Calculator Key-in and Computer Key-in, in which students use calculators to
complete problems and focus on BASIC programs, respectively. It is not clear what, if
any, calculator assumptions are made in the Saxon text.
The Fair and Bragg and Dolciani et al. texts contain numerous supplementary
pages or sections focusing on special topics that do not appear to be integral to the
chapters, such as sections describing careers, historical notes containing biographies or
information about mathematical symbolism, or the use of algebra in the real world.
Instruments
This section describes the instruments used in the study, most of which are
included in Appendix C.
Iowa Algebra Aptitude Test
This was an 80-item test used as a pretest for the purposes of matching pairs in
participating schools. Among the items were 34 items dealing with sequences, 20 with
open phrases, and 10 with dependence and variation. The Kuder-Richardson KR20
ranged between 0.85 and 0.90.
Fall Student Opinion Survey
This 15-item survey, developed at the UCSMP, asks students for demographic
information as well as their opinion about mathematics and their confidence in doing
mathematics.
High –School Subject Tests: Algebra
The High School Subject Tests: Algebra (American Testronics 1988) is a 40-item
standardized multiple-choice test focusing on algebraic concepts. Eight of the items deal
with operations with polynomials, six with linear equations/inequalities in one variable,
four with evaluating expressions for given values of a variable, four with linear
relationships in two variables, three with quadratics, three with operations involving
radicals, three with properties of numbers, three with solving linear systems, three with
operations involving rational expressions, one with factoring, one with solving literal
equations, and one with proportions. Of the 40 items, 29 (72.5%) deal with skills, 5
(12.5%) with properties, 4 (10%) with uses, and 2 (5%) with representations.
Calculators were not permitted on the test. The test manual indicates the KuderRichardson KR20 = 0.86; for the samples in this study, the KR20 was roughly 0.80.
24
Algebra Test
This UCSMP constructed test consists of 40 multiple-choice items covering the
following content areas: ten requiring translation from verbal to symbolic form; six
involving linear relationships with two variables; five on quadratic equations and
relationships; five dealing with geometric relationships; four dealing with statistics or
probability; three involving percent applications; two focusing on graph interpretation;
two involving exponential relationships; and three miscellaneous items (pattern
identification, factorial simplification, application with multiplication counting principle).
Among the 40 items, 4 (10%) focus on skills, 1 (2.5%) on properties, 22 (55%) on uses,
and 13 (32.5%) on representations.
Calculators were permitted on this test. The test has a Kuder-Richardson KR20
between 0.81 and 0.83.
Algebra Problem-Solving and Understanding Test
The third posttest is an open-ended problem-solving test developed at the
UCSMP. This test was administered in two different forms. Half of the students in each
class were randomly assigned the even form of the test and the other half was assigned
the odd form.
The even form consists of four items: one dealing with creating and solving a
word problem for a linear equation in one variable; one with the distributive property;
one with a general rule for finding the cost of an item after a discount; and one with
graphing a quadratic equation. The odd form also consists of four items: one dealing with
creating and solving a word problem for a linear system; one with using data to make an
estimate; one with rules for expanding the square of a binomial; and one with sketching
the graph of a linear inequality in two variables.
Rubrics were developed for scoring the problem-solving items (see Appendix D),
using procedures applied in studies by Malone, Douglas, Kissane, and Mortlock (1980),
Senk (1989), and Thompson and Senk (1993). Five of the eight items were scored using a
0 to 4 rubric, with each score level having the broad meaning outlined in Table 2. The
maximum score on the even form was 16; the maximum for the odd form was 14.
Three of the items were scored using modified versions of the rubric in Table 2
because the nature of the items suggested it would be difficult to distinguish among five
levels of responses or there were multiple parts to the item for which separate scores were
desired. The item on the odd form dealing with making an estimate from a set of data was
scored using a 0 to 2 rubric, with 2 considered successful and 1 considered partially
successful. Each of the two items requiring students to graph consisted of two parts, with
students sketching a graph and then using the graph to answer another question. Each part
was scored on a 0 to 2 rubric.
25
Table 2. General Scoring Rubric: Problem-Solving and Understanding Test
Score
Description of Performance
Successful Responses
4
Solution is complete and response is fine.
3
Student works out a reasonable solution, but minor errors occur in notation
or form; the errors are not conceptual in nature.
Unsuccessful Responses
2
Response is in the proper direction, but student makes major conceptual
errors; however, the response displays some substance that indicates a
chain of reasoning.
1
Student makes some initial progress but reaches an early impasse.
0
Work is meaningless; the student makes no progress.
Item-specific rubrics were developed for each item to indicate the mathematical
content and knowledge to be demonstrated on each task (see Appendix D). Anchor
papers were used to train raters in using the rubrics. Each student response was then
scored independently and blindly by two raters. Raters had no way of knowing which
responses were from UCSMP Second Edition students, from UCSMP First Edition
students, or from non-UCSMP students, or what score was received on a previous item. If
two raters disagreed, a third rater scored the response and the median score was recorded
as the final score. The inter-rater reliabilities for each item on the even form were as
follows: 78.8% (item 1); 89.7% (item 2); 74.2% (item 3); 93.1% (item 4a); and 89.4%
(item 4b). For the odd form, the inter-rater reliabilities were as follows: 79.3% (item 1);
82.9% (item 2); 88.1% (item 3); 92.8% (item 4a); and 94.8% (item 4b).
Spring Student Survey of Opinions About Mathematics
This opinion survey consists of 26 items, of which six are similar to those
administered in the Fall survey. Items deal with opinions about mathematics, about the
textbook and instructional strategies used in the current school year, and about students’
study habits in the course. The items were developed internally at the UCSMP and were
similar to those used in earlier project studies.
Algebra Mathematics Teacher Questionnaire
Teachers were asked for demographic information, including degrees,
certification areas, and number of years teaching.
26
Opportunity-to-Learn Form
The Opportunity-to-Learn form is designed to provide information about the
extent to which the curriculum represented on the posttest measures was taught. It
provides information, from the perspective of the participating teachers, on the extent to
which the posttest measures are fair, regardless of the curriculum materials used.
For each of the 40 items on the High School Subject Tests: Algebra, the 40 items
on the Algebra Test, and the four items on each form of the Problem-Solving and
Understanding Test, the teacher was asked, “During this school year, did you teach or
review the mathematics needed for your average students to answer this item correctly?”
Teachers were given three choices:
•
Yes, it is part of the text I used.
•
Yes, although it is not part of the text I used.
•
No.
As the results in Chapters 3 and 4 demonstrate, the three groups of students had
varying opportunities to learn the content needed to answer items on the posttests. Hence,
the achievement results will be presented with the opportunity-to-learn taken into
consideration. That is, achievement results are presented by matched pairs in three ways
as appropriate:
•
overall achievement on each posttest, with indicators to identify the
percentage of items for which teachers responded that students had an
opportunity to learn the content;
•
achievement by school on a “fair” subtest consisting of only those items for
which both teachers in the pair indicated their students had an opportunity to
learn the necessary content:
•
and achievement on a “conservative” test consisting of only those items for
which all teachers in the groups being compared indicated their students had
an opportunity to learn the needed content.
More information about these “fair” and “conservative” tests is given with the
appropriate achievement results in Chapter 4.
Chapter Evaluation Forms
All teachers using the Second Edition of UCSMP Algebra were asked to complete
a Chapter Evaluation form on each chapter of the textbook they taught, rating the text and
problems for each lesson as well as the overall chapter. The chapter ratings provided
information to the curriculum developers as part of the Formative Evaluation and also
provided valuable insights in preparation for commercial publication.
27
Description of the Samples
This section describes the two samples (the Second Edition and First Edition
sample and the Second Edition and non-UCSMP sample), including information about
the schools, results on the pretest, and demographic information about the students and
teachers.
Matched-Pair Sample: UCSMP Second Edition and UCSMP First Edition
Students in eight schools (B, C, D, E, G, H, I, and J) in seven states comprise the
Second Edition and First Edition sample.
School B. This middle school of about 980 students in grades six through eight is
located in a suburban region in the West serving a middle- to upper-middle
socioeconomic class with few minorities. Expectations for success are high and most
students are expected to attend college after high school. Parental participation is also
high. Students were in the upper academic track, taking algebra in the eighth grade; the
mean national percentile for students in the algebra class was 62%. The school used
UCSMP Transition Mathematics, UCSMP Algebra, and UCSMP Geometry, so most
students had completed the previous UCSMP course.
School C. This middle school, comprised of about 1500 students in grades six
through eight, is in a high-income suburban area in the South. The school used both
UCSMP Transition Mathematics and UCSMP Algebra the previous school year, so most
students had studied from the previous UCSMP text. The mean national percentile
ranking for students in the sample was 60%, with a range of from 50-90%. The school
contained a number of portable classrooms because of overcrowding and school ran from
9:30 a.m. until after 4:00 p.m.
School D. This small high school of roughly 630 students in grades 9-12 is
located in a rural area in the Midwest in which the surrounding community consists
primarily of White residents. The school used four of the UCSMP texts in the previous
year: Transition Mathematics, Algebra, Geometry, and Advanced Algebra. Students in
the study had a mean national percentile ranking of roughly 50%.
School E. This middle school of about 750 students in grades 7 and 8 serves a
lower-income socioeconomic population in an urban area in the Midwest. During the
previous year, the school had used both UCSMP Transition Mathematics and UCSMP
Algebra, with most students having studied from the previous UCSMP text. The faculty
and student body were racially mixed. The school building itself was rather old.
School G. This high school, serving students in grades 9-12, is located in a small
town in the Northeast with a middle- to lower-middle socioeconomic class of primarily
White residents. Students in the sample were of average ability as honors students took
algebra in the eighth grade. Some students had previously studied from Transition
Mathematics.
School H. This high school of approximately 900 students in grades 9-12 is
located in a suburban community in the Midwest with only a few minority families.
During the previous year, the school had used UCSMP Transition Mathematics, UCSMP
28
Algebra, and UCSMP Geometry, so some students had studied from the previous
UCSMP text.
School I. This suburban high school of roughly 800 students in grades 9-12 is
located in a blue-collar community in the Midwest consisting of a predominantly White
population. Approximately half of the graduating class matriculated to a four-year
college; the remainder typically went to a trade school, into the armed forces, or into the
work force. During the previous year, the school had used UCSMP Algebra and UCSMP
Geometry.
School J. This high school of roughly 1050 students in grades 9-12 is located in
an upper-middle class suburb in the West consisting of primarily a White population.
Students in the study classes were of average ability. During the previous year, all six of
the UCSMP texts were used at the school.
Table 3 contains the demographics for the students in the Second Edition and
First Edition sample. As seen in the table, most of the students in the study were in eighth
or ninth grade, with slightly more than 40% at each of those grades.
Table 3. Number (Percent) of Students in the Second Edition and First Edition Sample
Group
Grade 8 Grade 9 Grade 10 Grade 11 Unknown
UCSMP Second Edition
71
68
23
1
1
(43.3%) (41.5%) (14.0%) (0.6%)
(0.6%)
Total
164
UCSMP First Edition
73
(42.9%)
77
(45.3%)
14
(8.2%)
1
(0.6%)
5
(2.9%)
170
Overall
144
(43.1%)
145
(43.4%)
37
(11.1%)
2
(0.6%)
6
(1.8%)
334
Of the 164 students using the Second Edition of Algebra, 51.2% were female;
47.6% of the 170 students using the First Edition were female. No information was
collected about the ethnic background of the students.
Table 4 contains the pretest results, by matched pair, for this sample on the Iowa
Algebra Aptitude Test. Although there are no significant differences in prerequisite
knowledge among the classes in each pair at the 0.05 level, there are some differences
among the pairs from school to school. For the Second Edition classes, the mean score
ranges from 42.00 to 54.81 out of 80; for the First Edition classes, the mean score ranges
from 37.50 to 58.09 out of 80.
29
Table 4. Pretest Means, by Matched Pair: Second Edition and First Edition
School Pair
Code
ID
B
2
C
4
C
5
D
6
D
7
E
8
E
9
G
12
H
14
H
15
I
17
J
18
J
19
n
Mean
SD
18
46.94
9.51
12
42.00
11.43
12
45.58
8.08
6
47.17
9.95
11
47.45
5.96
19
52.26
7.33
10
54.20
10.41
10
50.20
8.22
16
43.56
9.18
11
48.36
8.85
16
50.19
11.53
16
54.81
7.30
7
45.71
9.72
UCSMP First Edition
n
Mean
SD
19
45.32
9.05
20
43.70
7.12
8
37.50
7.48
10
49.60
9.70
12
50.25
7.90
11
58.09
8.51
15
50.73
13.15
11
43.82
6.90
12
47.08
9.45
11
44.82
5.62
12
48.50
7.88
18
54.61
9.17
11
52.91
12.36
SE
3.05
3.27
3.59
5.05
2.94
2.95
4.96
3.30
3.55
3.16
3.87
2.87
5.53
t
0.53
-0.52
2.25
-0.48
-0.95
-1.98
0.70
1.93
-0.99
1.12
0.44
0.07
-1.30
df
35
30
18
14
21
28
23
19
26
20
26
32
16
Note: The maximum score on the pretest is 80.
The teachers of the Second Edition and First Edition students had roughly
comparable backgrounds. Among the Second Edition teachers, the number of years
teaching mathematics ranged from 3 to 35 with a mean of 21 years (s.d. = 11.5 years) and
a median of 25.5 years. Among First Edition teachers, the number of years teaching
mathematics ranged from 3 to 28 years with a mean of 16.6 years (s.d. = 9.5 years) and a
median of 20 years. Seven of the eight Second Edition teachers and six of the eight First
Edition teachers had previously taught from UCSMP Algebra. Five of the Second Edition
and four of the First Edition teachers were female.
Five of the Second Edition teachers had Master’s degrees. Five had undergraduate
degrees in mathematics, one had an undergraduate degree in education but a Masters in
mathematics, one had an elementary education background with a mathematics minor,
and one had a history degree with a mathematics minor. Two of the First Edition teachers
had Master’s degrees and two others had graduate hours. Six had undergraduate degrees
in mathematics, one had an undergraduate degree in geography but was pursuing a
Master’s degree in mathematics education, and one had an undergraduate degree in
chemistry with a mathematics minor.
Three of the Second Edition teachers were certified in mathematics 7-12, one in
mathematics 5-8, one in mathematics secondary, one in mathematics 9-12, one in
mathematics K-12, and one in general secondary 6-12. Among the First Edition teachers,
six were certified in mathematics 7-12, one in mathematics 6-12, and one as a
mathematics supervisor K-12.
The demographic information suggests that these teachers were somewhat better
prepared mathematically than is typical of many mathematics teachers, particularly those
in the middle grades (Weiss, Matti, & Smith 1994).
30
p
0.597
0.607
0.037
0.638
0.352
0.058
0.492
0.068
0.330
0.275
0.667
0.945
0.212
Matched-Pair Sample: UCSMP Second Edition and non-UCSMP
Students in three schools (X, Y, and Z) participated in the Second Edition and
non-UCSMP study. Following are brief descriptions of the schools.
School X. This high school is a large, ethnically diverse school on the West Coast
serving roughly 2800 students in grades 9-12 from inner-city and suburban environments.
The only UCSMP text previously used at the school was Geometry.
School Y. This suburban high school in the Northeast serves about 950 students in
grades 9-12 from a middle- to upper-middle class socioeconomic population. The mean
national percentile for students in the study was about 79%, with a range of 38-97%. No
previous UCSMP texts had been used at the school.
School Z. This suburban high school of about 2800 students in grades 9-12 is
located in a middle- to upper-middle class neighborhood in the South serving a large
Hispanic community. The school regularly won statewide honors in athletics and
academics and most students were expected to matriculate to a four-year college. No
previous UCSMP texts had been used at the school. Students in the study were generally
at grade level or slightly below.
Table 5 provides the overall demographics for the students in the Second Edition
and non-UCSMP sample.
Table 5. Number (Percent) of Students in the Second Edition and non-UCSMP Sample
Group
Grade 9 Grade 10 Grade 11 Grade 12 Unknown
83
14
1
0
0
(84.7%) (14.3%) (1.0%)
Total
98
non-UCSMP
78
(85.7%)
10
(11.0%)
1
(1.1%)
1
(1.1%)
1
(1.1%)
91
Overall
161
(85.2%)
24
(12.7%)
2
(1.1%)
1
(0.5%)
1
(0.5%)
189
Of the 98 students in the Second Edition sample, 53.1% were female; 50.5% of
the non-UCSMP sample were female. No information about ethnicity was solicited from
students. The students in the Second Edition and non-UCSMP sample were generally
somewhat older than the students in the Second Edition and First Edition sample.
Table 6 contains the pretest results, by matched pair, on the Iowa Algebra
Aptitude Test. Although there are no significant differences in pretest results between
Second Edition and non-UCSMP students by pair, there is variability in the results across
schools. For Second Edition students, the class means range from 40.86 to 51.74 out of
80; for non-UCSMP students the class means range from 40.55 to 55.63 out of 80.
31
Table 6. Pretest Means, by Matched Pair: Second Edition and non-UCSMP
School Pair
Code ID
X
21
Y
22
Y
23
Y
24
Z
25
Z
26
n
Mean
SD
14
40.86
8.72
21
48.90
11.80
19
51.74
13.95
25
47.68
8.33
11
47.18
13.80
8
41.00
12.28
non-UCSMP
Mean
SD
49.64
11.06
55.63
7.51
48.82
11.5
43.19
9.87
45.29
12.98
40.55
7.71
n
14
16
17
16
17
11
SE
3.76
3.38
4.29
2.87
5.15
4.58
t
-2.33
-1.99
0.68
1.57
0.37
0.10
df
26
35
34
39
26
17
Note: Maximum score on the pretest is 80.
Two of the Second Edition and two of the non-UCSMP teachers were female. The
number of years teaching mathematics ranged from 8 to 21 years for the Second Edition
teachers (mean = 14.7 years, s.d. = 6.5 years, median = 15 years); for the non-UCSMP
teachers, the number of years teaching mathematics ranged from 2 to 18 years (mean =
8.3 years, s.d. = 8.5 years, median = 5 years). Overall, the non-UCSMP teachers had
somewhat less teaching experience than the Second Edition teachers. None of the
teachers had previously taught from a UCSMP text.
Two of the Second Edition teachers had undergraduate degrees in mathematics,
one of whom also had a Master’s in mathematics education; the third had an
undergraduate degree in secondary education, a Master’s in Curriculum and Instruction
and another Master’s in mathematics. Among the non-UCSMP teachers, two had
undergraduate degrees in mathematics and one had an undergraduate degree in biology
with a mathematics minor; none of the teachers in this group had a graduate degree.
All teachers in both groups were certified in mathematics for grades 7-12 or
grades 6-12.
32
p
0.028
0.055
0.502
0.125
0.717
0.922
CHAPTER 3
THE IMPLEMENTED CURRICULUM AND INSTRUCTION
This chapter describes the content covered and teachers’ use of specific
pedagogical strategies in the UCSMP Algebra classes, both first and second edition, and
the non-UCSMP classes. Knowledge of the implemented curriculum helps the reader
understand any achievement differences that may exist between students studying from
the two different curricula in the two samples.
The first section of the chapter deals with content coverage in the various classes
comprising the two samples. The second section deals with instructional practices from
both the student and teacher perspectives.
This chapter is based on data from the classroom observations, teacher interviews
(see Appendix E), and from the student survey completed at the end of the school year.
Additional data are from the Chapter Evaluation Forms completed by teachers using the
Second Edition (see Appendix C) and from the Opportunity-to-Learn forms completed by
all teachers for each of the items on the posttests.
Content Coverage
Second Edition and First Edition Sample
Second Edition Classes. During the interviews conducted in the spring, most of
the teachers indicated the number of chapters they expected to complete by the end of the
year. The amount of time remaining in the school year varied from site to site depending
on the time of the visit. Nevertheless, seven of the eight Second Edition teachers
generally expected to complete from 10 to 12 chapters, with a median of about 11.5
chapters. One teacher did not indicate the number of chapters expected to be completed
but was in the middle of Chapter 7 in late March.
More information about the actual content covered in Second Edition classes can
be inferred from the chapter evaluation forms completed by most of the teachers and
from their responses to the opportunity-to-learn forms. The chapter evaluation forms
were designed to provide the curriculum developers with specific information about each
lesson, such as the difficulty levels of the text and problems, the length of time needed to
complete a lesson, and the overall difficulty of a chapter. Hence, completion of these
forms not only provided information about the actual chapters covered by Second Edition
teachers but also provided information about the extent to which the pace of a lesson-aday was feasible.
Table 7 contains the number of days spent on each chapter by the Second Edition
teachers, including time spent on projects, Chapter Review, Self-Test, and a final chapter
test for assessment purposes. Although forms for some chapters were not returned, the
table, together with the interview comments, suggests that most teachers completed at
least the first 10 chapters.
33
Table 7. Days Spent on Each Chapter of the Second Edition, Including Testing, by
Teachers in the Second Edition and First Edition Sample
Chapter
(Number of Lessons)
School
B
C
D
E
Ave.
G
H
I
a
J
Days
16.5
13.8
1.
Uses of Variables (9)
13
13
12
16
15
13
12
2.
Multiplication in Algebra (10)
15
17
14
15
16
15
13
18
15.4
3.
Linear Expressions Involving Addition (10)
11
15.5
13
17
16
15
16
20
15.4
b
16
14.3
4.
Linear Expressions Involving Subtraction
(9)
14
11
13
14
17
15
14
5.
Linear Sentences (8)
13
15
13
17
15
13
14
13
14.1
6.
Division in Algebra (10)
15
16
15
15
16
16+
17
14
15.5
7.
Slopes and Lines (9)
12
14
12
17
15
13
16
15
14.3
8.
Exponents and Powers (9)
12
17
12
14
14
16+
12
12
13.6
9.
Quadratic Equations and Square Roots (9)
11
NR
13
NR
16
17
17
12
14.3
NR
c
NR
16
c
c
c
5
c
13
NR
d
5
e
9
h
na
10. Products, Factors, and Quadratics (8)
11. Systems (8)
10
c
14
12
f
12
g
NR
NR
NR
NR
3
NR
NR
NR
NR
NC
11
NR
11
13. Functions (6)
NC
NR
NC
C
15
NR
12. Polynomials and Sets (7)
5
20
NR
na
Note: NR indicates that the form was not returned; NC indicates that the form was
returned and the chapter was not covered.
a
The teacher did not indicate the total number of days; these were determined based on 1
day for each of sections 1-5, and the fact that lessons 6-8 were rated, indicating they were
taught. Three days were spent on the review.
b
The teacher did not complete Lesson 4-9 with the class.
c
The teacher did not complete Lessons 10-9, 10-10, or 10-11 with the class.
d
The teacher did not complete Lessons 11-5, 11-6, 11-7, or 11-8 with the class.
e
f
The teacher indicated the chapter was covered but did not test; no indication was given
for the number of days spent per lesson.
g
The teacher did not cover Lesson 12-1.
h
The teacher covered only the first three lessons of the chapter.
Information in Table 7, together with teachers’ responses to the opportunity-tolearn forms for each posttest, provides a picture of content taught by Second Edition
teachers. The teacher at School I covered the least amount of content, at least as
measured by the OTL forms; this teacher did not cover statistics or probability,
quadratics, polynomial operations, systems, or basics of radicals. In general, however,
Second Edition teachers covered solving equations and inequalities, translating verbal
34
forms of a problem into symbolic form, slopes and graphs of lines, solving systems
(except at School G), basics of quadratics (i.e., graphs and the quadratic formula), and
linear equations in two variables. They did not cover simplifying radical expressions or
division with polynomials.
First Edition Classes. A picture of the content coverage for First Edition classes
can also be constructed from the interviews and the opportunity-to-learn responses.
During the interviews, First Edition teachers indicated that they expected to cover from
11 to 13 chapters, with a median of 12 chapters; at School B the teacher did not provide
any indication of the chapters to be covered and the teacher at School I was at the end of
Chapter 10 during the observation in late April.
The OTL responses, together with the expected chapter coverage reported by the
teachers, suggests that First Edition students at School I covered the least amount of
content. In general, First Edition students studied solving equations and inequalities,
translating verbal forms of a problem into symbolic form, slopes and graphs of lines,
solving systems (except at Schools H and I), and linear equations in two variables.
Quadratics were apparently studied only at Schools B, D, and J. First Edition students did
not study simplifying radical expressions or division with polynomials. First Edition
teachers at Schools H and I reported that their students did not have an opportunity to
learn statistics content assessed on the posttests.
Summary. Comparison of the content coverage for the Second Edition and First
Edition students suggests that, in general, they studied fairly comparable content, with the
exception of the study of quadratics. First Edition teachers were less likely than Second
Edition teachers to teach quadratics, perhaps a reflection of the fact that quadratics appear
later in the text of the First Edition than of the Second Edition. Hence, teachers who
worked through the text in order were less likely to reach the chapter on quadratics.
Second Edition and non-UCSMP Sample
Second Edition Classes. During the interviews conducted as part of the spring
visit, the three Second Edition teachers each expected to cover 12 chapters, with the
teacher at School Z expecting to cover only the first few sections of that chapter.
Table 8 contains the number of days spent on each chapter by the Second Edition
teachers in this sample. Information in the table, together with information gleaned from
teachers’ responses to the opportunity-to-learn forms and from the interviews, indicates
that Second Edition students generally studied solving equations and inequalities,
translating verbal forms to symbolic forms, solving systems, solving linear equations in
two variables, graphs and slopes of lines, and quadratics. Only students at School Y
apparently studied basics of statistics and probability.
35
Table 8. Days Spent on Each Chapter of the Second Edition, Including Testing, by
Teachers in the Second Edition and non-UCSMP Sample
Chapter
(Number of Lessons)
School
X
Y
Z
Ave.
Days
1.
Uses of Variables (9)
11
NR
21
16
2.
Multiplication in Algebra (10)
13
14
15
14
3.
Linear Expressions Involving Addition (10)
11
14
17
14
b
4.
Linear Expressions Involving Subtraction (9)
9
16
15
13.3
5.
Linear Sentences (8)
12
15
15
14
6.
Division in Algebra (10)
8a
NR
NR
na
c
14
7.
Slopes and Lines (9)
17
13
12
8.
Exponents and Powers (9)
14
13
11
12.7
9.
Quadratic Equations and Square Roots (9)
12d
13
12
12.3
15
11
e
10
f
10. Products, Factors, and Quadratics (8)
13
11. Systems (8)
e
12
12. Polynomials and Sets (7)
13. Functions (6)
g
8
5
NC
NC
9
13
10.3
NC
na
NC
na
Note: NR indicates that the form was not returned; NC indicates that the form was
returned and the chapter was not covered.
a
The teacher did not cover lesson 6-6 or 6-8 and spent 0.5 days on each of 6-9 and 6-10.
b
c
d
e
f
g
The teacher only covered the first four lessons of the chapter.
Non-UCSMP Classes. During the interviews, the teacher at School Y indicated
that she expected to cover 10 or 11 of the 12 chapters in the text and the teacher at School
Z expected to cover 12 of the 13 chapters. As previously indicated, the text used at
School X is not structured with chapters and the teacher gave no indication of the number
of lessons that she expected to cover.
Information from the interviews and the responses to the opportunity-to-learn
forms suggests that non-UCSMP students had an opportunity to study solving of
equations and inequalities as well as graphs of lines and equations for lines. Students at
Schools X and Y apparently studied systems of equations. Students at Schools Y and Z
apparently studied operations with polynomials and rational expressions. At none of the
36
three schools did non-UCSMP students appear to study graphs of quadratic equations.
Furthermore, the OTL results suggest that non-UCSMP students at Schools X and Y had
limited exposure to application problems or to problems focusing on properties of
numbers. In particular, the OTL responses from the teacher at School X to the
standardized test items raises questions about what content the teacher expected students
to master, including typical content in non-UCSMP texts such as factoring and
polynomial operations. More detail about this lack of exposure will be discussed in
Chapter 4 in which item-level results are presented together with the OTL results.
Summary. There appear to be some major differences in the opportunities to learn
algebra content between the Second Edition students and the non-UCSMP students. Both
groups of students had an opportunity to study solving equations and inequalities, graphs
of lines, and solving systems of linear equations. Non-UCSMP students at Schools Y and
Z studied polynomial operations and rational expressions. However, UCSMP students
generally studied applications of the concepts in the algebra text while non-UCSMP
students at Schools X and Y appeared to have limited exposure to applications. Perhaps
the wording of questions on both the UCSMP-constructed tests and the standardized tests
was sufficiently different from the wording in the textbooks to cause non-UCSMP
teachers to report that students did not have an opportunity to learn the content needed to
answer many of the items.
Instructional Practices and Issues
Some information about instructional practices, such as willingness to use group
work, use of reading from the text, or use of calculators, was obtained during the teacher
interviews. Other information was collected from students on the student survey.
Time Spent on Instruction
Classes in the Second Edition and First Edition sample ranged in length from 40
minutes (Schools H and J) to 50 minutes (Schools C and E), with a mean of 44.5 minutes
(s.d. = 3.9 minutes). For classes in the Second Edition and non-UCSMP sample, class
length ranged from 43 minutes (School Y) to 58 minutes (School X), with a mean of 51.3
minutes (s.d. = 7.6 minutes).
Technology Use
In the Second Edition and First Edition sample, students generally had access to
calculators; students either had their own calculator or had calculators available for use in
class. However, the Second Edition teacher at School C made the following comment
about calculator use:
They do it [work] on the calculator, and they don’t think they need to write down
what they’re going to do on the calculator. They think you can just punch in these
numbers, and you get your answer. It’s going to be right ‘cause you did it on the
calculator. … If you see what they’re actually doing, then you can say, you turned
37
your number around backwards, or you left out a parentheses, … you can find
what they did. But if you can’t, then you can’t tell them. (Second Edition teacher,
School C)
The First Edition teacher at School B also reported some use of graphing calculators.
On the student survey, students were asked, How often on the average, have you
used a calculator? Table 9 reports results from the Second Edition and First Edition
students. Overall, for students in this sample at least three-fourths of the students in both
groups reported using calculators almost every day; most of the other students indicated
using calculators 2-3 times a week. Hence, among students in this sample, the use of
calculators appears to be fairly pervasive. There were, however, some differences among
classes and pairs in terms of frequency of calculator use. At School C, both First Edition
classes reported somewhat less frequent use than the corresponding Second Edition
classes. Also, the frequency of calculator use at School G appeared to be somewhat less
overall than was true of the other classes in this sample.
Even though calculator technology use was regular, computer access was much
more limited. Teachers typically reported either lack of computer access or limited access
in terms of a computer lab for which they had to sign-up in order to take a class to the
lab. The Second Edition teacher at School E reported having students complete an
activity in the computer lab about once a month; likewise the Second Edition teacher at
School I reported using the computer lab for some graphing and some spreadsheet work.
In the Second Edition and non-UCSMP sample, calculator access was more
varied and somewhat less regular than in the Second Edition and First Edition sample,
particularly among non-UCSMP students (see Table 10). Second Edition teachers
reported that students had scientific calculators, with the teachers at Schools Y and Z
reporting some use of graphing calculators. In addition, over half of the Second Edition
students reported almost daily use of calculators, with most of the rest reporting use about
2-3 times per week.
However, among non-UCSMP students, a fourth of the non-UCSMP students
reported calculator use at less than once a month, with another fourth reporting use only
2-3 times a month. In fact, the non-UCSMP teacher at School Y indicated that she really
did not use calculators very much and made the following comment:
In this class we use the calculators, but not a lot; we don’t use the
calculators very much at all. The book doesn’t really offer itself much for
any calculator use except for when you’re doing maybe percents. … We
might get into it with the graphing, but the students don’t usually buy
graphing calculators. So most of them have a regular old scientific
calculator, so they don’t usually use the graphing unless I pass mine
around and let them all see it, which I’ll do, because when they get into
the higher level math areas they may want to get a graphing calculator.
(non-UCSMP teacher, School Y)
The responses from the non-UCSMP students at School Y agree with the comments from
the teacher, as most non-UCSMP students at this school reported limited calculator use.
Although the non-UCSMP teacher at School Z indicated use of calculators, he indicated
38
Table 9. Percent of Students Reporting Levels of Use of Calculators: Second Edition and First Edition
Frequency of
Calculator Use
almost every day
2-3 times a week
School B
Pair 2
2nd
1st
n = 18 n = 19
83
84
17
16
School C
Pair 4
2nd
1st
n = 12 n = 20
92
45
8
2-3 times a month
Pair 5
2nd
1st
n = 12 n = 8
100
50
40
almost every day
2-3 times a week
20
15
School H
Pair 14
Pair 15
2nd
1st
2nd
1st
n =16 n = 12 n = 11 n = 11
69
92
82
64
25
8
9
27
Pair 7
2nd
1st
n = 11 n = 12
55
92
36
8
Pair 8
2nd
1st
n = 19 n = 11
79
82
21
18
School I
Pair 17
2nd
1st
n = 16 n = 12
82
100
School J
Pair 18
Pair 19
2nd
1st
2nd
1st
n = 16 n = 18 n = 7 n = 11
94
94
100
82
18
6
9
39
6
Pair 9
2nd
1st
n = 10 n = 15
90
73
10
27
School G
Pair 12
2nd
1st
n = 10 n = 11
60
36
40
45
9
10
9
6
School E
9
12
2-3 times a month
less than once a month
Pair 6
2nd
1st
n = 6 n = 10
100
70
38
Frequency of
Calculator Use
School D
Overall
Results
2nd
1st
n = 164 n = 170
82
75
9
16
20
9
1
4
1
1
Table 10. Percent of Students Reporting Levels of Use of Calculators: Second Edition and non-UCSMP
Frequency of
Calculator Use
almost every day
School X
Pair 21
2nd
non
n = 14 n = 14
29
43
2-3 times a week
43
36
2-3 times a month
7
14
Pair 22
2nd
non
n = 21 n = 16
76
24
School Y
Pair 23
2nd
non
n = 19 n = 17
58
Pair 24
2nd
non
n = 25 n = 16
68
School Z
Pair 25
Pair 26
2nd
non
2nd non
n = 11 n = 17 n = 8 n = 11
64
71
25
64
31
26
6
20
6
18
24
50
14
44
16
41
4
44
18
6
25
7
25
53
4
50
40
27
9
Overall
Results
2nd
non
n = 98 n = 91
58
27
28
21
9
26
3
25
that he did not want to overemphasize them. However, his students reported regular use
of calculators.
In general, neither Second Edition nor non-UCSMP teachers had much access or
opportunity to engage students in computer work. The non-UCSMP teacher at School X
sometimes went to a computer lab and the Second Edition teacher did some computer
work on a large class monitor. However, in general, the teachers in this sample, like those
in the Second Edition and First Edition sample, did not use computers much.
Use of Reading
As is true of all the UCSMP texts, there is an assumption that students will read
their Algebra textbook, whether using the First Edition or the Second Edition. During the
teacher interviews, all of the Second Edition and First Edition teachers who were asked
about reading indicated they expected students to read, and consequently, assigned the
reading. Teachers indicated they handled the reading in different ways. At School B, the
Second Edition teacher indicated that sometimes the class read the lesson together and
sometimes students read it on their own. At School C, the Second Edition teacher usually
had students complete the Covering the Reading section of the problems on their own.
The Second Edition teacher at School G had students take notes on the reading.
However, the First Edition teacher at School C indicated that many students
fought her on the reading, the First Edition teacher at School H indicated that it was not
always clear that students did the reading, and the Second Edition teacher at School H
indicated that some students were reading at the third- and fourth-grade levels. One
teacher made the following comment about reading in relation to lecturing:
… I seem to have found that the more I explain the less they read. Because
if I’m going to tell them tomorrow anyway, they don’t need to read. So I
try to really emphasize not explaining very much except during the
answering of questions. (Second Edition teacher, School D)
Students were asked to respond to the statement, I read the explanations in my
algebra textbook …, with choices from almost always to almost never. Table 11 contains
responses from the students in the Second Edition and First Edition sample. For the
students in this sample, most students in both groups indicated that they read their
textbook at least sometimes. However, when considering only the almost always and very
often responses, Second Edition students reported reading somewhat more often than
their First Edition counterparts. There are some class differences of note. For instance,
slightly more than a fourth of the Second Edition students at School B indicated hardly
ever reading their text. Also a fourth of the First Edition students in pair 14 at School H
indicated almost never reading their textbook, reinforcing comments from the teacher
about students’ reading habits.
Among the Second Edition and non-UCSMP teachers, all three Second Edition
teachers expected students to read; the issue of reading was not discussed with the nonUCSMP teachers. Table 12 contains the responses from students in the Second Edition
41
Table 11. Percent of Students Indicating Frequency of Reading Textbook Explanations: Second Edition and First Edition
Frequency of
Reading Textbook
almost always
School B
Pair 2
2nd
1st
n = 18 n = 19
11
16
School C
Pair 4
2nd
1st
n = 12 n = 20
17
10
School D
Pair 5
2nd
1st
n = 12 n = 8
17
38
Pair 6
2nd
1st
n = 6 n = 10
33
30
School E
Pair 7
2nd
1st
n = 11 n = 12
18
42
Pair 8
2nd
1st
n = 19 n = 11
26
Pair 9
2nd
1st
n = 10 n = 15
30
33
School G
Pair 12
2nd
1st
n = 10 n = 11
30
very often
28
21
33
25
17
25
33
30
27
42
37
27
50
20
30
27
sometimes
33
53
42
65
50
37
33
30
45
8
26
73
10
33
30
55
not very often
28
9
8
11
almost never
Frequency of
Reading Textbook
almost always
11
8
31
sometimes
50
not very often
6
10
10
School H
Pair 14
Pair 15
2nd
1st
2nd
1st
n =16 n = 12 n = 11 n = 11
13
36
18
very often
almost never
17
School I
Pair 17
2nd
1st
n = 16 n = 12
19
25
10
School J
Pair 18
Pair 19
2nd
1st
2nd
1st
n = 16 n = 18 n = 7 n = 11
6
22
43
27
Overall
Results
2nd
1st
n = 164 n = 170
21
19
36
27
19
25
56
33
43
18
34
25
58
18
36
56
42
19
33
14
55
34
45
17
9
18
6
8
19
11
10
5
2
6
25
42
13
18
Table 12. Percent of Students Indicating Frequency of Reading Textbook Explanations: Second Edition and non-UCSMP
Frequency of
Reading Textbook
almost always
School X
Pair 21
2nd
non
n = 14 n = 14
14
Pair 22
2nd
non
n = 21 n = 16
14
13
School Y
Pair 23
2nd
non
n = 19 n = 17
16
18
Pair 24
2nd
non
n = 25 n = 16
28
6
very often
21
29
29
38
16
12
24
sometimes
57
29
48
31
47
47
36
not very often
21
10
13
5
12
almost never
21
6
16
12
School Z
Pair 25
Pair 26
2nd
non
2nd non
n = 11 n = 17 n = 8 n = 11
18
65
18
9
6
50
55
29
4
19
8
25
43
Overall
Results
2nd
non
n = 98 n = 91
17
21
27
19
18
75
36
49
37
9
13
9
6
12
9
12
9
7
12
and non-UCSMP sample to the survey item, I read the explanations in my algebra
textbook …. About equal percentages of Second Edition and non-UCSMP students
indicated reading their textbook almost always or very often. However, about a fourth of
the non-UCSMP students indicated very little reading of their textbook.
Homework
Mathematics teachers typically assign homework to support students’ learning.
Although teachers were not queried about their homework expectations, students were
asked to identify the amount of time spent on homework and the frequency of needing
help. Table 13 contains the responses from the Second Edition and First Edition sample
to the two survey items about homework.
About half of the Second Edition students and a third of the First Edition students
reported spending 16-30 minutes per day on homework; about a third of the Second
Edition and a fourth of the First Edition students reported spending from 31-45 minutes
per day on homework. About 5% of the students in each group reported spending more
than 1 hour per day working on homework.
Most Second Edition and First Edition students reported needing help with their
homework at least sometimes, with about 70% of the students in each group needing
some level of help. Overall, roughly 10-15% of the students reported almost never
needing help. There was considerable variability in responses to this question across
classes. At least a fourth of the Second Edition students in pairs 4, 6, 17, and 18 reported
almost never needing help with their homework; likewise, at least a fourth of the First
Edition students in pairs 12 and 14 reported this minimal level of needed help.
Table 14 contains results from the two homework items for the Second Edition
and non-UCSMP sample. There were some differences in the response patterns from
these two groups of students. About half of the Second Edition students reported
spending at most 30 minutes per day on homework; slightly more than 70% of nonUCSMP students reported this level of homework. Over 40% of the Second Edition
students at School X reported spending more than 45 minutes per day on homework and
over a fourth of Second Edition Students in pairs 24 and 25 reported this much daily time
on homework.
Second Edition and non-UCSMP students responded in similar ways to the item
dealing with needing help with homework. Over a third of the students in each group
reported seldom needing help with homework.
44
Table 13. Percent of Students Spending Various Amounts of Time on Homework and Needing Various Levels of Help With Their
Homework: Second Edition and First Edition
Time on Homework
and
Frequency of Help
School B
Pair 2
2nd
1st
n = 18 n = 19
School C
Pair 4
Pair 5
2nd
1st
2nd
1st
n = 12 n = 20 n = 12 n = 8
School D
Pair 6
Pair 7
2nd
1st
2nd
1st
n = 6 n = 10 n = 11 n = 12
School E
Pair 8
Pair 9
2nd
1st
2nd
1st
n = 19 n = 11 n = 10 n = 15
School G
Pair 12
2nd
1st
n = 10 n = 11
0-15 minutes per day
17
About how much time have you spent, on the average, on your algebra homework?
11
17
30
17
25
20
27
11
10
7
20
36
33
37
33
55
17
50
17
40
18
42
21
18
40
27
10
36
28
21
25
10
42
13
83
20
27
42
47
55
40
47
40
18
17
11
17
5
8
20
9
8
11
9
10
13
30
16
8
18
8
11
18
more than 1 hour per day
17
12
How often have you needed help in doing your algebra homework?
8
20
17
13
10
9
8
11
almost always
11
21
very often
11
37
8
15
25
25
17
20
27
25
26
18
sometimes
61
32
25
35
25
62
50
40
55
25
42
45
not very often
11
11
25
20
17
20
9
33
5
almost never
6
33
10
17
8
16
33
45
10
7
10
13
18
7
40
18
70
20
10
27
18
10
47
40
9
18
10
13
10
27
Table 13 continued
Time on Homework
and
Frequency of Help
School H
Pair 14
Pair 15
2nd
1st
2nd
1st
n =16 n = 12 n = 11 n = 11
School I
Pair 17
2nd
1st
n = 16 n = 12
School J
Pair 18
Pair 19
2nd
1st
2nd
1st
n = 16 n = 18 n = 7 n = 11
25
67
9
27
38
25
13
11
36
38
33
Overall
Results
2nd
1st
n = 164 n = 170
17
22
27
18
50
33
12
39
29
45
27
37
12
45
36
12
8
56
39
29
18
35
25
19
18
25
6
11
29
13
8
6
5
6
almost always
19
42
27
36
6
14
9
10
15
very often
44
8
18
18
6
8
13
8
43
18
21
19
sometimes
19
17
36
36
44
42
31
42
14
27
38
34
not very often
13
8
9
9
25
33
25
33
14
36
16
22
almost never
6
25
9
25
17
25
17
14
9
15
10
9
8
46
Table 14. Percent of Students Spending Various Amounts of Time on Homework and Needing Various Levels of Help With Their
Homework: Second Edition and non-UCSMP
Time on Homework
and
Frequency of Help
School X
Pair 21
2nd
non
n = 14 n = 14
Pair 22
2nd
non
n = 21 n = 16
School Y
Pair 23
2nd
non
n = 19 n = 17
Pair 24
2nd
non
n = 25 n = 16
School Z
Pair 25
Pair 26
2nd
non
2nd non
n = 11 n = 17 n = 8 n = 11
Overall
Results
2nd
non
n = 98 n = 91
7
14
29
25
11
47
16
19
18
12
25
36
17
25
21
14
38
63
47
29
28
75
27
59
50
45
35
48
14
29
24
6
32
12
24
6
27
29
25
9
25
15
14
36
5
6
5
6
28
9
9
12
9
29
7
5
5
6
4
18
9
2
almost always
21
9
6
11
very often
29
sometimes
29
not very often
almost never
6
12
8
25
9
12
43
6
26
29
20
25
36
24
13
18
29
18
57
19
38
21
18
40
19
27
29
50
45
30
33
7
36
19
38
21
18
20
13
18
29
37
9
19
24
7
7
19
6
32
24
12
19
9
6
18
15
13
47
Other Instructional Issues
Teachers were queried about the extent to which they had students work in small
groups. Among the Second Edition teachers in the Second Edition and First Edition
study, all teachers indicated they used some group work, with some of that work
occurring in groups of two. The Second Edition teacher at School G used group work
particularly when going over homework. Among First Edition teachers, only the teachers
at Schools B and J reported not using much group work. The First Edition teacher at
School C did use group work but expressed some concern that students get too social.
The First Edition teacher at School D reported doing a group activity once or twice per
chapter. The following comments reflect perspectives about the use of group work:
I like to use group work, especially as a discovery type of activity with the
answers. (Second Edition teacher, School D)
[in response to a question about using groups] I’ll say work in groups for
your daily [work]… you can pair up. Some do, some don’t. And at one
point, … I said “Let’s talk about this, how about we set up some group
work, or started to work in groups, and let’s say I collect one paper from
the group.” And [a student] said, “But I’m not going to be responsible for
so-and-so not doing their work.” And I kind of feel that way too, well,
why should I make a student responsible for somebody else who’s a
chronic not-doer. (First Edition teacher, School H)
The Second Edition of UCSMP Algebra included projects at the end of each
chapter; these were designed to provide opportunities for students to work on extended
tasks outside of class. Not all teachers were queried about the use of projects. However,
the Second Edition teacher at School B reported using some projects and the teacher at
School C used projects at the end of each chapter as extra credit. At School G, projects
were assigned each quarter and accounted for 20% of the quarter grade. Both teachers at
Schools I and J reported some use of projects, although the teacher at School J indicated
not using them as much as he should have. The following comments were made about
projects:
They like those [projects]. I’ve had them required and I’ve had them
optional. They work well on them either way, whether it’s optional or
whether it’s something that they’ve got to do. On my team we have A, B,
C, and I. No D and F grades. An I becomes an F, so the lowest grade
really they can get is a C. So, the projects for those kids who need just a
little bit more, those projects are real good to assign, just to get them up to
a C minus. (Second Edition teacher, School B)
A few teachers also discussed writing. In particular, the Second Edition teacher at
School B used journals, the Second Edition teacher at School H also expected students to
write, and the First Edition teacher at School B gave students some writing experience.
48
Keeping a journal, quite often I have them do other writings, like in the
teacher’s notes, they’ll give you suggestions for other things I have them
write. Write about that or I think up something to write or I’ll bring a
problem out of the blue for them to write about. (Second Edition teacher,
School B)
One other instructional issue arose during the teacher interviews. UCSMP
generally advocates doing a pace of a lesson a day, and recommends this pace because of
the continual review that is built into the exercise sets. The First Edition teacher at School
H had slowed down and provided extra worksheets for students during lessons on
equation solving. However, the teacher noted that other teachers were about a chapter
ahead of her and were getting the same results as she was getting while still keeping the
pace of a lesson a day. The issue of the built-in review was commented upon by one of
the teachers:
That they do get them [skills and practice] in the review, here it comes
again, here it comes again, and I bet by the time you’re done with the
chapter they had just as much practice on that [a particular concept]. It just
hasn’t been there on the first day. (First Edition teacher, School I)
In the Second Edition and non-UCSMP sample, all the Second Edition teachers
indicated that they had students work in groups; the non-UCSMP teachers at Schools X
and Z indicated some use of group work and the non-UCSMP teacher at School Y used
group projects.
The Second Edition teacher at School X indicated that she would have students do
some of the projects.
Summary
This chapter has compared the implemented curriculum and instructional
environment in the classrooms of the Second Edition and First Edition sample as well as
the Second Edition and non-UCSMP sample. Thirteen pairs in eight schools comprise the
Second Edition and First Edition sample; six pairs in three schools comprise the Second
Edition and non-UCSMP sample.
The main research question addressed in this chapter is the following: How do
teachers’ instructional practices when using UCSMP Algebra (Second Edition, Field
Trial Version) compare to teachers’ instructional practices when using UCSMP Algebra
(First Edition) or the non-UCSMP materials currently being used in the schools? Overall,
the Second Edition and First Edition teachers implemented a curriculum with many
similarities. Students in both groups generally studied solving equations and inequalities,
translating verbal forms of a problem into symbolic form, slopes and graphs of lines,
solving systems, basics of quadratics, and linear equations in two variables. Both First
and Second Edition students at School I appeared to cover the least amount of content, at
least as measured by the opportunity-to-learn forms for the posttest measures.
The UCSMP Algebra students, regardless of whether they were using the Second
49
Edition or the First Edition, had access to calculators and used them very frequently.
About three-fourths of the students in both groups reported using calculators almost every
day, with most of the rest reporting use 2-3 times a week. Computer access, however, was
very limited or non-existent.
The UCSMP Algebra teachers, both First and Second Edition, expected their
students to read their textbook. Although both groups of students reported reading their
textbook at least sometimes, Second Edition students were somewhat more likely than
First Edition students to report reading their book almost always or very often (55% vs.
44%).
About a fourth of the Second Edition students reported spending 16-30 minutes
per day on homework, with another third reporting spending 31-45 minutes per day.
Among First Edition students, slightly more than a third reported spending 16-30 minutes
per day on homework, with another fourth reporting spending 31-45 minutes per day on
homework. Also, about 70% of the students in each group reported needing help with
homework at least sometimes; about 10-15% of each group reported almost never
needing help with their homework.
Among teachers in the Second Edition and First Edition sample, all Second
Edition teachers and six of the First Edition teachers reported having students work in
groups. Five of the Second Edition teachers also reported some use of the projects
provided in the text.
For students in the Second Edition and non-UCSMP sample, content coverage
was more varied. Although the indication of chapters covered by teachers in the two
groups would suggest that much similar content was covered, responses to the OTL
forms by the non-UCSMP teachers, particularly at Schools X and Y, suggest students had
limited exposure to application problems or to problems dealing with properties of
numbers. Both Second Edition and non-UCSMP students studied solving equations and
some inequalities and systems of linear equations (except for non-UCSMP students at
School Z). The OTL forms raise doubts about whether any quadratics were studied by
non-UCSMP students, although this topic was studied by Second Edition students.
In terms of calculator use, Second Edition students were likely to report more
frequent use of calculators than their non-UCSMP counterparts. A fourth of the nonUCSMP students reported almost never using a calculator. As in the other sample,
computer access was again extremely limited or non-existent.
Second Edition teachers in the Second Edition and non-UCSMP sample also
expected their students to read. Although about equal percentages of Second Edition and
non-UCSMP students reported reading their textbook almost always or very often (36%
vs. 39%), about a fourth of the non-UCSMP students indicated very rarely reading their
textbook.
About half of the Second Edition students and about 70% of non-UCSMP
students reported spending no more than 30 minutes per day on homework. Also, about a
third of the students in each group reported rarely needing help with their homework.
All teachers in the Second Edition and non-UCSMP sample reported having
students work in groups to some extent.
50
CHAPTER 4
THE ACHIEVED CURRICULUM
The achievement of students studying from UCSMP Algebra, both Second
Edition and First Edition, and the non-UCSMP curriculum in place at the school was
measured at the end of the school year by three instruments: the High School Subjects
Tests: Algebra (a standardized multiple-choice posttest); a UCSMP-constructed Algebra
Test; and one of two forms of a free-response UCSMP Problem-Solving and
Understanding Test. Copies of the UCSMP-constructed posttest instruments are provided
in Appendix C. Copies of the rubrics used for scoring both forms of the Problem-Solving
and Understanding Test are included in Appendix D.
The achievement results are presented in three main sections. The first section
deals with achievement on the standardized multiple-choice posttest. The second section
deals with achievement on the UCSMP-constructed Algebra Test, including overall
achievement as well as item-level achievement. The third and final section reports
achievement on the Problem-Solving and Understanding Test, again including overall
achievement and achievement by individual item.
Overall achievement on the High School Subject Tests: Algebra and on the
UCSMP Algebra Test is discussed in three ways. First, achievement on the entire test is
reported along with the percentage of the items for which teachers indicated that their
students had an opportunity-to-learn the content assessed on the items. Note that
achievement reported in this manner includes performance on all items, regardless of
whether or not students had an opportunity to learn the needed content. Second,
achievement is reported on a subtest consisting of only those items for which both
teachers in the school indicated that their students had an opportunity to learn the needed
content. Because both the Second Edition and First Edition teachers or both the Second
Edition and the non-UCSMP teachers at the school viewed these items as fair to their
students, this is called the Fair Test. Third, when possible, overall achievement is
reported on a test consisting of only those items for which all teachers in the sample
indicated that their students had an opportunity to learn the needed content. This test,
which assesses achievement on the intersection of the implemented curricula in all the
schools in the respective sample, is called the Conservative Test. To facilitate
comparisons across all three methods of reporting achievement, results on each test are
presented as mean percents correct.
Achievement on the High School Subject Tests: Algebra
Achievement on the Entire Test
Table 15 contains the results on the entire standardized test by matched pairs for
both samples.
51
Table 15. Mean Percent Correct and Teacher’s Reported OTL on the Content of the High
School Subject Tests: Algebra
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
Overall
2
4
5
6
7
8
9
12
14
15
17
18
19
X
Y
Y
Y
Z
Z
Overall
21
22
23
24
25
26
Comparison
n
Mean SD OTL
n
Mean SD OTL
SE
UCSMP Second Edition and UCSMP First Edition Samplea
18
37.6
18.8
93
19
44.2
13.5
98
2.15
12
39.0
11.6
90
20
37.8
11.8
78
1.71
12
39.8
10.5
90
8
41.6
17.6
78
2.51
6
48.3
13.1
93
10
51.8
12.9 100
2.67
11
48.6
11.9
93
12
46.5
10.5 100
1.70
19
57.4
11.4
93
11
60.5
15.0
98
1.94
10
58.8
8.0
93
15
55.5
17.7
98
2.39
10
44.3
10.1
78
11
33.9
9.9
85
1.75
16
30.0
10.4
93
12
33.5
11.6
68
1.67
11
41.6
14.1
93
11
34.3
9.0
68
2.02
16
47.0
15.6
53
12
50.6
11.5
73
2.14
16
48.6
13.3
95
18
45.3
16.9
83
2.10
7
43.6
15.1
95
11
49.8
14.7
83
2.87
164
44.8
15.0
170
44.8
15.4
14
21
19
25
11
8
98
41.1
54.5
57.0
45.5
39.5
39.7
47.9
UCSMP Second Edition and non-UCSMP Sampleb
18.2
95
14
51.3
13.4
58
17.3
98
16
53.9
15.7
80
13.9
98
17
46.5
17.0
80
13.5
98
16
43.9
14.8
80
13.6
93
17
43.4
11.6
93
13.1
93
11
34.3
9.5
93
16.3
91
46.0
14.9
2.42
2.20
2.06
1.80
1.92
2.07
t
df
p
-1.22
0.28
-0.28
-0.51
0.47
-0.64
0.54
2.37
-0.85
1.44
-0.67
0.63
-0.87
35
30
18
14
21
28
23
19
26
20
26
32
16
0.229
0.780
0.781
0.617
0.645
0.53
0.592
0.028*
0.405
0.164
0.508
0.533
0.400
-1.68
0.11
2.04
0.35
-0.80
1.04
26
35
34
39
26
17
0.104
0.911
0.049*
0.725
0.431
0.314
* indicates difference in means between the pairs is statistically significant.
a
A matched-pairs t-test indicates that the difference in achievement of students studying
from the Second Edition or First Edition curricula is not statistically significant
( x = −0.054, s x = 5.17, t = −0.038, p = 0.971) .
b
from the Second Edition or non-UCSMP curricula is not statistically significant
( x = 0.667, s x = 7.20, t = 0.27, p = 0.829) .
Because the classes in each pair were matched on the pretest scores, each pair
represents a replication of the study. The difference in the mean percent correct between
the scores of the Second Edition and First Edition classes varies from -6.6% (pair 2) to
10.4% (pair 12). Only for pair 12 at School G is the difference in the mean percent
correct statistically significant, in favor of the Second Edition class. However, a matchedpairs t-test on the mean of the pair differences indicates that Second Edition and First
Edition students performed about equally on this standardized test. 3
3
A matched-pairs t-test or repeated measures t-test is appropriate in this situation. Because the samples
were matched at the beginning of the study, they are considered dependent. The matched pairs t-test on the
52
For the Second Edition and First Edition sample, the mean percent correct by
class varies from 30.0% (Second Edition, pair 14) to 60.5% (First Edition, pair 8). These
percents correspond to mean raw scores by class from 12 to 24.2 out of 40. Because the
High School Subject Tests: Algebra is a standardized measure, it is possible to compare
the results of both groups to national percentile rankings. The lowest score for the Second
Edition and First Edition sample corresponds to the 18th percentile; the highest score
corresponds to the 69th percentile. For both the Second Edition and First Edition students
in this sample, the overall mean score corresponds to the 45th percentile. Table 15 also
highlights considerable variability in achievement across schools and between classes
within the same school. Among Second Edition classes, the range of percent correct is
about 28%; among First Edition classes, the range is about 27%.
There is considerable variability in the OTL percentages across schools and
between classes in the same school. For the Second Edition classes, OTL ranges from a
low of 53% (School I) to a high of 95%; for First Edition classes, OTL ranges from 68%
(School H) to 100% (School D). At Schools H and I, there is at least a 20% difference in
the OTL percentages between Second Edition classes and First Edition classes. However,
it is not clear that there is a relationship between OTL and percentage correct. Some
Second Edition classes with high OTL percentages have high achievement (e.g., classes
at Schools D and E) while others have low achievement (e.g., classes at School H);
among First Edition classes, some classes with low OTL percentages have high
achievement (e.g., at School I) and others have low achievement (e.g., at Schools C and
H).
For the Second Edition and non-UCSMP sample, the difference in the mean
percent correct between the scores of the Second Edition and non-UCSMP classes varies
from -10.2% (pair 21) to 10.5% (pair 23). Only for pair 23 at School Y is the difference
in the mean percent significantly different, in favor of the Second Edition class; notice
that there is a 18% difference in the OTL response for the pairs in this school,
corresponding to a difference of about 7 items for which students had an opportunity to
learn the content to answer the items. However, the results from a matched-pairs t-test on
the mean of the pair differences indicate that Second Edition and non-UCSMP students
performed comparably on this standardized test overall. This is true even though there is
considerable variability in the OTL among the non-UCSMP classes.
The mean percent correct for the classes in the Second Edition and non-UCSMP
sample varies from 34.3% (non-UCSMP, pair 26) to 57.0% (Second Edition, pair 23).
These percents correspond to mean raw scores by class from 13.7 to 22.8 out of 40. For
this sample, the lowest score corresponds to the 26th percentile; the highest score
corresponds to the 65th percentile. Overall achievement corresponds to the 48th
percentile for Second Edition students and the 45th percentile for non-UCSMP students.
Among Second Edition classes, the range of percent correct is 18%; among non-UCSMP
classes, the range of percent correct is about 20%.
mean of the differences between the pairs provides a method to test the overall effect of the two curricula
(Gravetter and Wallnau 1985, p. 373).
53
Achievement on the Fair Test
Table 16 contains the achievement results for the Fair Tests for the pairs in both
samples. Among the UCSMP Second Edition and First Edition sample, the number of
items in the Fair Tests varied from 13 items at School I to 37 items at Schools B and D.
Among the Second Edition and non-UCSMP sample, the number of items varied from 16
items at School X to 35 items at School Z.
As with achievement on the entire test, a matched-pairs t-test indicates that
achievement with the Second Edition and First Edition curricula is roughly comparable,
at least on the subtest of the standardized test for which both teachers in a pair indicated
that students had the chance to learn the assessed content. Once again, the difference in
the means for pair 12 at School G is significant, in favor of the Second Edition class. In
this pair, the Second Edition students did at least 20% better than the First Edition
students on over half of the items on the Fair Test.
Similarly, for the Second Edition and non-UCSMP sample, the two groups
performed comparably on the Fair Tests. There were no significant differences in
achievement between the classes in any pairs.
Achievement on the Conservative Test
Table 17 contains the mean percent correct by matched pair for the Conservative
Test for each sample. For the Second Edition and First Edition sample, the Conservative
Test consists of the 8 items for which all teachers indicated that students had an
opportunity to learn the assessed content. These items deal with evaluating a variable
expression for specific values, simplifying an algebraic expression, solving a linear
equation with multiple steps, finding the product of two rational fractions, finding the
area of a rectangle whose length and width are variable expressions, and solving an
application involving rates. Among these 8 items, 7 (87.5%) deal with skills and 1
(12.5%) deals with a real-world application.
Again, there is no overall significant difference in achievement between students
studying from the Second Edition or First Edition curricula. The difference in the mean
percent correct between the Second Edition and First Edition classes varies from -10.3%
(pair 2) to 11.1% (pair 12). There are no significant differences between the means at the
pair level.
Overall, for the Second Edition and First Edition sample, achievement on the
Conservative Test varies from 46.9% (First Edition, pair 14) to 84.1% (First Edition, pair
8). On only one item (simplifying the product of two rational fractions) was the
difference in the overall percent correct at least 10%.
For the Second Edition and non-UCSMP sample, the Conservative Test consists
of 13 items, including 5 of the 8 items from the Conservative Test in the Second Edition
and First Edition sample. Among these 13 items, three deal with evaluating an algebraic
expression for specific values, two with solving a linear equation with multiple steps, two
with multiplying fractions with algebraic expressions, two with simplifying a rational
expression, one with translating from verbal to symbolic form, one with the values for
which a rational expression is undefined, one with solving a quadratic equation with no
54
Table 16. Mean Percent Correct on the Fair Tests from the High School Subject Tests:
Algebra
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
X
Y
Y
Y
Z
Z
df
p
2
4
5
6
7
8
9
12
14
15
17
18
19
Comparison
n
Mean
SD
n
Mean
SD
t
18
39.8
19.2
19
46.8
14.6
-1.04
12
40.8
12.9
20
40.5
12.3
0.07
12
42.5
10.6
8
42.9
17.8
-0.06
6
48.2
13.1
10
52.2
14.2
-0.56
11
46.9
12.6
12
46.4
10.5
0.10
19
56.3
12.1
11
59.1
16.3
-0.54
10
58.3
8.8
15
54.3
18.5
0.63
10
45.2
11.0
11
34.7
10.2
2.27
16
32.7
11.5
12
38.1
14.4
-1.10
11
46.5
15.2
11
39.2
10.7
1.30
16
57.7
20.3
12
59.6
11.4
-0.29
16
51.4
15.1
18
48.3
18.2
0.54
7
47.3
16.9
11
52.8
15.3
-0.71
35
30
18
14
21
28
23
19
26
20
26
32
16
0.310
0.948
0.950
0.584
0.918
0.595
0.532
0.035*
0.280
0.208
0.774
0.595
0.485
21
22
23
24
25
26
44.6
21.5
14
56.3
16.6
-1.61
55.5
18.4
16
57.6
18.0
-0.36
58.1
15.6
17
48.9
17.5
1.67
47.3
14.8
16
46.5
16.5
0.16
38.7
14.1
17
43.5
12.2
-0.96
41.1
14.4
11
34.0
10.4
1.25
26
35
34
39
26
17
0.119
0.722
0.105
0.873
0.347
0.228
14
21
19
25
11
8
Note: Items comprising each Fair Test are as follows: School B, 37 items (1-30, 32, 3437, 39, 40); School C, 30 items (1-16, 18, 20-22, 25, 28-30, 32, 34, 36-39); School D, 37
items (1-8, 10-19, 21-35, 37-40); School E, 36 items (1-16, 18-23, 25, 26, 28-39); School
G, 27 items (1-6, 8-10, 12, 13, 15, 18-20, 22, 23, 25, 26, 28-30, 32, 34, 36, 37, 39);
School H, 26 items (1-13, 15, 18-20, 2, 25, 28-30, 32, 35-37); School I, 13 items (1-3, 6,
8, 9, 13, 25, 26, 29, 34, 36, 40); School J, 32 items (1-4, 6-16, 18, 20, 21, 22, 24-26, 28,
29. 31, 32, 34-37, 39, 40); School X, 16 items (1, 3, 4, 6-8, 10, 11, 23, 29, 31, 32, 34-36,
39); School Y, 32 items (1-12, 14-16, 18-26, 28, 29, 32-36, 40); and School Z, 35 items
(1-6, 8-13, 15, 16, 18-23, 25-32, 34-40).
a
from the Second Edition or First Edition curricula is not significantly different
( x = −0.10, s x = 5.21, t = −0.069, p = 0.946) .
b
from the Second Edition or non-UCSMP curricula is not significantly different
( x = −0.25, s x = 7.74, t = −0.079, p = 0.940) .
55
Table 17. Mean Percent Correct on the Two Conservative Subtests of the High School
Subject Tests: Algebra
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
Overall
2
4
5
6
7
8
9
12
14
15
17
18
19
X
Y
Y
Y
Z
Z
Overall
21
22
23
24
25
26
Comparison
n
Mean
SD
n
Mean SD
t
a
UCSMP Second Edition and UCSMP First Edition Sample
18
54.2
21.9
19
64.5
20.5
-1.17
12
59.4
22.7
20
63.8
24.3
-0.51
12
67.7
17.2
8
60.9
26.3
0.70
6
75.0
11.2
10
67.5
16.9
0.96
11
64.8
20.8
12
63.5
11.3
0.19
19
79.6
13.3
11
84.1
13.8
-0.88
10
76.3
10.9
15
77.5
20.7
-0.16
10
71.3
17.7
11
60.2
13.5
1.63
16
52.3
18.4
12
46.9
22.1
0.71
11
62.5
23.7
11
56.8
16.2
0.66
16
67.2
19.8
12
69.8
14.6
-0.38
16
71.9
20.2
18
66.7
25.7
0.65
7
57.1
21.5
11
61.4
21.3
-0.42
164
65.9
20.3
170
65.1
21.1
14
21
19
25
11
8
98
48.4
20.3
14
62.6
15.9
59.0
19.5
16
60.1
21.2
63.2
21.2
17
50.2
22.3
56.0
16.8
16
54.3
16.7
42.7
15.1
17
54.3
13.5
50.0
20.1
11
39.2
12.1
54.9
19.6
91
54.0
18.5
-2.06
-0.16
1.79
0.32
-2.12
1.46
df
p
35
30
18
14
21
28
23
19
26
20
26
32
16
0.253
0.615
0.492
0.353
0.852
0.386
0.874
0.121
0.487
0.518
0.705
0.520
0.683
26
35
34
39
26
17
0.049*
0.871
0.082
0.753
0.044*
0.162
Note: For the Second Edition and First Edition sample, the Conservative Test consists of
8 items (1, 2, 3, 6, 8, 13, 25, 29). For the Second Edition and non-UCSMP sample, the
Conservative Test consists of 13 items (1, 3, 4, 6, 8, 10, 11, 23, 29, 32, 34, 35, 36).
* indicates significant difference between the means of the pair.
a
A matched-pairs t-test indicates that the difference in achievement between students
studying from the Second Edition or First Edition curricula is not significant
( x = 1.21, s x = 6.26, t = 0.70, p = 0.500) .
b
studying from the Second Edition or non-UCSMP curricula is not significant
( x = −0.23, s x = 11.19, t = −0.05, p = 0.961) .
56
linear term, and one with finding an equation for a line; 11 of the 13 items (84.6%) deal
with skills, 1 (7.7%) with properties, and 1 (7.7%) with representations.
Again, there is no overall significant difference in achievement between students
studying from the Second Edition or non-UCSMP curricula. Differences in the mean
percent correct vary from 14.2% (pair 21) to 13% (pair 23). There are two pair
differences that are significant, both in favor of the non-UCSMP class.
At the class level, achievement varies from 39.2% (non-UCSMP, pair 26) to
63.2% (Second Edition, pair 23). On three of the thirteen items, the overall difference in
the mean percent correct was greater than 10%. On one of the three items, dealing with
evaluating an expression for specific values, the difference was in favor of the nonUCSMP students. On the other two items, dealing with multiplying fractions with
algebraic expressions and solving a quadratic equation with no linear term, the
differences favored the Second Edition students.
Summary
The results for the High School Subject Tests: Algebra, a standardized test,
indicate that there are no significant differences in achievement between students
studying from the Second Edition or First Edition curricula or between students studying
from the Second Edition or non-UCSMP curricula, regardless of how the data are
analyzed. As might be expected, when OTL was controlled on the Fair Tests and on the
Conservative Tests, the achievement was higher than on the overall test.
Achievement on the UCSMP Algebra Test
The UCSMP-constructed Algebra Test consists of 40 multiple-choice items. This
section discusses overall achievement as well as achievement at the item level.
Overall Achievement
Achievement on the Entire Test. Table 18 contains the results on the entire test by
matched pairs. A matched-pairs t-test on the mean of the pair differences for the Second
Edition and First Edition sample indicates that the difference in achievement between
students using the two curricula is not significantly different. The difference in the mean
percent correct between the scores of the Second Edition and First Edition students varies
from -8.8% (pair 19) to 18.3% (pair 12). For three pairs (4, 12, and 15), the difference in
the mean percents was statistically significant, all in favor of the Second Edition classes.
Table 18 highlights considerable variability in achievement across schools. For
the Second Edition and First Edition sample, the mean percent correct by class varies
from 35.6% (First Edition, pair 14) to 70.2% (First Edition, pair 8). These correspond to
mean raw scores by class from 14.2 to 28.1 out of 40, respectively. Among the Second
Edition classes, the range of percent correct is about 28%; for the First Edition classes in
this sample, the range of percent correct is about 35%.
57
Table 18. Mean Percent Correct and Teachers’ Reported OTL on the Content of the
UCSMP Algebra Test
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
Overall
2
4
5
6
7
8
9
12
14
15
17
18
19
X
Y
Y
Y
Z
Z
Overall
21
22
23
24
25
26
Comparison
n
Mean SD OTL
n
Mean SD OTL
SE
a
18
52.4
14.1
95
19
58.3
12.6 100
1.76
12
55.2
16.1
95
20
43.8
13.0
80
2.08
12
50.6
16.9
95
8
39.7
15.8
80
3.01
6
55.0
11.2
93
10
59.8
12.6
95
2.50
11
61.6
12.0
93
12
57.7
11.0
95
1.91
19
64.7
12.0
98
11
70.2
12.6
93
1.85
10
66.0
11.3
98
15
59.0
17.2
93
2.48
10
61.0
12.4
98
11
42.7
17.7
70
2.69
16
36.6
13.2 100
12
35.6
15.4
88
2.17
11
54.5
11.7 100
11
42.3
9.0
88
1.78
16
54.1
14.1
70
12
57.3
15.6
68
2.25
16
62.2
14.6
90
18
56.3
12.0
85
1.82
7
51.4
20.7
90
11
60.2
13.5
85
3.20
164
55.6
15.6
170
52.8
16.3
14
21
19
25
11
8
98
40.2
52.5
60.9
44.9
50.0
44.1
49.5
17.5
83
14
33.6
14.1
83
17.5 100
16
48.4
13.6
23
15.3 100
17
29.0
12.5
23
11.3 100
16
31.6
11.1
23
14.7
93
17
46.0
15.2
95
15.5
93
11
33.4
11.6
95
16.3
91
37.3
14.9
2.40
2.11
1.88
1.44
2.33
2.48
t
-1.35
2.20
1.45
-0.76
0.81
-1.18
1.13
2.71
0.17
2.76
-0.57
1.30
-1.10
1.10
0.77
6.81
3.71
0.68
1.72
df
35
30
18
14
21
28
23
19
26
20
26
32
16
26
35
34
39
26
17
* indicates difference in the means is significant.
a
from the Second Edition or First Edition curricula is not significantly different
( x = 3.26, s x = 8.51, t = 1.38, p = 0.192) .
b
from the Second Edition or non-UCSMP curricula is significantly different
( x = 11.77, s x = 10.53, t = 2.74, p = 0.041) .
For the Second Edition and non-UCSMP sample, the difference in achievement
for students using the two curricula is significant, with the UCSMP students scoring
about 12% higher than the non-UCSMP students. In this sample, the difference in the
mean percent correct by class varies from 4% (pair 25) to 31.9% (pair 23). For two of the
six pairs, both at School Y, the difference in the mean percents is statistically significant,
in favor of the Second Edition classes. However, care must be taken in interpreting these
results because of the low opportunity-to-learn percentage among non-UCSMP classes at
School Y. The non-UCSMP teacher at this school reported that students had an
opportunity to learn the content for less than a fourth of the items.
58
p
0.187
0.036*
0.163
0.459
0.426
0.246
0.270
0.014*
0.864
0.012*
0.571
0.202
0.288
0.282
0.447
0.000*
0.001*
0.501
0.104
As with the previous sample, there is considerable variability in achievement
across classes and schools. The mean percent correct by class varies from 29.0% (nonUCSMP, pair 23) to 60.9% (Second Edition, pair 23), corresponding to mean raw scores
of 11.6 to 24.2 out of 40, respectively. For Second Edition classes, the range of percent
correct is roughly 21%; for non-UCSMP classes, the range of percent correct is roughly
19%.
Achievement on the Fair Tests. Table 19 contains the achievement results for the
Fair Tests constructed from the UCSMP Algebra Test. For the Second Edition and First
Edition sample, the number of items on the Fair Test varies from 19 items at School I to
38 items at School B. For the Second Edition and non-UCSMP sample, the number of
items on the Fair Test varies from 9 items at School Y to 35 items at School Z.
For the Second Edition and First Edition sample, the difference in the mean
percent correct varies from -8.9% (pair 19) to 18.1% (pair 12). There are significant
differences between the means only for pairs 12 and 15, both in favor of the Second
Edition classes. A matched-pairs t-test, however, shows that there is no significant
difference in the achievement of students studying from the Second Edition or First
Edition curricula when OTL is controlled at the school level.
For the Second Edition and non-UCSMP sample, the difference in the mean
percent correct varies from -6.2% (pair 21) to 32.8% (pair 23). Only one of the
differences between the pairs is significant, in favor of the Second Edition class.
Nevertheless, a matched-pairs t-test shows that the achievement results of students
studying from the Second Edition and non-UCSMP curricula are not significantly
different when OTL is controlled at the school level.
Achievement on the Conservative Test. There were 11 items for which all Second
Edition and First Edition teachers reported that their students had an opportunity to learn
the content needed to answer the item. These items deal with writing an algebraic
expression for a contextual situation, writing an expression for the area between two
rectangles, using the distributive property to write an algebraic expression, writing a
linear equation for a contextual problem, finding the length of the leg in a right triangle,
interpreting the meaning of slope in a context, finding the percent of a number in a
context, writing an expression for an exponential context, finding an expression using
d = rt, writing a linear inequality for a contextual situation, and evaluating an equation for
a specific value. Table 20 contains the results for the Conservative Test for each sample.
Overall, a matched-pairs t-test indicates no significant difference in achievement
between students studying from the Second Edition and First Edition curricula on the 11
items that comprise the Conservative Test. The difference in the mean percent correct
between the Second Edition and First Edition classes varies from -9.7% (pair 19) to
11.3% (pair 12). There are no pairs for which the difference in the means is significant.
For the Second Edition and non-UCSMP sample, there were only five items for
which all teachers in both groups reported that their students had an opportunity to learn
the needed content; three of the five items were also on the Conservative Test for the
59
Table 19. Mean Percent Correct on the Fair Tests from the UCSMP Algebra Test
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
X
Y
Y
Y
Z
Z
df
p
2
4
5
6
7
8
9
12
14
15
17
18
19
Comparison
Mean
SD
n
Mean
SD
t
a
18
51.8
14.3
19
57.6
12.3
-1.07
12
55.7
16.3
20
46.3
12.4
1.84
12
50.0
17.0
8
41.0
16.2
1.18
6
55.0
12.4
10
59.2
12.8
-0.64
11
61.4
11.1
12
56.5
11.3
1.05
19
64.0
11.7
11
72.0
13.0
-1.80
10
66.1
11.7
15
59.3
19.4
0.99
10
65.6
10.9
11
47.5
20.7
2.47
16
37.3
13.5
12
37.9
18.2
-0.10
11
56.9
12.1
11
44.7
9.8
2.60
16
61.2
18.0
12
64.9
19.3
-0.52
16
62.9
15.1
18
58.3
12.0
0.99
7
52.1
19.1
11
61.0
14.2
-1.14
35
30
18
14
21
28
23
19
26
20
26
32
16
0.296
0.075
0.253
0.531
0.307
0.083
0.332
0.023*
0.921
0.017*
0.606
0.330
0.273
21
22
23
24
25
26
39.7
18.7
14
34.9
17.8
55.6
16.9
16
61.8
14.0
66.1
15.0
17
33.3
20.0
44.4
12.8
16
47.2
20.5
49.9
16.7
17
46.4
14.8
43.2
16.1
11
33.2
12.5
26
35
34
39
26
17
0.493
0.243
0.000*
0.592
0.566
0.145
n
14
21
19
25
11
8
0.70
-1.19
5.60
-0.54
0.58
1.53
Note: The items comprising the Fair Tests are as follows: School B, 38 items (1-4, 6-13,
15-40); School C, 32 items (1-9, 11, 12, 13, 15-20, 23, 26-32, 34-36, 38-40); School D,
37 items (1-16, 18-30, 32, 34-40); School E, 36 items (1-9, 11-23, 25, 27-36, 38-40);
School G, 27 items (1-5, 7-9, 11, 13, 15, 17-21, 23-25, 27, 29, 30, 31, 33-36); School H,
35 items (1-9, 11-19, 21-25, 27-32, 34-36, 38-40); School I, 19 items (1-3, 6-8, 10, 11,
13, 14, 18, 19, 22, 25, 28-30, 32, 35); School J, 34 items (1-3, 5-7, 9-11, 13, 15-19, 2132, 34-40); School X, 27 items (1-3, 5-11, 14, 16, 18, 19, 22-24, 26, 28-31, 33-35, 38,
39); School Y, 9 items (1, 4, 8, 15, 17, 18, 23, 29, 36); and School Z, 35 items (1-4, 6-11,
13-21, 23, 24, 25, 27-36, 38-40).
* indicates difference between the means is significant.
a
( x = 2.60, s x = 8.46, t = 1.11, p = 0.289) .
b
studying from the Second Edition and non-UCSMP curricula is not significant
( x = 7.02, s x = 13.87, t = 1.24, p = 0.270) .
60
Table 20. Mean Percent Correct on the Two Conservative Tests from the UCSMP
Algebra Test
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
Overall
2
4
5
6
7
8
9
12
14
15
17
18
19
X
Y
Y
Y
Z
Z
Overall
21
22
23
24
25
26
Comparison
n
Mean
SD
n
Mean SD
t
a
18
60.6
14.6
19
69.9
14.5
-1.52
12
53.8
23.4
20
54.5
20.2
-0.09
12
54.5
23.3
8
50.0
21.7
0.43
6
66.7
16.9
10
70.0
15.5
-0.40
11
67.8
19.7
12
69.7
16.2
-0.25
19
79.4
14.5
11
86.0
13.1
-1.24
10
76.4
18.3
15
69.1
21.7
0.87
10
70.0
14.2
11
58.7
25.2
1.25
16
43.8
19.2
12
43.2
23.6
0.07
11
65.3
17.6
11
56.2
21.8
1.08
16
65.9
21.1
12
72.7
17.8
-0.90
16
75.6
16.5
18
67.7
10.9
1.66
7
59.7
22.8
11
69.4
16.4
-1.05
164
64.7
20.7
170
64.5
20.6
14
21
19
25
11
8
98
52.9
16.8
14
42.9
23.3
1.30
56.2
19.6
16
71.3
14.5
-2.59
74.7
18.7
17
40.0
26.5
4.58
52.8
19.0
16
56.3
26.6
-0.49
61.8
28.9
17
67.1
24.4
-0.52
62.5
22.5
11
36.4
29.4
2.10
59.6
21.5
91
53.4
27.2
df
p
35
30
18
14
21
28
23
19
26
20
26
32
16
0.141
0.929
0.669
0.696
0.802
0.224
0.391
0.227
0.941
0.294
0.376
0.106
0.308
26
35
34
39
26
17
0.204
0.014*
0.000*
0.626
0.606
0.051
Note: For the Second Edition and First Edition sample, the Conservative Test consists of
11 items (1, 2, 3, 7, 11, 13, 18, 19, 29, 30, 35). For the Second Edition and non-UCSMP
sample, the Conservative Test consists of 5 items (1, 8, 18, 23, 29).
* indicates difference between the means is significant.
a
( x = 0.18, s x = 7.26, t = 0.09, p = 0.928) .
b
studying from the Second Edition or non-UCSMP curricula is not significant
( x = 7.82, s x = 19.43, t = 0.99, p = 0.370) .
61
Second Edition and First Edition sample. The five items deal with writing an algebraic
expression for a contextual situation, writing an equation for a linear combination
context, finding the percent of a number in a context, finding the percent one number is
of another, and finding an expression using d = rt. Again, there is no significant
difference overall in achievement between students studying from the UCSMP or nonUCSMP curricula on the five items that comprise the Conservative Test. For the Second
Edition and non-UCSMP sample, the difference in the mean percent correct varies from 15.1% (pair 22) to 34.7% (pair 23). For two pairs the difference in the means between the
classes is significant, once in favor of the Second Edition class and once in favor of the
non-UCSMP class.
Item-Level Achievement
Figure 1 contains the stems of the items on the Algebra Test, grouped by content.
Table 21 contains the percent of students in each pair of the Second Edition and First
Edition sample who were able to answer each item in Figure 1 successfully, along with
the overall percentage of the students in each group who were successful.
Overall, in the Second Edition and First Edition sample, the item-level
performance of the two groups is roughly comparable. There are seven items for which
the difference in performance is at least 10%. On six of these (item 22 – finding a linear
equation for a set of data, item 10 – finding solutions to a quadratic equation, item 39 –
solving the square of a binomial equals a number, item 15 – finding a variable expression
for the perimeter of a polygon, item 12 – finding the probability for a coin toss, and item
14 – finding the number in the tenth figure of a pattern), the differences favor the Second
Edition students. On one item (item 6 – interpreting a quadratic graph) the difference
favors the First Edition students.
On four items, all dealing with translating words to symbols (items 1, 7, 8, and
17), at least 80% of both Second Edition and First Edition students were successful on the
item. On one additional item (item 18 – finding the total bill including a tip), at least 80%
of the Second Edition students were successful.
There was one item (item 4 – finding the ratio of one number to another in
context) in which fewer than 20% of either Second Edition or First Edition students were
successful.
In general, students in both the Second Edition and First Edition groups were
reasonably successful at translating from words to symbols. For most linear situations,
the percentage successful was generally better than 80%, except when inequalities were
involved. However, except for a rate situation involving d = rt, situations involving
percentages, ratio comparisons, or rates were harder than linear situations for students to
translate from words to symbols.
Students were also moderately successful with linear relationships with two
variables and with geometric relationships, with the percentage successful generally
better than 50% on the items on each of these subtests. In addition, students were
moderately successful at identifying an exponential pattern for compound interest.
62
Table 22 contains the item-level results for the students in the Second Edition and
non-UCSMP sample. There are 26 items on which the difference in the percent
successful is at least 10%, 24 of these favor the Second Edition students and two favor
the non-UCSMP students. On nine of the items, all in favor of Second Edition students,
the percent difference is greater than 20% (item 3 – translating from words to symbols
when percent is involved, item 7 – translating from words to symbols for a linear
equation, item 31 – translating from words to symbols for a linear combination, item 13 –
interpreting the slope of a line in context, item 2 – finding the area of a rectangle when
another rectangle is removed from the inside, item 24 – finding the angle measures in an
isosceles triangle, item 12 – finding the probability of a coin toss, item 19 – modeling
compound interest, and item 9 – solving a problem using the Multiplication Counting
Principle). Non-UCSMP students did at least 10% better than Second Edition students on
two items (item 11 – finding the length of a leg in a right triangle, and item 18 – finding
the total amount of a bill including a tip).
There were three items on which fewer than 20% of either Second Edition or nonUCSMP students were successful (item 4 – translating from words to symbols in a ratio
comparison, item 39 – solving a square of a binomial equal to a number, and item 38 –
evaluating a quotient of factorials). On one additional item (item 36 – finding percent
increase) fewer than 20% of Second Edition students were successful; on two additional
items (item 33 – solving a quadratic equation in context and item 23 – finding the percent
one number is of another), fewer than 20% of non-UCSMP students were successful.
Second Edition students in the Second Edition and non-UCSMP sample were
moderately successful at translating from words to symbols, dealing with linear
relationships with two variables, and working with geometric relationships. Overall, nonUCSMP students were much less successful on the items on the UCSMP-constructed test
than their Second Edition counterparts; however, these results are heavily influenced by
results from students in the three classes at School Y in which the opportunity-to-learn
for this test is only 23%. At Schools X and Z, results were somewhat more consistent at
the pair level.
63
Figure 1. Stems of UCSMP Algebra Test Items by Content Strand
Item Stem
Item
SPUR
No. Category
Translating Words to Symbols
1
U12
A rope is r meters long. A 10-meter piece is cut from one end. Give
an expression for the remaining length.
3
U1
A computer regularly selling for C dollars is now advertised on sale
at 40% off. What is the sale price of the computer?
4
U
Ace Cinema charges m dollars for admission to a movie. Brown
Cinema charges n dollars for admission to the same movie. How
many times as much as Ace does Brown charge?
7
U1
Bruce started a diet when his weight was 100.4 kg. He is losing 0.7
kg per week. If he now weighs 92 kg, which equation can be solved
to find W, the number of weeks he has been on the diet?
8
U2
Jill bought x exercise books for $3 each and y folders for $2 each. She
spent $d altogether. Which sentence represents this situation?
17
R
Here is a diagram of a balance scale with weights on both sides. … If
the circle represents one-kilogram weights and the weight of each box
is B, which sentence best describes this situation?
29
U12
How many miles will a plane travel if it is flown at k miles per hour
for s hours?
30
U1
A city has a population of 325,000 which is increasing at the rate of
1100 per year. If n is the number of years, which sentence can be
solved to find when the population will be more than 350,000?
31
U
Soda costs a cents for each bottle, including the deposit, but there is a
refund of b cents on each empty bottle. How much will Harry have to
pay for x bottles if he brings back y empties?
34
U
A laser printer prints P pages per minute. How many minutes will it
take to print D documents, each of which has L pages?
Linear Relationships with Two Variables
13
P1
It has been claimed that, in this century, the world record t (in
seconds) for the men’s mile run in the year y can be estimated by
t = 914.2 – 0.346y. According to this claim, how is the record
changing?
64
16
U
The hourly temperatures during a day were recorded as follows.
… [table of times and temperatures]
What was the average rate of change of temperature between 8 a.m.
and 6 p.m.?
22
R
The graph at right shows the winning time in seconds for the girls
50 m freestyle at the first ten annual swimming meets of a school.
The line shows the trend of the data. Which is the best model for
describing these data?
26
U
One electrical company charges $35 for the first hour of labor and
$27 for each additional hour. Another company charges $40 for the
first hour and $26 for each additional hour. The solution to which
system below will tell you the number of hours for which the two
companies will charge the same?
28
R
Water in a pool is 5 inches deep and rising at the rate of an inch every
3 hours. Which of the graphs below represents the relationship
between water level and time?
35
U1
Fahrenheit and Celsius temperatures are related by the formula
9
F = C + 32 . To find the Fahrenheit equivalent to 10° C, what
5
equation is appropriate?
Quadratic Equations and Relationships
10
S
The solutions to 5x2 – 11x – 3 = 0 are …
21
U
The table at the right compares the height from which a ball is
dropped (d) and the height to which it bounces (b). Which formula
describes this relationship?
33
U
In a vacuum chamber, an object on the Earth will fall d meters in t
seconds, where d = 4.9t2. How many seconds would it take an object
to fall 8 m?
37
R
Which of these could be the graph of y = x2 – 4x + 3?
39
S
Solve (z – 1)2 = 361.
Geometric Relationships
2
R1
Find the area of the shaded region between the rectangles.
11
R1
Find the value of k in the figure at right. [k is the leg of a right
triangle in which the other leg and hypotenuse are given.]
65
15
R
Give an expression for the perimeter of this polygon. [Sides are
labeled p or m.]
24
R
In the triangle at right, find b. [Isosceles triangle is given with the
measure of the vertex angle provided.]
27
R
What is the volume of a cube with edges of length 2e?
5
R
A group of boy scouts was asked how long they had been involved in
scouting. This dot frequency diagram shows their responses. …
Which of the following statements is not true?
12
U
In three tosses of a fair coin, heads turned up twice and tails turned
up once. What is the probability that heads will turn up on the fourth
toss?
20
U
After 5 tests, Jerry has an 87 average. What is the least score he can
make on the next test and still have an average of 85?
40
U
Suppose that 50 people respond to the question, “Which is your
favorite season?” as follows: summer, 16; autumn, 4; winter, 9;
spring, 15; no preference, 6. Estimate the probability that a person
selected randomly from this group would choose the spring as their
favorite season.
Percent Applications
18
U12
A restaurant bill is $16.40, and you want to leave a 15% tip. To the
nearest dollar, how much money should you leave altogether?
23
U2
There are an estimated 80,000,000,000,000 insects on the Earth.
1,300,000,000,000 of these are estimated to be in North America.
What percent of the Earth’s insects are in North America?
36
U
In 1970, the population of Gainesville, Florida was 64,510. In 1980,
the population was 81,371. What was the approximate percent
increase between 1970 and 1980?
Graph Interpretation
6
R
Use the graph at the right. It shows the height h of a ball (in feet) t
seconds after it is thrown in the air. For how long was the ball over 20
feet high?
66
32
R
The graph shows the speed of a train between two stations. For how
many minutes between the two stops is the train traveling at its top
speed?
Exponential Relationships
19
U1
If you invest $100 for 8 years at a 7% annual yield, then how many
dollars will you have at the end of this time?
25
S
Miscellaneous
9
U
14
R
38
S
Which point is on the graph of y = 5x?
A menu offers a choice of 3 soups, 5 entrees and 6 desserts. How
many different meals consisting of one soup, one entrée, and one
dessert can be ordered?
Matchsticks are arranged as follows … If the pattern is continued,
how many matchsticks are used in making the 10th figure?
Evaluate
102!
.
100!
1
Item is part of the Conservative Test for the Second Edition and First Edition sample.
2
Item is part of the Conservative Test for the Second Edition and non-UCSMP sample.
67
Table 21. Percent Successful on the UCSMP Algebra Test by Item and Content Strand: Second Edition and First Edition
Item
No.
SPUR
School B
Pair 2
2nd
1st
n=18 n=19
1
3
4
7
8
17
29
30
31
34
U
U
U
U
U
R
U
U
U
U
83
39
39
61
100
100
72
61
61
22
100
53
21
84
84
79
68
68
37
26
13
16
22
26
28
35
P
U
R
U
R
U
50
83
56
72
61
44
58
79
47
74
63
53
10
21
33
37
39
S
U
U
R
S
22
72
28
17
44
42
63
37
42
32
2
11
15
24
27
R
R
R
R
R
50
44
56
72
39
84
63
68
74
68
School C
School D
Pair 4
Pair 5
Pair 6
Pair 7
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=12
n=20
n=12 n=8
n=6 n=10
n=11
n=12
83
85
83
88
100
100
91
100
17
15
17
50
50
10
36
42
42
5
8
25
17
20
18
17
75
55
75
88
83
80
82
100
83
90
75
75
100
90
82
100
92
95
75
75
100
100
82
100
67
40
50
38
50
70
45
17
58
70
92
25
50
80
64
75
50
35
25
38
33
50
45
58
50
30
33
0
50
60
64
25
42
40
25
50
67
80
45
33
83
40
58
25
50
40
82
42
50
40
67
38
50
50
55
33
58
60
67
38
83
80
82
83
58
45
50
25
83
60
73
75
33
25
33
38
33
60
64
92
58
5
58
0
33
60
27
75
75
45
75
50
67
80
82
58
50
35
67
63
33
50
64
58
42
20
42
0
67
40
55
42
50
35
33
38
50
40
27
8
50
70
58
63
100
90
82
92
42
50
42
13
50
50
64
50
67
45
42
25
83
40
64
50
67
50
50
50
83
70
82
75
33
50
58
75
67
50
91
50
68
School E
Pair 8
Pair 9
2nd
1st
2nd
1st
n=19 n=11
n=10 n=15
School G
Pair 12
2nd
1st
n=10 n=11
100
47
21
95
100
89
79
79
74
37
100
64
45
100
100
100
73
100
73
45
80
50
20
80
90
90
50
90
70
40
87
40
13
100
93
93
47
67
47
53
90
40
20
100
90
90
40
100
50
50
100
36
18
82
64
55
36
64
27
27
68
53
74
79
74
63
64
91
82
91
73
91
70
90
100
70
80
90
53
53
67
93
73
60
80
80
50
60
60
50
45
45
45
45
36
27
68
68
63
47
47
9
55
45
18
36
60
70
70
60
50
27
60
60
7
47
20
90
50
10
40
9
64
18
0
9
89
79
79
89
79
100
82
45
100
100
100
60
40
70
90
73
80
53
100
67
60
50
70
80
70
91
55
27
64
45
Table 21 continued.
Item
No.
SPUR
School H
1
3
4
7
8
17
29
30
31
34
P
U
U
U
U
R
U
U
U
U
75
38
8
63
44
88
31
25
31
13
83
33
8
58
58
67
42
25
25
33
13
16
22
26
28
35
P
U
R
U
R
U
44
69
38
44
50
38
33
42
42
17
42
17
10
21
33
37
39
S
U
U
R
S
31
31
31
19
25
17
42
33
0
8
2
11
15
24
27
R
R
R
R
R
56
31
19
63
25
42
33
33
58
25
Pair 14
2nd
1st
n=16 n=12
School I
School J
Pair 15
Pair 17
Pair 18
Pair 19
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=11 n=11
n=16 n=12
n=16 n=18
n=7 n=11
100
91
94
92
94
100
86
100
36
36
44
42
56
44
14
36
0
9
13
25
6
17
29
18
100
55
88
83
94
89
71
82
73
64
94
92
88
89
86
91
91
55
88
92
94
89
71
100
35
55
75
50
75
56
57
82
64
55
63
67
81
78
71
45
55
9
50
67
56
44
43
36
64
27
50
58
50
44
14
45
64
64
56
100
69
50
57
55
64
55
69
67
75
78
43
73
82
45
44
67
88
39
57
45
64
55
69
83
63
89
57
100
55
36
63
75
69
67
29
82
36
18
56
75
75
39
43
55
27
18
44
17
50
22
43
9
64
55
69
50
56
56
57
73
45
9
25
42
50
22
29
36
45
18
25
42
44
44
43
45
36
0
50
17
13
28
43
18
91
82
81
83
81
94
71
100
64
64
50
42
31
39
57
55
45
9
50
42
44
61
71
27
73
91
31
67
81
78
57
82
45
73
63
75
63
72
57
36
Overall
Results
2nd
1st
n=164 n=170
89
38
18
82
85
89
59
68
51
40
94
38
18
81
85
85
52
65
42
37
56
70
62
66
62
52
55
58
49
71
59
49
43
66
46
37
38
24
57
38
26
25
73
51
54
69
59
82
53
44
74
61
Underlined percents indicate items for which teachers indicated that students did not have a chance to learn the content for the item.
69
Table 21 continued.
Item
No.
SPUR
School B
Pair 2
2nd
1st
n=18 n=19
School C
5
12
20
40
R
U
U
U
50
39
33
33
84
42
26
68
75
67
33
50
40
25
15
45
18
23
36
U
U
U
89
33
28
74
42
21
67
33
42
85
15
25
6
32
R
R
44
56
68
42
42
33
30
40
19
25
U
S
72
11
63
37
58
17
65
30
9
14
38
U
R
S
50
78
28
100
58
37
100
67
50
65
45
50
Pair 4
2nd
1st
n=12 n=20
School D
Pair 5
Pair 6
Pair 7
2nd
1st
2nd
1st
2nd
1st
n=12 n=8
n=6 n=10
n=11 n=12
83
63
33
90
91
75
58
13
50
50
55
33
25
0
0
40
55
17
67
50
33
60
64
75
83
50
50
90
91
83
33
25
0
0
45
25
25
13
33
30
18
33
58
50
17
90
55
83
50
50
33
50
55
33
42
50
100
60
82
83
8
25
67
30
36
42
Miscellaneous
67
38
50
100
82
75
58
50
67
80
64
50
8
25
33
20
27
50
School E
Pair 8
2nd
1st
n=19 n=11
84
53
21
58
100
64
18
82
90
60
10
60
67
40
53
80
100
60
50
80
45
45
36
27
95
21
26
100
55
36
100
40
20
73
40
47
100
50
30
45
45
0
42
47
91
73
60
40
60
33
50
60
45
36
79
53
73
55
70
100
80
27
60
30
64
18
79
74
16
91
45
45
70
80
10
73
47
27
80
70
30
82
73
9
70
Pair 9
2nd
1st
n=10 n=15
School G
Pair 12
2nd
1st
n=10
n=11
Table 21 continued.
Item
No.
SPUR
5
12
20
40
R
U
U
U
18
23
36
U
U
U
6
32
R
R
19
25
U
S
9
14
38
U
R
S
School H
School I
School J
Pair 14
Pair 15
Pair 17
Pair 18
Pair 19
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=16 n=12
n=11 n=11
n=16 n=12
n=16 n=18
n=7 n=11
50
75
73
73
75
75
63
61
57
91
25
17
45
45
50
42
75
28
43
73
31
33
9
27
25
8
50
39
43
27
25
33
73
45
56
67
75
50
43
82
69
67
73
64
69
83
88
72
57
82
13
17
9
18
19
8
25
28
57
27
19
17
36
45
38
25
25
33
14
64
38
42
55
55
63
67
69
78
57
100
13
33
64
18
63
50
56
50
29
36
13
42
55
36
50
83
88
83
71
73
19
8
36
0
19
17
25
28
43
64
Miscellaneous
63
58
91
55
69
92
81
89
71
64
56
50
18
36
50
50
94
72
71
91
6
17
27
27
19
17
31
11
43
9
Overall
Results
2nd
1st
n=164 n=170
71
52
30
55
71
39
27
59
81
28
27
75
27
30
51
47
65
42
63
32
67
29
73
66
24
77
57
28
71
Table 22. Percent Successful on the UCSMP Algebra Test by Item and Content Strand: Second Edition and non-UCSMP
Item
No.
SPUR
School X
Pair 21
2nd
non
n=14 n=14
1
3
4
7
8
17
29
30
31
34
U
U
U
U
U
R
U
U
U
U
79
21
14
79
57
79
50
36
29
21
71
14
0
43
57
43
50
64
7
43
95
48
33
71
76
90
52
67
62
62
100
44
25
75
81
88
75
75
50
31
13
16
22
26
28
35
P
U
R
U
R
U
43
43
36
43
43
21
29
29
36
71
43
36
48
67
52
67
57
67
44
75
44
56
63
44
10
21
33
37
39
S
U
U
R
S
7
79
36
43
7
29
36
29
50
7
38
38
29
19
14
13
50
6
0
19
2
11
15
24
27
R
R
R
R
R
43
14
50
43
29
29
29
50
43
29
81
43
76
71
67
69
63
56
63
75
Pair 22
2nd
non
n=21 n=16
School Y
Pair 23
Pair 24
2nd
non
2nd
non
n=19 n=17
n=25 n=16
79
65
84
88
58
35
40
13
42
18
0
25
84
47
64
38
89
65
80
44
95
41
80
75
84
12
48
56
95
41
52
31
58
18
40
31
63
35
32
38
63
29
48
6
68
47
76
44
63
18
44
31
84
29
60
44
79
47
44
19
47
12
40
31
32
6
16
25
68
29
40
38
53
29
36
6
42
12
28
6
37
6
12
13
79
29
68
13
58
47
32
50
68
18
48
31
79
24
72
25
47
35
56
25
72
School Z
Pair 25
Pair 26
2nd
non
2nd non
n=11 n=17
n=8 n=11
Overall
Results
2nd
non
n=98 n= 91
91
55
0
91
55
82
45
64
45
36
94
18
0
53
71
82
53
65
29
35
88
50
13
63
88
88
75
63
50
13
73
9
36
45
27
82
18
27
18
27
86
45
18
74
76
86
58
63
48
42
82
23
16
51
59
68
45
52
26
35
55
45
27
73
82
36
41
53
59
53
47
18
63
63
38
50
63
25
36
36
36
27
36
18
52
63
46
64
59
43
31
48
37
47
43
26
36
45
9
27
18
53
65
18
18
18
13
50
13
13
38
9
36
9
9
27
24
52
33
30
19
23
43
16
15
14
82
64
36
82
55
59
65
29
59
65
63
25
25
50
63
45
64
36
73
36
70
40
55
68
53
41
53
36
46
45
Table 22 continued.
Item
No.
SPUR
School X
Pair 21
2nd
non
n=14 n=14
5
12
20
40
R
U
U
U
50
29
14
21
57
29
43
36
67
62
19
33
50
56
38
19
18
23
36
U
U
U
43
36
36
29
7
21
52
5
19
81
19
31
6
32
R
R
29
50
50
21
62
43
63
56
19
25
U
S
50
50
0
14
57
43
0
38
9
14
38
U
R
S
71
43
43
29
43
0
76
57
14
50
56
0
Pair 22
2nd
non
n=21 n=16
School Y
Pair 23
Pair 24
2nd
non
2nd
non
n=19 n=17
n=25 n=16
58
29
52
31
68
24
64
31
42
18
32
31
58
29
40
38
74
53
40
63
47
6
12
31
16
24
8
13
42
53
56
44
37
29
40
13
84
6
72
0
32
24
20
31
Miscellaneous
89
29
68
19
68
41
48
63
5
0
4
13
School Z
Pair 25
Pair 26
2nd
non
2nd non
n=11 n=17
n=8 n=11
64
64
27
45
71
18
29
35
88
63
38
63
45
55
55
27
60
59
29
42
47
34
34
31
91
27
27
94
24
24
13
50
13
36
27
27
53
26
18
62
19
23
45
36
59
35
38
0
45
0
48
38
53
27
36
18
65
29
50
13
27
9
62
31
16
25
82
73
27
82
47
12
75
25
0
45
36
0
77
54
14
43
48
4
73
Overall
Results
2nd
non
n= 98 n= 91
Achievement on the Problem-Solving and Understanding Test
Achievement on the Problem-Solving and Understanding (PSU) Test is reported
with two types of analyses: overall achievement by matched pairs; and item level results.
Overall Achievement
Table 23 contains the mean total scores on the odd form of the PSU Test by
matched pairs; Table 24 contains the related scores on the even form of the PSU Test. For
the Second Edition and First Edition sample, there is a significant difference in the means
between the two classes on the odd form for only pair 8, in favor of the First Edition
class; on the even form there is a significant difference between the class means for only
pair 6, in favor of the Second Edition class. Furthermore, a matched pairs t-test on the
mean of the differences indicates that there is no significant difference in achievement
between students studying from the Second Edition or First Edition curricula on either
form of the test.
In general, achievement on the odd form of the PSU Test is not particularly high,
with achievement by the Second Edition and First Edition students at around 31% and
34%, respectively. Achievement is somewhat better on the even form, with overall
achievement for Second Edition and First Edition students at 51% and 48%, respectively.
For the Second Edition and non-UCSMP sample, there are two significant
differences between the class means on the odd form of the PSU Test, both in favor of the
Second Edition classes. However, in both of these classes, the Second Edition teacher
reported covering the content needed for all of the items while the non-UCSMP teacher
did not. For the even form of the PSU test, there are significant differences between the
class means for two pairs, both in favor of the Second Edition classes. Again, the OTL
likely is related to achievement as the Second Edition teacher reported covering the
content for 100% of the items and the non-UCSMP teacher reported covering the content
for only 25% of the items.
Overall, for the Second Edition and non-UCSMP sample, there is a significant
difference in achievement between students studying from the Second Edition or nonUCSMP curricula on both forms of the Problem-Solving and Understanding Test. For the
Second Edition students, achievement was roughly 37% on the odd form and 46% on the
even form; for the non-UCSMP students, achievement on the odd and even forms was
19% and 26%, respectively. However, in both cases, there are differences in the
opportunities to learn the needed content and these are likely related to the achievement
differences.
74
Table 23. Mean Score on the Odd Form of the Problem-Solving and Understanding Test
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
Overall
2
4
5
6
7
8
9
12
14
15
17
18
19
X
Y
Y
Y
Z
Z
Overall
21
22
23
24
25
26
Comparison
n
Mean SD OTL
n
Mean SD OTL
t
a
10
6.2
4.3
100
9
4.3
2.7
100
1.14
4
2.8
1.7
75
12
3.9
2.0
75
-0.98
8
3.9
2.3
75
6
3.8
4.2
75
0.06
2
3.5
0.7
100
5
5.6
3.8
NA
-0.74
5
6.2
2.9
100
7
4.3
2.7
NA
1.17
10
4.3
2.2
75
5
9.0
3.4
100
-3.27
5
5.8
3.8
75
10
7.4
4.3
100
-0.70
6
3.3
2.9
75
5
2.6
1.5
75
0.49
9
3.7
2.5
100
7
1.7
2.2
75
1.67
5
3.6
3.0
100
5
3.0
2.1
75
0.37
7
4.3
3.2
75
6
5.0
3.5
50
-0.38
9
5.1
3.1
100
9
6.2
3.8
75
-0.67
3
2.3
2.3
100
5
5.4
4.8
75
-1.03
-0.79
83
4.4
3.0
91
4.8
3.6
7
12
9
13
6
5
3.9
6.3
5.9
4.5
4.8
5.0
52
5.2
4.4
100
6
2.3
1.4
50
3.6
100
7
3.3
2.4
50
2.9
100
12
3.3
3.0
50
2.6
100
9
2.0
1.0
50
2.4
100
9
3.3
2.3
75
4.0
100
6
1.0
1.1
75
3.3
49
2.7
2.2
df
p
17
14
12
5
10
13
13
9
14
8
11
16
6
172
0.271
0.342
0.955
0.495
0.270
0.006*
0.494
0.639
0.117
0.724
0.714
0.511
0.345
0.429
0.85
1.95
1.99
2.73
1.22
2.37
11
17
19
20
13
9
0.414
0.067
0.061
0.013*
0.245
0.042*
4.45
99
2.239×10-5
Note: The First Edition teacher at School D returned the opportunity-to-learn form but
provided no responses for the items. The maximum score on the odd form is 14.
a
studying from the Second Edition or First Edition curricula is not significantly different
( x = − 0.55, s x = 2.03, t = − 0.98, p = 0.346 ).
b
studying from the Second Edition or non-UCSMP curricula is significantly different
( x = 2.53, s x = 0.93, t = 6.68, p = 0.001 ).
75
Table 24. Mean Score on the Even Form of the Problem-Solving and Understanding Test
School
Code
Pair
ID
B
C
C
D
D
E
E
G
H
H
I
J
J
Overall
2
4
5
6
7
8
9
12
14
15
17
18
19
X
Y
Y
Y
Z
Z
Overall
21
22
23
24
25
26
Comparison
n
Mean SD OTL
n
Mean SD OTL
8
8.4
4.0
100
9
10.0
3.4
100
8
6.4
3.5
100
8
4.4
3.1
75
4
8.3
3.6
100
2
2.5
2.1
75
4
12.0
2.7
100
5
7.8
2.5
NA
6
8.5
3.8
100
5
7.4
2.8
NA
9
11.0
2.5
100
6
11.0
2.8
100
6
12.0
4.1
100
5
6.8
4.5
100
4
5.5
1.3
100
6
6.5
4.0
75
7
4.3
2.8
100
5
3.6
3.4
75
6
5.8
1.9
100
6
4.3
2.6
75
9
7.7
3.1
100
6
9.5
1.9
50
5
8.8
5.8
100
9
9.7
4.4
75
4
8.0
6.1
100
6
9.8
3.6
75
80
8.1
4.0
78
7.6
4.0
7
9
10
12
5
3
46
4.3
1.7
100
8
4.0
2.5
50
9.8
4.1
100
9
7.1
2.9
25
9.2
3.9
100
5
3.0
1.0
25
7.5
3.2
100
7
3.0
2.3
25
5.2
2.9
100
8
3.3
2.2
75
5.3
3.5
100
5
3.4
1.1
75
7.4
3.8
42
4.2
2.7
t
df
p
-0.89
1.21
2.04
2.42
0.54
0.00
2.01
-0.48
0.39
1.14
-1.26
-0.33
-0.59
15
14
4
7
9
13
9
8
10
10
13
12
8
0.387
0.246
0.111
0.046*
0.605
1.000
0.076
0.647
0.704
0.280
0.228
0.748
0.569
0.24
1.61
3.44
3.25
1.34
1.18
9
16
13
17
11
6
0.816
0.126
0.004*
0.005*
0.206
0.284
Note: The First Edition teacher at School D returned the opportunity-to-learn form but
provided no responses for the items. The maximum score on the even form is 16.
a
studying from the Second Edition or First Edition curricula is not significantly different
( x = 1.03, s x = 2.63, t = 1.41, p = 0.184 ).
b
studying from the Second Edition or non-UCSMP curricula is significantly different
( x = 2.92, s x = 2.11, t = 3.39, p = 0.020 ).
76
Item-Level Achievement on the Odd Form
Tables 25 and 26 contain the item-level analyses for the odd form of the PSU Test
for the Second Edition and First Edition sample and the Second Edition and non-UCSMP
sample, respectively. A copy of the test can be found in Appendix C; rubrics for the items
can be found in Appendix D.
Because of the small sample sizes at the class level due to the use of two forms of
the Problem Solving and Understanding Test, no reliable conclusions can be drawn from
performance on the PSU. On none of the items did the overall achievement of any of the
four groups of students reach 50% of the possible points.
Item-Level Achievement on the Even Form
Tables 27 and 28 report the means for the items on the even form of the ProblemSolving and Understanding Test for the Second Edition and First Edition sample and the
Second Edition and non-UCSMP sample, respectively. A copy of the even form of the
test can be found in Appendix C; the rubrics for the items are found in Appendix D.
As with the odd form, the small sample sizes at the class level make it difficult to
draw any reliable conclusions about the achievement on the even form. However, on
items 1 and 2 dealing with writing a real situation that can be solved by an equation of the
form ax + b = cx + d and reasoning related to the distributive property, respectively, all
UCSMP students, whether studying from the Second Edition or First Edition curriculum,
earned at least 50% of the possible points overall.
77
Table 25. Item Means (Standard Deviations) for the Odd Form of the Problem-Solving and Understanding Test: Second Edition and
First Edition
Item
No.
School Da
School B
Pair 2
2nd
1st
n=10 n=9
1.5
1.4
(1.6) (1.5)
Pair 4
2nd
1st
n=4 n=12
0.3
1.6
(0.5) (0.9)
Pair 5
2nd
1st
n=8 n=6
0.3
0.7
(0.5) (1.6)
Pair 6
2nd
1st
n=2
n=5
0.0
2.0
(0.0) (1.2)
Pair 7
2nd
1st
n=5
n=7
1.0
1.3
(1.2) (1.3)
Pair 8
2nd
1st
n=10 n=5
1.1
3.0
(1.8) (1.4)
Pair 9
2nd
1st
n=5 n=10
1.0
2.3
(1.4) (1.5)
School G
Pair 12
2nd
1st
n=6 n=5
0.5
0.4
(1.2) (0.5)
2
1.1
(0.6)
0.9
(0.8)
1.0
(0.8)
0.3
(0.5)
1.0
(0.5)
0.3
(0.5)
0.0
(0.0)
1.2
(0.4)
1.0
(0.7)
0.7
(0.5)
1.3
(0.7)
1.0
(0.7)
1.0
(0.7)
1.1
(0.9)
0.5
(0.5)
0.6
(0.9)
3
2.5
(1.8)
1.3
(1.8)
1.5
(1.7)
0.8
(1.4)
2.6
(1.7)
1.7
(1.5)
0
(0)
1.2
(1.6)
2.8
(1.6)
1.3
(1.9)
0.4
(1.3)
1.4
(1.9)
2.4
(2.2)
1.8
(1.9)
2.2
(2.0)
1.4
(1.7)
4a
0.5
(0.7)
0.2
(0.4)
0
(0)
0.9
(0.9)
0
(0)
0.8
(1.0)
1.5
(0.7)
0.8
(0.8)
0.8
(0.8)
0.4
(0.5)
0.4
(0.7)
1.6
(0.5)
0.6
(0.5)
1.0
(0.8)
0.2
(0.4)
0.2
(0.4)
4b
0.6
(1.0)
0.4
(0.9)
0
(0)
0.4
(0.8)
0
(0)
0.3
(0.8)
2.0
(0)
0.4
(0.9)
0.6
(0.9)
0.6
(1.0)
1.1
(0.9)
2.0
(0)
0.8
(1.1)
1.2
(1.0)
0
(0)
0
(0)
1
School C
School E
Note: Underlined percents indicate the teacher reported that students did not have an opportunity to learn the content needed for the
item. The maximum score for items 1 and 3 is 4 and for items 2, 4a, and 4b is 2.
a
The First Edition teacher at School D returned the opportunity-to-learn form but with no responses marked to the items.
78
Table 25 continued
Item
No.
School H
Pair 14
Pair 15
2nd
1st
2nd
1st
n =9 n=7
n=5
n=5
0.3
0.1
0.6
0.2
(0.7) (0.4)
(0.9) (0.4)
School I
Pair 17
2nd
1st
n=7 n=6
1.3
1.2
(1.7) (1.6)
School J
Pair 18
Pair 19
2nd
1st
2nd
1st
n=9
n=9
n=3
n=5
1.6
2.0
0.0
2.2
(1.6) (1.9)
(0.) (1.6)
Overall
Results
2nd
1st
n=83 n=91
0.9
1.5
(1.3)
(1.5)
2
0.6
(0.5)
0.1
(0.4)
1.0
(0.7)
0.4
(0.5)
0.6
(0.8)
0.8
(0.8)
0.9
(0.6)
0.8
(0.8)
0.7
(0.6)
0.4
(0.5)
0.9
(0.7)
0.7
(0.7)
3
2.0
(1.6)
1.1
(2.0)
1.4
(1.9)
2.2
(1.8)
2.4
(2.0)
2.3
(1.9)
2.1
(1.9)
1.7
(2.0)
1.7
(2.1)
1.6
(1.7)
1.9
(1.8)
1.5
(1.7)
4a
0.1
(0.3)
0
(0)
0.2
(0.4)
0.2
(0.4)
0
(0)
0.5
(0.5)
0.2
(0.4)
0.7
(0.9)
0
(0)
0.4
(0.5)
0.3
(0.6)
0.6
(0.8)
4b
0.7
(1.0)
0.3
(0.8)
0.4
(0.9)
0
(0)
0
(0)
0.2
(0.4)
0.3
(0.7)
1.1
(1.1)
0
(0)
0.8
(1.1)
0.5
(0.8)
0.6
(0.9)
1
a
79
Table 26. Item Means (Standard Deviations) for the Odd Form of the Problem-Solving and Understanding Test: Second Edition and
non-UCSMP
Item
No.
School X
Pair 21
2nd
non
n=7 n=6
1.4
0.5
(1.8) (0.5)
Pair 22
2nd
non
n=12 n=7
1.9
1.7
(1.4) (1.7)
School Y
Pair 23
2nd
non
n=9 n=12
1.9
1.3
(1.5) (1.7)
Pair 24
2nd
non
n=13 n=9
0.9
1.2
(1.0) (0.8)
School Z
Pair 25
Pair 26
2nd
non
2nd
non
n=6
n=9
n=5
n=6
0.7
0.3
1.8
0.3
(0.5) (1.0)
(1.6) (0.5)
Overall
Results
2nd
non
n=52 n=49
1.4
1.0
(1.4)
(1.3)
2
0.1
(0.4)
0.7
(0.8)
1.3
(0.7)
0.7
(0.8)
1.1
(0.8)
0.4
(0.5)
1.1
(0.9)
1.0
(1.0)
1.0
(0.9)
0.9
(0.8)
0.8
(0.8)
0.5
(0.5)
1.0
(0.8)
0.6
(0.7)
3
1.0
(1.5)
0.2
(0.4)
1.6
(1.7)
0.6
(1.5)
0.7
(1.4)
1.0
(1.3)
0.9
(1.4)
0.0
(0.0)
2.2
(1.8)
2.0
(1.6)
1.6
(1.8)
0.2
(0.4)
1.3
(1.6)
0.7
(1.3)
4a
0.7
(1.0)
0.8
(0.8)
0.6
(0.8)
0.3
(0.5)
0.6
(0.7)
0.3
(0.5)
0.5
(0.7)
0.4
(1.0)
0.5
(0.5)
0.1
(0.3)
0.2
(0.4)
0.0
(0.0)
0.5
(0.7)
0.3
(0.6)
4b
0.6
(1.0)
0.2
(0.4)
0.9
(1.0)
0.0
(0.0)
1.7
(0.7)
0.2
(0.6)
1.2
(1.0)
0.0
(0.0)
0.5
(0.8)
0.0
(0.0)
0.6
(0.9)
0.0
(0.0)
1.0
(1.0)
0.1
0.3
1
80
Table 27. Item Means (Standard Deviations) for the Even Form of the Problem-Solving and Understanding Test: Second Edition and
First Edition
Item
No.
School Da
School B
Pair 2
2nd
1st
n=8 n=9
2.3
3.2
(1.7) (1.4)
Pair 4
2nd
1st
n=8
n=8
0.6
1.3
(0.7) (1.8)
Pair 5
2nd
1st
n=4
n=2
1.3
0.5
(1.0) (0.7)
Pair 6
2nd
1st
n=4
n=5
3.8
1.8
(0.5) (1.1)
Pair 7
2nd
1st
n=6
n=5
3.3
2.4
(1.0) (1.8)
School E
Pair 8
Pair 9
2nd
1st
2nd
1st
n=9
n=6
n=6
n=5
2.8
3.7
3.2
2.0
(1.6) (0.8)
(1.3) (1.9)
School G
Pair 12
2nd
1st
n=4 n=6
1.0
1.8
(1.4) (1.6)
2
2.4
(1.1)
2.7
(0.7)
2.8
(1.0)
1.9
(0.6)
3.3
(1.0)
1.5
(0.7)
3.0
(1.2)
2.2
(0.4)
2.2
(1.0)
2.6
(0.5)
3.6
(0.7)
2.8
(1.0)
3.0
(0.6)
2.6
(0.9)
2.8
(1.3)
2.2
(1.0)
3
3.1
(1.2)
2.4
(1.4)
1.5
(1.6)
0.9
(1.4)
1.8
(1.0)
0
(0)
2.8
(1.9)
2.0
(2.0)
1.5
(1.4)
1.2
(1.8)
2.2
(0.8)
3.3
(1.2)
3.2
(1.6)
1.4
(1.5)
1.5
(0.6)
2.2
(1.8)
4a
0.3
(0.7)
1.4
(0.9)
1.4
(0.9)
0.1
(0.4)
1.5
(1.0)
0.5
(0.7)
1.5
(1.0)
1.4
(0.9)
1.0
(1.1)
1.0
(1.0)
1.3
(1.0)
0.7
(1.0)
1.8
(0.4)
0.4
(0.5)
0.3
(0.5)
0.2
(0.4)
4b
0.4
(0.5)
0.7
(0.7)
0.1
(0.4)
0.3
(0.5)
0.5
(1.0)
0
(0)
1.0
(1.2)
0.4
(0.5)
0.5
(0.8)
0.2
(0.4)
0.9
(1.1)
0.5
(0.5)
0.7
(0.8)
0.4
(0.5)
0
(0)
0.2
(0.4)
1
School C
item. The maximum score for items 1, 2, and 3 is 4 and for items 4a and 4b is 2.
a
81
Table 27 continued
Item
No.
Pair 14
2nd
1st
n=7
n=5
1.3
0.6
(1.4) (0.5)
Pair 15
2nd
1st
n=6
n=6
2.0
0.8
(1.4) (1.0)
School I
Pair 17
2nd
1st
n=9 n=6
2.1
3.3
(1.6) (1.2)
2
2.0
(0.6)
1.8
(1.3)
2.0
(1.1)
2.3
(1.4)
2.7
(0.9)
2.8
(1.0)
2.4
(1.5)
3.2
(1.1)
2.8
(1.5)
2.2
(0.4)
2.7
(1.0)
2.4
(0.9)
3
0.6
(0.8)
1.2
(1.8)
1.2
(1.3)
0.8
(1.0)
2.0
(1.3)
3.3
(0.8)
2.8
(1.3)
2.2
(1.5)
1.8
(1.0)
2.2
(1.7)
2.0
(1.4)
1.9
(1.6)
4a
0.3
(0.5)
0
(0)
0.7
(1.0)
0.2
(0.4)
0.7
(0.9)
0
(0)
1.2
(1.1)
1.0
(0.9)
1.0
(1.2)
1.5
(0.8)
1.0
(1.0)
0.7
(0.9)
4b
0.1
(0.4)
0
(0)
0
(0)
0.2
(0.4)
0.2
(0.4)
0
(0)
1.0
(1.0)
0.8
(0.8)
1.0
(1.2)
0.7
(1.0)
0.5
(0.8)
0.4
(0.6)
1
School H
School J
Pair 18
2nd
1st
n=5
n=9
1.4
2.4
(1.7) (1.7)
Pair 19
2nd
1st
n=4
n=6
1.5
3.3
(1.7) (1.2)
Overall
Results
2nd
1st
n=80 n=78
2.1
2.2
(1.6)
(1.6)
82
Table 28. Item Means (Standard Deviations) for the Even Form of the Problem-Solving and Understanding Test: Second Edition and
non-UCSMP
Item No.
School X
Pair 21
2nd
non
n=7 n=8
0.6
1.3
(0.5) (1.4)
Pair 22
2nd
non
n=9 n=9
2.7
2.2
(1.8) (1.4)
School Y
Pair 23
2nd
non
n=10 n=5
2.9
0.4
(1.5) (0.5)
Pair 24
2nd
non
n=12 n=7
2.0
0.9
(2.0) (1.1)
School Z
Pair 25
Pair 26
2nd
non
2nd
non
n=5
n=8
n=3
n=5
1.4
0.3
2.0
0.6
(1.7) (0.5)
(1.0) (0.5)
Overall
Results
2nd
non
n=46 n=42
2.0
1.0
(1.7)
(1.2)
2
1.9
(0.9)
1.8
(0.9)
3.1
(0.8)
2.3
(0.5)
2.4
(1.1)
2.0
(0.0)
2.4
(0.7)
1.3
(1.1)
2.2
(0.5)
2.0
(1.2)
1.7
(0.6)
1.8
(0.4)
2.5
(0.9)
1.9
(0.9)
3
0.7
(0.8)
0.8
(1.0)
1.8
(1.3)
2.2
(1.9)
2.6
(1.7)
0.4
(0.9)
2.3
(1.4)
0.7
(1.1)
1.0
(0.7)
1.0
(1.4)
1.0
(1.0)
1.0
(1.2)
1.8
(1.4)
1.1
(1.4)
4a
0.3
(0.8)
0.3
(0.7)
1.1
(0.9)
0.0
(0.0)
0.6
(0.8)
0.0
(0.0)
0.5
(0.9)
0.0
(0.0)
0.2
(0.5)
0.0
(0.0)
0.7
(1.2)
0.0
(0.0)
0.6
(0.9)
0.1
(0.3)
4b
0.4
(0.8)
0.0
(0.0)
1.1
(0.9)
0.3
(0.5)
0.7
(0.8)
0.2
(0.5)
0.3
(0.6)
0.1
(0.4)
0.4
(0.6)
0.0
(0.0)
0.0
(0.0)
0.0
(0.0)
0.5
(0.8)
0.1
(0.3)
1
83
Summary
This chapter has described the achievement of students in two samples: those
using the Second Edition or First Edition of UCSMP Algebra; or those using the Second
Edition of UCSMP Algebra or the non-UCSMP curricula in place in the comparison
classes. The research described here answers the question, How does the achievement of
students in classes using UCSMP Algebra (Second Edition, Field-Trial Version) compare
to that of students using UCSMP Algebra (First Edition) or to students using nonUCSMP materials?
The results on the High School Subject Tests: Algebra, a standardized measure,
indicate that differences in achievement between the Second Edition and First Edition
students were not significant. This was true whether the data were analyzed on the basis
of the entire test, on the basis of a Fair Test using only items for which both teachers at
the school indicated that students had an opportunity to learn the necessary content, or on
the basis of a Conservative Test consisting of only those items for which all teachers in
the sample indicated that students had an opportunity to learn the necessary content.
Overall, achievement of both Second Edition and First Edition students corresponded to
the 45th percentile.
On both the overall test and the Fair Test, significant differences between the
means of Second Edition and First Edition classes existed only for the pair at School G,
in favor of the Second Edition class.
On the UCSMP Algebra Test, a UCSMP-constructed test, there were no overall
significant differences in achievement among Second Edition and First Edition students,
regardless of how the analysis was completed. For the overall test analysis and the Fair
Tests, significant differences in the class means existed for two pairs, the pair at School G
and one of the pairs at School H, both in favor of the Second Edition classes.
Overall achievement on the Problem-Solving and Understanding Test was
between 31% and 34% for the Second Edition and First Edition students, respectively, on
the odd form and at 51% and 48% for the even form, respectively. The small class sizes
because of the use of two forms of the test in each class make it difficult to draw any
reliable conclusions about students’ achievement on the PSU Test. Nevertheless, the
results suggest that students at this level need considerable experience with extended
problems in which they need to explain their thinking.
Hence, for the Second Edition and First Edition sample, there are no overall
significant differences in achievement for students using the Second Edition or First
Edition of UCSMP Algebra. This result is not entirely unexpected given that there were
no major differences in content between the two editions of the text.
For the Second Edition and non-UCSMP sample, there were no overall significant
differences in achievement on the standardized measure, regardless of how the data were
analyzed. In this sample, the overall achievement of the Second Edition students
corresponded to the 48th percentile and to the 45th percentile for the non-UCSMP
students. On the entire test, there was one significant difference in achievement at the pair
level, in favor of the Second Edition class; there were no significant pair differences on
84
any of the Fair Tests. On the Conservative Test consisting of 13 items, there were two
significant differences at the pair level, both in favor of the non-UCSMP classes.
On the UCSMP-constructed Algebra Test, there was a significant difference in
achievement overall for the entire test, in favor of Second Edition students. However,
these results must be viewed with care because of differences in the opportunity-to-learn
measures. Among students in the three non-UCSMP classes at School Y, students had an
opportunity to learn the content needed to answer only 23% of the items on the test.
There were no overall differences in achievement between students using the two
curricula on the Fair Tests or the Conservative Test; the Conservative Test consisted of
only 5 items.
There were differences in achievement, in favor of Second Edition students, on
both forms of the Problem-Solving and Understanding Test. However, once again, the
non-UCSMP students had limited opportunities to learn the content needed to answer
these items so results must be interpreted with caution. In addition, the small class sizes
make it difficult to reach reliable conclusions.
On the standardized measure, the performance of students in the Second Edition
and non-UCSMP sample was slightly better than the performance of students in the
Second Edition and First Edition sample. However, on the UCSMP-constructed Algebra
Test, students in the Second Edition and First Edition sample performed somewhat better
than students in the Second Edition and non-UCSMP sample.
85
86
CHAPTER 5
ATTITUDES
The data reported in this chapter attempt to answer the following research
question: How do attitudes of students and teachers using UCSMP Algebra (Second
Edition) compare to those of students and teachers using UCSMP Algebra (First Edition)
or non-UCSMP materials? The first section of this chapter discusses students’ attitudes;
the second highlights teachers’ opinions about the text.
The results discussed in this chapter come from the Fall and Spring Student
Opinion Surveys and the Teacher Interview.
Students’ Attitudes
Both Student Opinion Surveys were designed with blocks of related items. Thus,
the results are reported in these blocks: attitudes toward mathematics as a discipline;
confidence in mathematics; calculator use; attitudes toward the current course; and
attitudes about the textbook and its features. For the 15 attitudinal items used in the fall
and the 17 used in the spring, students responded on a four-point Likert scale using
strongly agree, agree, disagree, strongly disagree. In this chapter, the percentages of
students who strongly agree and agree are grouped together as are the percentages who
strongly disagree and disagree.
Six items on the fall and spring survey were the same: four dealing with attitudes
toward mathematics as a discipline and two dealing with attitudes toward calculators. For
these items, both sets of data are reported to allow for comparisons within samples from
fall to spring as well as for comparisons between samples in the spring. Those items
dealing with attitudes toward the current course or toward the textbook and its features
were only administered in the spring and so only comparisons between the groups at the
end of the course are of interest.
The relatively small class sizes, together with the multiple comparisons, make it
impractical to conduct statistical tests at the pair level. So, χ2 tests were computed on the
overall results for each sample; for only three items, all in the Second Edition and nonUCSMP sample, was the p value less than 0.05. However, the multiple comparisons
require lowering the p value to 0.003 (i.e., 0.05 ÷ 16), making the standard for
significance high. Therefore, rather than considering levels of significance, trends in the
data are discussed and special note is made when the overall difference in the group
percentages is at least 15%, when the overall difference from fall to spring (where
appropriate) is at least 15%, when the difference in the pair percentages is at least 25%,
or when there is some result that seems anomalous when compared with other results.
Attitudes Toward Mathematics as a Discipline
Tables 29 and 30 contain the results for the Second Edition and First Edition
87
sample and the Second Edition and non-UCSMP sample, respectively, on the four items
dealing with students’ opinions about the nature of mathematics as a discipline.
Overall, results indicate that the students in the Second Edition and First Edition
sample have roughly comparable positive attitudes toward mathematics. Somewhat more
than 60% of the Second Edition and First Edition students agreed with the statement,
Mathematics is an interesting subject, at both administrations of the survey. However,
attitudes were not uniformly positive at the pair level. In the fall, the percentage
agreement with the statement differed by at least 25% for pairs 2, 5, and 6, with Second
Edition students more likely to agree than their First Edition peers in all three pairs. In the
spring, this level of difference occurred only for pairs 5 and 15, again with the Second
Edition students more likely to agree. A drop of at least 25% in the percentage of students
from fall to spring agreeing that mathematics is interesting occurred for Second Edition
students in pairs 5, 6, 7, and 8 and for First Edition students in pairs 7 and 12. At School I
within pair 17, the percentage agreement increased by at least 25% from fall to spring.
Almost all students in both groups disagreed with the statement, Mathematics is
more for boys than for girls. Only for the Second Edition students in pair 5 in the spring
and the First Edition students in pair 5 in the fall did at least 25% of the students agree
with the statement. For students in pair 5, the percentage agreement with the statement
increased by at least 25% from fall to spring for the Second Edition students and dropped
by this percentage for the First Edition students.
Overall, among Second Edition and First Edition students, slightly more than 60%
of the students in both the fall and the spring disagreed with the statement, There is
nothing creative about mathematics; it is just memorizing formulas and things. Again,
there are some large differences in the attitudes at the pair level. In the fall, the difference
in percentage disagreement differs by at least 25% for students in pairs 4, 5, 6, and 19; for
all but pair 6, this large difference is also present in the spring, except that for pair 5 the
direction of the difference is reversed. For pairs 2, 8, 12, and 14, there is also at least a
25% difference in percentage disagreement in the spring between the pairs, with the
Second Edition students more likely to disagree with the statement than the First Edition
students in all but pair 8.
Over 80% of the Second Edition and First Edition students disagreed with the
statement, Outside of science and engineering, there is little need for mathematics in
jobs, in both the fall and the spring. At least a 25% difference, with First Edition students
more likely to disagree, occurs in pair 14; for the First Edition students in both pairs at
School H, the percentage disagreement with the statement increased by at least 25% from
fall to spring, perhaps due to a teacher effect or to the use of applications in a wide
context within the curriculum.
For the Second Edition and non-UCSMP sample, results are fairly comparable for
both groups on the four items at both fall and spring administrations of the survey; there
are no overall differences that are at least 15%. In general, the results suggest that both
groups have a somewhat positive view of mathematics.
On the item, Mathematics is an interesting subject, better than 70% of both
groups agreed with the statement in the fall. The overall percentage who agreed in the
spring was lower for both groups but only for the Second Edition students did the
88
Table 29. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Attitudes Toward Mathematics
as a Discipline: Second Edition and First Edition
School B
Pair 2
2nd
1st
n=18 n=19
School C
Pair 4
2nd
1st
n=12 n=20
School D
Pair 5
2nd
1st
n=12 n=8
Fall
agree
disagree
78
22
47
53
42
58
60
40
67
33
25
75
Spring
agree
disagree
56
39
42
58
58
42
70
30
42
58
Fall
agree
disagree
94
5
89
8
92
5
95
8
92
Spring
agree
disagree
100
16
84
Fall
agree
disagree
39
61
Spring
agree
disagree
School E
Pair 6
Pair 7
2nd
1st
2nd
1st
n=6 n=10
n=11 n=12
Mathematics is an interesting subject.
91
9
90
10
87
13
80
20
82
18
13
87
50
70
36
50
68
50
30
64
50
32
Mathematics is more for boys than for girls.
91
9
70
30
80
20
90
10
55
45
25
75
5
94
100
10
90
20
73
100
100
100
5
33
20
8
95
67
100
100
80
100
92
100
100
100
There is nothing creative about mathematics; it is just memorizing formulas and things.
6
93
100
100
37
63
50
50
20
80
20
80
20
80
27
73
33
67
68
32
50
50
10
90
27
73
10
90
45
55
Fall
agree
disagree
11
83
11
89
25
75
20
80
25
75
Spring
agree
disagree
33
67
16
79
42
58
20
80
17
83
63
37
100
17
83
70
30
100
50
50
64
27
100
27
73
75
25
Pair 9
2nd
1st
n=10 n=15
94
5
33
67
100
Pair 8
2nd
1st
n=19 n=11
School G
Pair 12
2nd
1st
n=10 n=11
100
17
83
5
95
9
91
100
67
37
20
45
50
42
9
20
33
63
100
80
55
50
58
91
80
Outside of science and engineering, there is little need for mathematics in jobs.
100
17
83
38
63
17
83
100
18
80
8
91
10
90
100
8
83
89
100
9
91
10
90
100
100
9
82
5
90
9
91
20
80
6
93
10
90
9
91
Table 29 continued
School H
Pair 14
Pair 15
2nd
1st
2nd
1st
n=16 n=12
n=11 n=11
Fall
agree
disagree
56
44
58
42
64
36
55
45
Spring
agree
disagree
56
44
33
67
73
27
36
64
Fall
agree
disagree
100
17
83
100
91
Spring
agree
disagree
School I
School J
Pair 17
Pair 18
Pair 19
2nd
1st
2nd
1st
2nd
1st
n=16 n=12
n=16 n=18
n=7 n=11
86
14
82
18
74
25
66
33
94
83
63
67
71
6
17
32
28
29
73
27
64
34
62
37
100
3
96
7
91
100
17
9
18
3
83
91
100
100
100
100
100
100
82
97
7
93
Fall
agree
disagree
50
50
50
50
26
74
25
74
Spring
agree
disagree
50
50
75
25
55
45
19
25
25
28
29
36
45
55
81
75
63
72
72
100
63
34
66
Fall
agree
disagree
12
88
42
58
18
82
36
64
6
94
8
92
6
94
11
89
100
9
91
11
89
13
83
Spring
agree
disagree
50
50
17
83
100
9
91
100
17
83
13
87
11
89
14
86
9
82
18
82
13
85
27
73
9
82
69
31
100
31
69
58
42
8
92
25
75
81
12
6
94
6
94
67
28
Overall
Results
2nd
1st
n=164 n=170
11
89
17
83
100
29
74
100
Note: Percentages do not always add to 100% as some students did not answer some items.
90
Table 30. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Attitudes Toward Mathematics
as a Discipline: Second Edition and non-UCSMP
School X
Pair 21
2nd
non
n=14 n=14
Pair 22
2nd
non
n=21 n=16
School Y
School Z
Pair 23
Pair 24
Pair 25
Pair 26
2nd
non
2nd
non
2nd
non
2nd non
n=19 n=17
n=25 n=16
n=11 n=17
n=8 n=11
Fall
agree
disagree
86
14
93
7
81
19
88
12
74
26
Spring
agree
disagree
64
34
100
67
33
50
50
47
53
Fall
agree
disagree
7
93
100
5
95
94
100
Spring
agree
disagree
7
93
14
86
Fall
agree
disagree
36
64
Spring
agree
disagree
71
29
75
25
55
36
74
26
71
26
29
60
44
45
65
70
40
56
55
30
75
25
73
27
59
41
58
41
100
100
4
96
1
98
5
5
8
6
95
100
95
100
92
100
100
94
100 100
5
95
3
97
7
93
38
57
27
73
34
65
23
75
36
64
29
71
52
48
38
37
41
48
44
64
38
25
9
62
63
58
52
56
36
59
75
90
44
55
35
65
Fall
agree
disagree
14
86
57
43
9
90
100
5
95
24
76
16
84
25
75
100
23
76
37
63
27
73
12
87
25
75
Spring
agree
disagree
29
71
29
71
14
86
25
75
5
95
35
65
20
80
44
56
9
91
12
88
25
75
27
73
16
83
28
71
25
75
16
84
58
41
6
94
35
53
68
32
8
92
40
60
63
31
100
13
87
64
36
Overall
Results
2nd
non
n=98 n=91
100
45
55
100
29
70
Note: Percentages do not always add to 100% as some students did not answer some items.
91
25
75
percentage drop by at least 15%. Only for pairs 21 and 25 and only in the spring were the
pair differences at least 25%, with Second Edition students more likely to disagree with
the statement in both cases than their non-UCSMP peers.
For both groups, students tended to disagree with the statement, Mathematics is
more for boys than for girls. The percentage who agreed with the statement was less than
10% in all classes except the non-UCSMP class in pair 21.
In both fall and spring, about 60% of the students in both groups disagreed with
the statement, There is nothing creative about mathematics; it is just memorizing
formulas and things. Although there were differences at the pair level, for no pairs is the
difference at least 25% and for no pairs did the change in percentage from fall to spring
approach a 25% difference.
Likewise, most Second Edition and non-UCSMP students in both fall and spring
disagreed with the statement, Outside of science and engineering, there is little need for
mathematics in jobs. Only for pair 23 was the pair difference at least 25%, with Second
Edition students more likely to disagree with the statement than their non-UCSMP peers.
Confidence in Mathematics
Tables 31 and 32 contain students’ responses to items dealing with students’
confidence toward mathematics for the Second Edition and First Edition sample and the
Second Edition and non-UCSMP sample, respectively. As mentioned earlier, these items
were only administered on the spring survey.
Overall, students in the Second Edition and First Edition sample generally had
similar views about the three items dealing with confidence toward mathematics.
However, there were differences of at least 25% among the percentages for several of the
class pairs for each of the three items.
About half of the students in each group agreed with the statement, Mathematics
is confusing to me. However, for Second Edition students in pairs 6, 17, and 18 and for
First Edition students in pairs 6, 8, 9, 17, and 19, the percentage agreement was at most a
third, indicating that few of the students in these classes viewed mathematics as
confusing. For pairs 5, 8, 15, and 17, the pair difference is at least 25%, with Second
Edition students responding more negatively (that is, more likely to agree) in pairs 8 and
19.
About 60% of both Second Edition and First Edition students agreed with the
statement, I am good at math. Differences of at least 25% exist, with Second Edition
students more likely to agree, in pairs 6 and 15 and First Edition students more likely to
agree in pairs 4 and 19.
Among Second Edition and First Edition students, slightly more than 50% overall
agreed with the statement, I like mathematics. Less than 30% of the Second Edition
students in pair 4 and the First Edition students in pairs 2, 14, and 15 reported liking
mathematics. Pair differences of at least 25%, with Second Edition students more likely
to agree, exist in pairs 2, 6, 14, and 15; differences of this level, with First Edition
students more likely to agree, exist in pairs 4 and 8.
92
Table 31. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Confidence Toward
Mathematics: Second Edition and First Edition
School B
Pair 2
2nd
1st
n=18 n=19
School C
Pair 4
Pair 5
2nd
1st
2nd
1st
n=12 n=20
n=12 n=8
agree
disagree
67
28
74
21
75
25
60
40
50
50
75
25
agree
disagree
61
33
42
58
42
50
70
25
50
50
50
50
agree
disagree
39
55
16
84
25
75
75
25
42
58
62
38
School H
Pair 14
Pair 15
2nd
1st
2nd
1st
n=16 n=12
n=11 n=11
agree
disagree
69
25
83
17
55
45
91
9
agree
disagree
31
69
33
67
73
27
18
73
agree
disagree
44
56
17
83
55
45
27
73
School D
Pair 6
Pair 7
2nd
1st
2nd
1st
n=6 n=10
n=11 n=12
Mathematics is confusing to me.
17
30
45
50
67
70
36
50
I am good at math.
67
40
64
67
33
60
36
33
I like mathematics.
67
40
55
42
17
60
45
50
School I
School J
Pair 17
Pair 18
Pair 19
2nd
1st
2nd
1st
2nd
1st
n=16 n=12
n=16 n=18
n=7 n=11
13
33
25
44
57
27
87
67
69
50
43
73
I am good at math.
94
83
69
50
43
82
6
8
19
50
57
18
I like mathematics.
75
83
63
55
71
73
25
17
31
33
14
27
School E
Pair 8
Pair 9
2nd
1st
2nd
1st
n=19 n=11
n=10 n=15
63
37
18
82
40
60
20
80
60
40
45
55
63
37
82
18
80
20
80
20
40
60
36
55
42
58
91
9
60
40
73
27
50
40
55
45
Overall
Results
2nd
1st
n=164 n=170
50
46
50
48
60
37
57
41
51
46
54
44
Note: Percentages may not add to 100% because some students did not respond to all of the items.
93
School G
Pair 12
2nd
1st
n=10 n=11
Table 32. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Confidence Toward
Mathematics: Second Edition and non-UCSMP
School X
Pair 21
2nd
non
n=14 n=14
Pair 22
2nd
non
n=21 n=16
agree
disagree
50
43
14
86
67
33
44
50
agree
disagree
50
43
79
21
47
47
56
38
agree
disagree
57
43
93
7
62
38
50
44
School Y
School Z
Pair 23
Pair 24
Pair 25
Pair 26
2nd
non
2nd
non
2nd
non
2nd non
n=19 n=17
n=25 n=16
n=11 n=17
n=8 n=11
42
47
64
38
36
53
37
45
58
53
36
62
64
47
63
55
I am good at math.
58
47
36
44
64
58
38
45
42
53
60
56
36
41
62
55
I like mathematics.
68
41
60
50
36
64
75
64
32
58
40
50
64
35
25
36
Note: Percentages may not add to 100% because some students did not respond to all of the items.
94
Overall
Results
2nd
non
n=98 n=91
53
46
40
58
46
49
55
43
60
40
59
40
For the Second Edition and non-UCSMP sample, both groups of students
responded in comparable ways to the three items dealing with confidence toward
mathematics. Approximately 60% of each group reported that they liked mathematics,
about half of each group reported they were good at mathematics, and about half reported
mathematics as confusing to them.
There were some differences at the pair level. For pair 21, the difference in
percentage was at least 25% for all three items about confidence in mathematics, with the
Second Edition students responding more negatively than their non-UCSMP counterparts
(i.e., less likely to agree) on all three items. For the statement, Mathematics is confusing
to me, Second Edition students in pair 24 were more likely to report mathematics as
confusing than their non-UCSMP peers. For the statement, I like mathematics, Second
Edition students responded more positively than their non-UCSMP peers in pair 23 and
less positively in pair 25.
Calculator Use
Tables 33 and 34 report students’ responses to the two items dealing with
calculator use for the Second Edition and First Edition sample and the Second Edition
and non-UCSMP sample, respectively.
For the Second Edition and First Edition sample, both groups of students had
similar attitudes toward calculators, although there was again variability in responses at
the pair level. Roughly 70% of students in both the fall and spring administrations agreed
with the statement, Using a calculator helps me learn mathematics, and there was little
overall change in views from the fall to the spring. In the fall, the difference in percentage
agreement was at least 25% for pairs 5 and 17, with Second Edition students more likely
to agree than their First Edition peers in pair 17; however, the difference for pair 17 did
not exist in the spring. In the spring, pair differences reached the 25% mark for pairs 7, 8,
14, and 19, with Second Edition students more positive in pairs 8 and 19. For pair 14, the
pair difference is due to a large decrease in percentage agreement from fall to spring
among Second Edition students; for pair 19, the pair difference is due to a large increase
in the percentage agreement among Second Edition students.
Although students viewed a calculator as helping them learn mathematics, a
surprisingly large percentage (46%) in both groups agreed with the statement, If you use
a calculator too much, you forget how to do mathematics. Only for pair 6 at the spring
administration did the pair difference reach the 25% mark. Within classes, the percentage
agreement with the statement (a negative response) increased or decreased from fall to
spring by more than 25% for Second Edition students in pairs 4, 12, and 17 and for First
Edition students in pairs 14 and 17.
For the Second Edition and non-UCSMP sample, there are some notable
differences in attitudes toward calculator use. For both the fall and spring administrations,
the Second Edition students responded more positively than their non-UCSMP peers to
the statement, Using a calculator helps me learn mathematics. Overall, there was no
change in the percentage agreement among the non-UCSMP students; among Second
Edition students, the change in the percentage agreement from fall to spring was 15%.
95
Table 33. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Calculators: Second
Edition and First Edition
School B
Pair 2
2nd
1st
n=18 n=19
a
School C
School D
School E
Pair 4
Pair 5
Pair 6
Pair 7
Pair 8
Pair 9
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=12 n=20
n=12 n=8
n=6 n=10
n=11 n=12
n=19 n=11
n=10 n=15
Using a calculator helps me learn mathematics.
Fall
agree
disagree
61
39
79
21
67
33
85
15
Spring
agree
disagree
89
11
89
11
58
42
70
30
Fall
agree
disagree
44
50
37
63
17
83
35
65
42
58
38
62
33
67
30
70
64
36
58
42
47
53
Spring
agree
disagree
50
44
37
63
58
42
50
45
42
58
62
38
50
50
20
80
73
27
58
42
63
37
75
25
100
67
17
60
40
55
45
50
50
79
21
55
45
80
20
67
33
80
20
82
18
70
30
60
40
60
40
64
36
27
73
40
60
60
40
50
50
36
64
36
64
30
70
53
47
20
80
36
64
75
50
67
40
45
75
74
45
17
50
33
60
45
17
26
55
If you use a calculator too much, you forget how to do mathematics.a
On the fall administration, the item had arithmetic in place of mathematics.
96
School G
Pair 12
2nd
1st
n=10 n=11
Table 33 continued.
School H
School I
School J
Pair 14
Pair 15
Pair 17
Pair 18
Pair 19
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=16 n=12
n=11 n=11
n=16 n=12
n=16 n=18
n=7 n=11
Fall
agree
disagree
81
19
83
17
82
18
Spring
agree
disagree
56
44
92
8
Fall
agree
disagree
25
75
Spring
agree
disagree
44
56
100
88
12
50
50
69
31
89
11
71
29
91
9
74
26
76
24
91
91
75
67
69
67
100
73
9
19
17
25
28
9
71
26
69
26
25
75
36
64
18
82
25
75
8
92
38
62
39
61
29
71
18
82
38
62
34
66
50
50
18
82
18
82
50
50
58
42
44
50
55
44
43
57
55
45
46
52
46
54
Note: Percentages may not add to 100% as some students did not answer all items.
a
Overall
Results
2nd
1st
n=164 n=170
97
Table 34. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Calculators: Second
Edition and non-UCSMP
School X
Pair 21
2nd
non
n=14 n=14
Pair 22
2nd
non
n=21 n=16
School Y
School Z
Pair 23
Pair 24
Pair 25
Pair 26
2nd
non
2nd
non
2nd
non
2nd non
n=19 n=17
n=25 n=16
n=11 n=17
n=8 n=11
Fall
agree
disagree
43
50
36
64
90
9
56
31
75
25
36
64
65
33
49
46
Spring
agree
disagree
71
29
43
50
90
10
75
68
47
80
25
73
70
75
25
26
53
20
75
27
29
25
27
73
80
21
49
49
Fall
agree
disagree
14
86
57
43
38
62
37
56
32
68
6
94
16
84
44
50
45
55
35
65
50
50
36
64
30
70
35
63
Spring
agree
disagree
50
50
50
50
33
67
38
62
21
79
18
82
32
68
44
56
27
73
65
35
63
37
64
36
35
65
45
55
47
47
82
11
62
36
38
56
73
27
Note: Percentages may not add to 100% as some students did not answer all items.
a
Overall
Results
2nd
non
n=98 n=91
98
41
59
This is the only item for which a χ2 test between the overall percentages (in the spring)
resulted in significance at the high standard of less than 0.003.
For the statement, Using a calculator helps me learn mathematics, the pair
differences in the fall are at least 25% for four of the six pairs, with the results more
positive for Second Edition students in three of the four. In the spring, the other two of
the six pairs (pairs 21 and 24) had differences of at least 25%, due primarily to an
increase in the percentage agreement for the Second Edition students. Perhaps these
differences are due to curricular differences, with the UCSMP Algebra curriculum
incorporating calculator technology on a regular basis.
Overall, Second Edition and non-UCSMP students responded comparably to the
statement, If you use a calculator too much, you forget how to do mathematics, in both
fall and spring; furthermore, there was little change from the fall to the spring. The pair
difference was at least 25% in the fall for three of the six pairs (pairs 21, 23, and 24), with
Second Edition students less likely to agree than non-UCSMP students in two of the three
pairs. However, these large differences no longer existed in the spring. In the spring, a
large pair difference existed only for pair 25, with Second Edition students less likely
than non-UCSMP students to agree with the statement; this difference is likely due to a
large increase in the percentage agreement from fall to spring among non-UCSMP
students, indicating a more negative view toward calculators.
Attitudes Toward the Current Course
Tables 35 and 36 report students’ responses to three items that assess attitudes
about the mathematics course during the year for the Second Edition and First Edition
sample and for the Second Edition and non-UCSMP sample, respectively.
For students in the Second Edition and First Edition sample, overall responses to
the three items were roughly comparable. Over 80% of the students in both groups agreed
with the statement, Most of the material covered in my mathematics class this year was
new to me, perhaps reflecting the emphasis on algebra concepts throughout the course.
Only for pair 7 was there a large pair difference, with Second Edition students more
likely to agree with the statement than First Edition students.
In general, over 60% of Second Edition and First Edition students disagreed with
the statement, I don’t feel I know what I am doing because there is not enough review
done in my mathematics class. This suggests that students using both Second Edition and
First Edition UCSMP materials viewed their respective text as having sufficient review
for them to learn. However, the pair difference was at least 25% for pairs 2, 5, and 12,
with First Edition students more likely to agree than their Second Edition peers in pairs 2
and 12.
About 60% of the students in both groups disagreed with the statement, My
teacher moves too quickly through the material for me to keep up. Given that each edition
of the UCSMP text is designed to be studied at a pace of a lesson per day, these responses
suggest that most students viewed the pace as appropriate. However, there are large
99
Table 35. Percentages of Students Agreeing or Disagreeing on the Student Opinion Survey to Items Dealing with Their Mathematics
Course: Second Edition and First Edition
School B
Pair 2
2nd
1st
n=18 n=19
agree
disagree
89
11
95
5
agree
disagree
22
72
58
42
agree
disagree
50
44
63
37
agree
disagree
agree
disagree
agree
disagree
School C
School D
School E
Pair 4
Pair 5
Pair 6
Pair 7
Pair 8
Pair 9
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=12 n=20
n=12 n=8
n=6 n=10
n=11 n=12
n=19 n=11
n=10 n=15
Most of the material covered in my mathematics class this year was new to me.
92
100
92
100
83
60
91
58
89
100
70
73
8
8
17
40
9
42
11
30
27
I don't feel I know what I am doing because there is not enough review done in my mathematics class.
40
25
42
33
10
36
25
32
18
20
7
58
70
50
100
67
90
64
75
68
82
80
93
My teacher moves too quickly through the material for me to keep up.
33
10
33
33
20
27
33
26
9
10
27
67
85
58
100
50
80
73
67
74
91
90
73
School H
School I
School J
Overall
Pair 14
Pair 15
Pair 17
Pair 18
Pair 19
Results
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=16 n=12
n=11 n=11
n=16 n=12
n=16 n=18
n=7 n=11
n=164 n=170
69
58
73
73
94
92
75
89
100
82
85
84
25
42
27
27
6
8
25
11
9
15
16
44
58
64
45
17
31
28
29
9
30
25
56
42
36
55
100
83
63
72
57
91
68
72
63
42
81
73
8
38
17
43
35
29
50
50
18
27
100
92
38
83
57
91
60
69
Note: Percentages may not add to 100% because some students did not answer all items.
100
School G
Pair 12
2nd
1st
n=10 n=11
90
10
91
9
100
27
74
20
80
73
27
Course: Second Edition and non-UCSMP
School X
Pair 21
2nd
non
n=14 n=14
agree
disagree
93
7
57
43
agree
disagree
43
57
100
agree
disagree
57
37
7
93
School Y
School Z
Overall
Pair 22
Pair 23
Pair 24
Pair 25
Pair 26
Results
2nd
non
2nd
non
2nd
non
2nd
non
2nd non
2nd
non
n=21 n=16
n=19 n=17
n=25 n=16
n=11 n=17
n=8 n=11
n=98 n=91
71
62
68
71
76
75
82
71
75
82
76
69
29
38
32
29
24
25
18
29
25
18
23
31
43
44
16
65
36
50
36
12
25
18
34
33
52
56
84
35
64
50
64
82
75
82
65
66
48
31
26
59
36
50
55
29
25
41
32
52
69
74
41
64
50
45
70
75
100
58
68
Note: Percentages may not add to 100% because some students did not answer all items.
101
differences in attitudes in pairs 5, 12, and 19, with Second Edition students more likely
than First Edition students to agree in pairs 5 and 19.
For the students in the Second Edition and non-UCSMP sample, responses were
roughly comparable overall on all three items. Overall, about 70% of the students in both
groups agreed that the course material was new. As previously mentioned, only for pair
21 was there a large difference in responses between students, with Second Edition
students more likely to agree that material was new. However, there is at least a 25%
difference in the percentage agreement on all three items for students in pair 21, with
Second Edition students agreeing more than non-UCSMP students with each statement.
About a third of the students in both groups reported not having enough review in the
course. However, only for two pairs was there at least a 25% difference in the percentage
agreement in the students’ responses, with Second Edition students more likely to agree
in one pair and non-UCSMP students in the other.
Slightly more than a third of the students in both groups reported the teacher as
moving through the material too quickly. For four of the six pairs, there was a difference
in percentage agreement of at least 25%, with Second Edition students more likely to
disagree in two of the four pairs.
Attitudes About the Textbook and Its Features
Tables 37 and 38 report students’ responses to four items dealing with the
textbook and its features for the Second Edition and First Edition sample and the Second
Edition and non-UCSMP sample, respectively.
Overall, responses to these four items were roughly comparable for Second
Edition and First Edition students. Over 80% of both groups of students agreed with the
statement, It is important to read your mathematics text if you want to understand
mathematics. However, the difference in the pair means was at least 25% for five pairs,
with Second Edition students more likely to agree with the statement than First Edition
students in pairs 6, 12, and 14 and First Edition students more likely to agree in pairs 4
and 5.
Over 70% of the students in both groups agreed with the statement, Many
problems in my textbook are not very interesting. Large pair differences exist for four
pairs (pairs 6, 8, 12, and 19), with First Edition students more likely to agree in three of
these four pairs.
Slightly more than 50% of Second Edition students agreed with the statement, I
find my textbook easy to understand; however, slightly less than 50% agreed that The
textbook helps us to understand what we did not quite understand during class. Although
students report finding the text easy to understand, they did not necessarily view the text
as helping them fill in gaps in understanding from class. For four of the thirteen Second
Edition classes, the percentage of students who reported finding the text easy to
understand was less than 35%. For three pairs, the difference in percentage agreement
reached the 25% mark, with First Edition students more likely than Second Edition
students to agree that the text helps understanding in two of the three pairs.
102
Textbook: Second Edition and First Edition
School B
Pair 2
2nd
1st
n=18 n=19
agree
disagree
83
17
84
16
agree
disagree
72
28
74
26
agree
disagree
61
33
26
74
agree
disagree
56
44
42
53
School C
School D
School E
Pair 4
Pair 5
Pair 6
Pair 7
Pair 8
Pair 9
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=12 n=20
n=12 n=8
n=6 n=10
n=11 n=12
n=19 n=11
n=10 n=15
It is important to read your mathematics text if you want to understand mathematics.
67
95
67
100
100
70
91
67
89
91
90
80
33
5
33
30
9
33
11
9
10
20
Many problems in my textbook are not very interesting.
75
60
67
50
50
80
91
67
89
45
60
73
25
40
33
50
33
20
9
25
11
55
40
27
I find my textbook easy to understand.
50
35
50
63
50
60
27
50
68
64
60
67
50
65
50
37
50
40
73
50
32
36
40
33
The textbook helps us to understand what we did not quite understand during class.
33
35
50
38
17
80
36
67
53
73
40
53
67
60
42
62
83
20
64
33
47
27
60
47
103
School G
Pair 12
2nd
1st
n=10 n=11
90
10
55
36
60
40
100
20
80
55
36
30
70
27
55
Table 37 continued
agree
disagree
agree
disagree
agree
disagree
agree
disagree
School H
School I
School J
Overall
Pair 14
Pair 15
Pair 17
Pair 18
Pair 19
Results
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
n=16 n=12
n=11 n=11
n=16 n=12
n=16 n=18
n=7 n=11
n=164 n=170
100
58
100
100
81
92
100
78
100
82
88
81
42
19
8
22
18
12
18
81
92
82
73
56
58
63
78
71
100
72
73
19
8
18
18
44
42
31
22
29
27
26
31
33
64
18
94
75
75
39
29
27
55
45
63
67
36
82
6
25
13
50
43
55
40
52
31
50
45
27
75
67
75
44
43
36
48
48
62
50
55
73
25
33
13
56
43
64
49
49
Note: Percentages may not add to 100% as some students did not respond to some items.
104
Textbook: Second Edition and non-UCSMP
School X
Pair 21
2nd
non
n=14 n=14
agree
disagree
93
7
50
50
agree
disagree
64
36
79
21
agree
disagree
14
86
29
71
agree
disagree
21
79
14
86
School Y
School Z
Pair 22
Pair 23
Pair 24
Pair 25
Pair 26
2nd
non
2nd
non
2nd
non
2nd
non
2nd non
n=21 n=16
n=19 n=17
n=25 n=16
n=11 n=17
n=8 n=11
81
56
53
47
80
31
91
94
100 100
19
44
47
53
20
69
9
57
87
63
88
64
81
82
41
38
45
43
13
37
6
36
19
18
59
62
55
62
50
63
35
56
56
45
88
63
91
38
50
37
65
40
44
55
12
37
9
57
38
53
47
56
62
64
94
63
91
43
56
47
53
44
38
36
6
37
9
Note: Percentages may not add to 100% as some students did not respond to some items.
105
Overall
Results
2nd
non
n=98 n=91
80
20
62
37
62
38
71
27
52
47
57
43
52
48
57
42
For students in the Second Edition and non-UCSMP sample, a larger percentage
of Second Edition students as compared to non-UCSMP students (80% vs. 62%) agreed
with the statement, It is important to read your mathematics text if you want to
understand mathematics. For three of the six pairs (pairs 21, 22, and 24), there is a large
difference in the percent agreement, with Second Edition students more likely than nonUCSMP students to agree in all three pairs. These results suggest that the UCSMP
students internalized the message in the text about the importance of reading.
Over 60% of the students in both groups agreed with the statement, Many
problems in my textbook are not very interesting. Large pair differences exist for pairs 22,
23, and 25, with non-UCSMP students more likely than Second Edition students to agree
in two of the three pairs.
Slightly more than half of the students in each group agreed with the statement, I
find my textbook easy to understand. Given that textbook developers design their
materials so that students can use them for learning, this result is encouraging. For pairs
23, 25, and 26, the pair difference in the percentage agreement is at least 25%, with nonUCSMP students more likely than Second Edition students to agree in two of the three
pairs.
About half of the students in each group agreed that the textbook could be used to
help understand what was not clear from class. For both pairs at School Z, the class
differences reached the 25% mark, with non-UCSMP students more likely than Second
Edition students to agree with the statement. Thus, at this school, the non-UCSMP text
was viewed as a learning tool more positively than the UCSMP Algebra text.
Teachers’ Attitudes
Teachers were not given a questionnaire or survey at the end of the year to assess
their opinions about the course or the textbook. However, some comments about UCSMP
Algebra, both Second Edition and First Edition, and its features were obtained as part of
the interviews conducted during the visits to each school conducted during the spring.
Several teachers commented about the content of the course. The following
comments are illustrative of these views.
I think they’re [the students] a lot more knowledgeable [than previous
students]. They don’t realize it all of the time … . They had things that
five years ago [they] couldn’t have, like statistics, and graphs, and charts
and data and a lot of things. (Second Edition teacher, School B)
When I taught algebra from the traditional texts, there were so few real
applications, the word problems were all contrived. … And this book
provides the applications on a daily basis. We generate a discussion just
about every other day. … That’s the thing I think that I enjoy the most …
not having to say “ When will we ever use this?” Well, we used it today.
We used it yesterday, you’re going to use it tomorrow. (First Edition
teacher, School B)
106
Some comments focused on general features of the text, including the practice of
continual review and the varied difficulty level of the problems.
… By using the technology of the calculator, I think it’s broadened the
amount of mathematics that they can do. So they learn a lot and they think
they’re learning new material, where in our old program in the 7th grade,
you didn’t learn any new materials. (Second Edition teacher, School C)
I like the way we review; I think the review is really important. Because
they seem to really need it. (Second Edition teacher, School E)
And my kids in the Chicago Algebra, they just know now that you don’t
stop after 10 questions because they [the problems] get harder; sometimes
they get easier. They never know. So they know, try each one. Because the
difficult ones are not always at the end. (First Edition teacher, School C)
Well, the nice thing about this book that I found [is] … that they don’t
have to feel like they’ve mastered it [the content] at that particular chapter,
because of the reviews. (First Edition teacher, School G)
That they do get them [skill practice] in the review, here it comes again,
here it comes again, and I bet by the time they’re done with the chapter
they had just as much practice on that. It just hasn’t been there on the first
day. (First Edition teacher, School I)
Other comments focused on the difference between the UCSMP textbook
and more traditional textbooks.
… when they look at some sample books over on the old shelves, they
look at those and say “Ugh! It looks boring, this looks awful.”And so they
find the Chicago books a lot more interesting, and like a lot more fun.
(Second Edition teacher, School B)
When I ask them to factor, they’re going to be far behind classes that I had
with Houghton. But when they get out in the real world and try to get a
real job, they’re probably not going to be asked to factor, they’re going to
be asked to use their math skills for things that we’re teaching with the
UCSMP. So I definitely feel that this is the route to go. … But skill wise, I
really think it’s [UCSMP] going to hurt them a lot. (Second Edition
teacher, School D)
107
In the interviews, several teachers made general comments about the textbook that
reflected their own attitudes or perceptions or those of their students.
I think its [Second Edition] a lot more fun to teach because there’s more
variety and it’s not so repetitious or boring, like doing the same thing over
and over. (Second Edition teacher, School C)
I like this one even better than the First Edition. It flows better … I think
it’s more interesting. (Second Edition teacher, School H)
I really believe its [UCSMP] better for the students, because they have to
learn how to learn, which is what our big emphasis needs to be. (Second
Edition teacher, School D)
I think there’s maybe a little more responsibility on the students now to try
and understand it than before. It was just the routine, the same thing over
and now it’s new and when they look back they figure out for themselves.
(First Edition teacher, School C)
They [students] think it’s a little bit more fun, more interesting, more their
own work. More in terms of their discovery, things on their own. I don’t
hear the question, “When are we ever going to have to use this?” … I
think they enjoy it more. (First Edition teacher, School G)
I like it. I think it’s [UCSMP Algebra] for a higher level student because
of the reading involved. I think it’s meatier. I think the topics that are
taught or mentioned are a lot higher quality, a higher level. (Second
Edition teacher, School X)
There are a lot of students that can sit down and memorize the formula; if
they knew exactly where to put certain things in the formula, they could
crank out answers. For some students, that may be more beneficial than
this [UCSMP] method. But on the whole this [UCSMP] is a better method,
because it is better for kids to think, to analyze, to apply. (First Edition
teacher, School J)
More [students] have been turned on to some degree than have been
turned off a bit. I think the book is, I’ll say refreshing, but it’s not that
strong. But I do know that more students come to class thinking this is
108
going to make sense, “I will be able to see how to use it” than in the
traditional textbook. (Second Edition teacher, School Y)
Other teachers expressed concerns about potential limitations of the UCSMP
materials and made suggestions for improvement.
I like the way it’s [UCSMP Algebra] presented. I think it deals with the
age level and the ability. It kind of gets down to their level, where I think
they need a little bit more examples, more reading, more help than maybe
high school kids do. I think it’s good for them for that reason. Sometimes,
like I said, I think it could use a little bit more depth too, but basically I
think it’s a good program. (Second Edition teacher, School E)
I would like to see a little bit more difficult problems. Yet I realize that it
[UCSMP Algebra] was geared, when it first came out, to be for the socalled “average-level” students. I think that it was put together well. I
think that the de-emphasis on some of the material, such as in factoring
and so on, [is] certainly appropriate now that we’ve got graphing
calculators. (First Edition teacher, School G)
A few comments focused on issues related to parents.
They [parents] just say “They have to read! That’s not math, if you have to
read! There’s not enough problems. There’s not enough practice.”
(Second Edition, School C)
Maybe the negative ones [parents] weren’t calling her [the principal]. I
also know that the principal has received a couple of letters from parents
saying that this program’s really neat “my child was usually doing D work
and now they are doing high C or B work, and he has a new attitude
toward math.” (Second Edition teacher, School Y)
Summary
This chapter has discussed data to answer the research question, how do attitudes
of students and teachers using UCSMP Algebra (Second Edition, Field-Trial Version)
compare to those of students and teachers using UCSMP Algebra (First Edition) or nonUCSMP materials?
Overall, the attitudes of Second Edition and First Edition students are often quite
similar, although the tables clearly indicate some large differences at the pair level. About
60% of both groups of students found mathematics interesting and did not believe that
mathematics is mostly memorizing formulas and things. Over 80% of both groups did not
109
agree that mathematics is needed primarily for science and engineering. Very few
students believed that mathematics was more for boys than girls.
About half of the Second Edition and First Edition students thought mathematics
was confusing and reported liking mathematics; roughly 60% reported being good at
mathematics. Roughly 70% of students in both groups indicated that a calculator was
helpful to learn mathematics. However, 46% of both groups agreed that too much use of
a calculator could cause one to forget how to do mathematics.
A clear majority of both Second Edition and First Edition students reported that
most of the material in the course was new, likely related to the algebraic content of the
course. Most students in both groups indicated that the pace of the class, likely a lesson a
day, was appropriate in order for them to keep up.
Over 80% of both Second Edition and First Edition students seemed to recognize
the importance of reading in order to understand mathematics. However, over 70%
reported the problems in the textbook as not very interesting. About half of the students
reported their respective text as easy to understand and as useful to fill in gaps related to
understanding from the class.
For the Second Edition and non-UCSMP sample, there were many similarities in
responses, although differences did exist on issues related to calculators and textbook
features. Overall, almost 60% of both groups of students agreed that mathematics is
interesting. About 44% of Second Edition and 35% of non-UCSMP students agreed that
mathematics is mostly memorizing; however, only 16% of Second Edition and 28% of
non-UCSMP students reported little need for mathematics outside of science and
engineering.
Among students in the Second Edition and non-UCSMP sample, students
responded comparably to the three items dealing with confidence toward mathematics,
with no overall differences of at least 15%. About 60% of both groups reported liking
mathematics and slightly less (46% Second Edition and 55% non-UCSMP) reported
being good at mathematics. For all three items in this block, large pair differences existed
at School X, with Second Edition students responding less positively than their nonUCSMP peers on all three items.
In the spring, Second Edition students were more likely than non-UCSMP
students to report that a calculator was useful in learning mathematics (80% vs. 49%);
this was the only attitude item for which a χ2 test indicated a significant difference
(p < 0.003) overall in responses. A smaller percentage of Second Edition than nonUCSMP students (35% vs. 45%, respectively) thought too much use of a calculator
resulted in forgetting how to do mathematics. These differences are likely related to the
more frequent use of calculators by the Second Edition students as compared to their nonUCSMP counterparts.
Among students in this sample, both groups viewed the material in the course as
new (76% Second Edition and 69% non-UCSMP). About a third of both groups did not
find enough review in the course in order to understand the material.
A higher percentage of Second Edition than non-UCSMP students (80% vs. 62%)
recognized the importance of reading the mathematics text in order to understand
110
mathematics. Slightly more than half of both groups of students found their textbook easy
to understand and useful for helping to understand what was missed during class.
Only minimal attitudinal data were collected from teachers. Nevertheless,
comments during interviews suggest that First Edition teachers and Second Edition
teachers from both samples generally liked the materials, particularly the applications and
the continual review. A few teachers expressed concerns about not enough skill practice,
not enough difficult problems, and not enough depth in content as compared to traditional
algebra textbooks.
There are limitations to the design of the data collected related to attitudes.
Although students were surveyed about a number of issues, it would have been beneficial
to interview a number of students to understand better the nature of their attitudes and
factors that might influence changes in those attitudes. Likewise, more detailed teacher
questionnaires would have provided another perspective on instructional practices and
opinions about the text. The only data collected from teachers relative to these issues
occurred during the interviews.
111
112
CHAPTER 6
SUMMARY AND CONCLUSIONS
This report has described the results from the Field Test of UCSMP Algebra
(Second Edition), the second textbook in the curriculum for grades 7-12 developed by the
Secondary Component of the University of Chicago School Mathematics Project. The
study contained both formative and summative features to assess the effectiveness of the
materials. Two studies comprise the summative portion of the field test. One study
compares the achievement and attitudes of students using UCSMP Algebra (Second
Edition, Field-Trial Version) to the achievement and attitudes of students using UCSMP
Algebra (First Edition). The second study compares the achievement and attitudes of
students using UCSMP Algebra (Second Edition, Field-Trial Version) to the achievement
and attitudes of students using the more traditional (non-UCSMP) texts already in use at
the school for this course.
The First Edition of UCSMP Algebra was developed in the mid-1980s in response
to seven problems that UCSMP staff did not believe could be resolved by minor changes
in traditional content or approach in typical algebra textbooks:
•
Large numbers of students do not see why they need algebra.
•
The mathematics curriculum has been lagging behind today’s widely available
and inexpensive technology.
•
Too many students fail algebra.
•
Even students who succeed in algebra often do poorly in geometry.
•
Students don’t read.
•
High school students know very little statistics and probability.
•
Students are not skillful enough, regardless of what they are taught.
UCSMP Algebra (Second Edition) not only continued to address these issues but
also incorporated changes based on discussions within the larger mathematics education
community as a result of the release of the Curriculum and Evaluation Standards, the
Professional Teaching Standards, and the Assessment Standards of the National Council
of Teachers of Mathematics. Hence, the Second Edition incorporates more writing
throughout the text, encourages the use of longer assessments such as outside projects,
and incorporates more technology, such as spreadsheets and automatic graphers (ie.,
graphing calculators). Hence, the Second Edition attempts to address an eighth problem
in regards to student achievement: Students are not very good at communicating
mathematics in writing.
Major changes in content were not made from the First Edition to the Second
Edition. Rather, changes were made primarily in organization to give some topics more
prominence. For example, the first seven chapters were reorganized to incorporate
equation-solving much earlier in the course. In addition, quadratic equations are
introduced in Chapter 9 in the Second Edition, rather than in Chapter 12 as in the First
113
Edition. More emphasis was placed in the Second Edition on factoring of various types of
polynomials and on using variables to generalize from patterns.
Three main research questions were of interest to the study:
•
How do teachers’ instructional practices when using UCSMP Algebra
(Second Edition, Field-Trial Version) compare to teachers’ instructional
practices when using UCSMP Algebra (First Edition) or the non-UCSMP
materials currently being used in the schools?
•
How does the achievement of students in classes using UCSMP Algebra
(Second Edition, Field-Trial Version) compare to that of students using
UCSMP Algebra (First Edition) or to students using non-UCSMP materials?
•
How do attitudes of students and teachers using UCSMP Algebra (Second
Edition, Field-Trial Version) compare to those of students and teachers using
UCSMP Algebra (First Edition) or non-UCSMP materials?
The study, conducted during the 1992-1993 school year, used a matched-pair
design, with one class in each pair using the Second Edition of Algebra and the other
class using either the First Edition or the non-UCSMP text currently in use for the course
at the school. Within each school, pairs were matched on the basis of performance on the
Iowa Algebra Aptitude Test, a standardized measure focusing on preparation for algebra,
to ensure that the matched pairs at each school were comparable in terms of entering
knowledge. Without such comparability, comparisons of achievement on posttests are
meaningless and cannot be attributed to differences in curricular emphasis. The matchedpair design also takes into consideration the natural variability that occurs across schools
due to socioeconomic influences and school environment.
A number of measures were obtained at the end of the year to determine
similarities and differences between groups. Students took the High School Subject Tests:
Algebra, a standardized measure. They also completed a UCSMP constructed Algebra
Test, one of two forms of a UCSMP constructed-response Problem-Solving and
Understanding Test scored with rubrics, and a survey of opinions about mathematics,
learning mathematics, and textbook features. Teachers completed a short questionnaire
about their professional background and an opportunity-to-learn form for each item on
the posttest instruments. Throughout the year, Second Edition teachers completed chapter
evaluation forms for each chapter they completed with the class. All teachers were
interviewed; most were observed teaching at least one class involved in the study.
The samples consist of thirteen matched pairs in eight schools for the Second
Edition (n = 164) and First Edition (n = 170) study and six matched pairs in three schools
for the Second Edition (n = 98) and non-UCSMP (n = 91) study.
This final chapter of the report summarizes the results of the two studies, draws
comparisons between the two studies when appropriate, discusses some of the issues that
arise when conducting such a study, and describes changes made from the Field-Trial
Version to the final commercial version of Algebra (Second Edition).
114
The Implemented Curriculum
Content Coverage
Much of the information about content coverage was inferred from the Chapter
Evaluation Forms completed by Second Edition teachers and from the Opportunity-toLearn Forms completed by all teachers to each item on the posttests. In the Second
Edition and First Edition sample, students in the two groups studied roughly comparable
content, with the exception of the study of quadratics. As indicated through the
opportunity-to-learn measures reported in Chapter 4, First Edition teachers were less
likely than Second Edition teachers to teach quadratics, perhaps a reflection of the fact
that quadratics appear later in the First Edition text than in the Second Edition text. In
general, both groups of students studied solving equations and inequalities, translating
verbal forms of a problem into symbolic form, slopes and graphs of lines, solving
systems (except Second Edition students at School G and First Edition students at
Schools H and I), and linear equations in two variables.
For the Second Edition and non-UCSMP sample, there were some major
differences in the opportunities that students in the two groups had to learn algebra
content. Students in both groups studied equations and inequalities, graphs of lines, and
solving systems of linear equations. Second Edition students also studied translating
verbal forms to symbolic forms. Non-UCSMP students at Schools Y and Z studied
polynomial operations and rational expressions while UCSMP students did not. Second
Edition students generally studied applications of the concepts while non-UCSMP
students at Schools X and Y appeared to have limited exposure to applications. It appears
that applications were either not covered in the non-UCSMP textbooks or the wording of
questions on both the UCSMP-constructed tests and the standardized tests was
sufficiently different from the wording in the textbooks so that non-UCSMP teachers
reported that students did not have an opportunity to learn the content needed to answer
many of the items.
Technology Access and Use
Both the First and Second Editions of UCSMP Algebra assume continual access
to a scientific calculator. In the Second Edition and First Edition sample, most students
had access to calculators, either through access for classroom use or through ownership
of their own calculator. When asked about the frequency of calculator use, at least threefourths of the students in both groups reported using calculators almost every day, with
most of the others indicating use 2-3 times a week. Computer use for both groups was
much more limited.
In the Second Edition and non-UCSMP sample, calculator access was more
varied and somewhat less regular than in the Second Edition and First Edition sample. In
general, Second Edition teachers reported that their students had scientific calculators,
with some access to graphing calculators. Over half of the Second Edition students
reported using calculators almost every day, with most of the rest reporting use 2-3 times
per week; only 12% of Second Edition students reported calculator use at 2-3 times a
month or less. Among non-UCSMP students, a fourth reported calculator use at less than
115
once a month and another fourth reported use 2-3 times a month. Again, the ability to
engage students in computer work was limited.
Instructional Practices
The mean amount of time spent in mathematics class for the Second Edition and
First Edition sample was 44.5 minutes (s.d. = 3.9 minutes), with class times ranging from
40 to 50 minutes. For students in the Second Edition and non-UCSMP sample, the mean
class time was 51.3 minutes (s.d. = 7.6 minutes), with class periods ranging from 43
minutes to 58 minutes.
In the Second Edition and First Edition sample, all of the teachers who were
asked about reading indicated they expected students to read, and consequently, assigned
the reading. When students were queried about reading their textbook, 55% of Second
Edition and 44% of First Edition students reported reading their textbook almost always
or very often.
About half of the Second Edition students and a third of the First Edition students
reported spending 16-30 minutes per day on homework; about a third of the Second
Edition and a fourth of the First Edition students reported spending from 31-45 minutes
per day on homework. About 70% of the students in both groups reported needing help
with their homework at least sometimes.
Teachers were not explicitly surveyed about their instructional practices.
However, during the teacher interviews, comments were solicited about the use of small
groups. All teachers in this sample indicated that they used some group work.
In the Second Edition and non-UCSMP sample, all three Second Edition teachers
expected their students to read; the reading issue was not discussed with the non-UCSMP
teachers. About equal percentages (36% and 39%, respectively) of Second Edition and
non-UCSMP students reported reading their textbook almost always or very often. About
a fourth of the non-UCSMP students indicated very little reading of their textbook.
About half of the Second Edition students reported spending at most 30 minutes
per day on homework; slightly more than 70% of non-UCSMP students reported this
level of homework. Students in both groups responded comparably to the item about help
with homework, with 35% of Second Edition and 29% of non-UCSMP students needing
help almost always or very often. About a third of the students in both groups reported
rarely needing help with homework.
Among teachers in the Second Edition and non-UCSMP sample, all Second
Edition teachers and two of the non-UCSMP teachers reported using some group work.
Summary
These results suggest that the implemented curriculum in the classes of the
Second Edition and First Edition sample was roughly comparable overall, at least in
terms of content coverage, technology access and use, and instructional practices such as
reading and the use of small groups. For the Second Edition and non-UCSMP sample,
116
there were some major differences in the implemented curriculum. Non-UCSMP students
were less likely to study applications than their Second Edition counterparts, used
technology less often than their Second Edition counterparts, and spent less daily time on
homework than their Second Edition peers.
The Achieved Curriculum
As indicated in Chapter 4, achievement on the High School Subject Tests:
Algebra and the UCSMP Algebra Test was analyzed in three ways whenever possible.
First, overall achievement on each of these measures is reported. Second, for each of
these two measures, a Fair Test was constructed at each school using only those items for
which both teachers at the school reported that students had an opportunity to learn
(OTL) the content needed to answer the items; hence, this test controls for OTL at the
school level. Third, for each of the two studies, a Conservative Test was constructed
using only those items for which all teachers in the respective sample reported that
students had an opportunity to learn the content needed to answer the items. Note that this
test controls for OTL for the entire sample.
Achievement on the High School Subject Tests: Algebra
On this 40-item standardized multiple-choice test, in the Second Edition and First
Edition sample, OTL measures ranged from 53% to 95% for Second Edition students and
from 68% to 100% for First Edition students. The overall mean score was 45% for both
Second Edition and First Edition students, corresponding to the 45th percentile.
Applying a repeated measures t-test to these results, overall achievement differences
between Second Edition and First Edition students are not significantly different.
For neither the Fair Tests nor the Conservative Test, both of which control for
OTL, was there any significant difference in achievement between students in the two
groups. The Conservative Test consists of only 8 of the 40 items, indicating some
disagreement regarding content across the eight schools in the sample.
For the Second Edition and non-UCSMP sample, OTL measures ranged from
93% to 98% for Second Edition students and from 58% to 93% for non-UCSMP
students. Mean scores for Second Edition and non-UCSMP students were 48% and 46%,
respectively, corresponding to the 48th and 45th percentiles, respectively.
On neither the Fair Tests nor the Conservative Test were achievement differences
between the two groups significant. However, for this sample the Conservative Test
consists of only 13 items.
The achievement results on this standardized measure, particularly for the Second
Edition and non-UCSMP sample, provide some answers to critics of Standards-based
curricula. Critics often assume that students studying from such curricula will not be
successful on traditional measures. In this case, students in the UCSMP Algebra classes
studied from a curriculum that is broader in scope than the traditional curriculum and
incorporates multiple perspectives (skills, properties, uses, and representations). Yet, they
scored comparably to their non-UCSMP counterparts.
117
Achievement on the UCSMP Algebra Test
This UCSMP-constructed test consists of 40 multiple-choice items, with ten
requiring translation from verbal to symbolic form, six involving linear relationships with
two variables, five on quadratic equations and relationships, five on geometric
relationships, four on statistics or probability, three on percent applications, two on graph
interpretation, two on exponential relationships, and three miscellaneous items.
On the entire test, overall achievement was roughly 56% for Second Edition
students and 53% for First Edition students. A repeated measures t-test indicates no
significant difference in overall achievement between the Second Edition and First
Edition students. Likewise, on both the Fair Tests and the Conservative Test, which
control for OTL, there were no significant differences overall in achievement between
students studying from the Second Edition or First Edition curriculum. The Conservative
Test consisted of 11 of the 40 items, again suggesting not as much agreement related to
curriculum between the Second and First Edition teachers as the curriculum developers
might have expected.
On four items, all dealing with translating words to symbols, at least 80% of both
Second Edition and First Edition students were successful; three of these items were
classified as uses and one as a property. In addition, at least 80% of Second Edition
students were successful on a percent item dealing with finding the total bill, including a
tip. Less than 20% of either Second Edition or First Edition students were successful at
finding the ratio of one number to another in context.
In the Second Edition and non-UCSMP sample, the Second Edition students were
successful on roughly 50% of the items; non-UCSMP students had a mean success
overall of 38%. A repeated measures t-test indicates significant achievement differences
between students studying from the two curricula. However, there were major differences
in the OTL measures which likely contributed to these differences; OTL ranged from
83% to 100% for the Second Edition students and from 23% to 95% for the non-UCSMP
students.
For the Fair Tests and the five-item Conservative Test, consisting of 5 items, no
overall significant differences existed between students studying from the Second Edition
or non-UCSMP curricula. Both of these tests control for OTL; hence, these results
suggest comparable achievement when opportunity to learn is considered.
Among the 40 items on this test, there were 9 items for which the difference in the
percent successful among Second Edition and non-UCSMP students was at least 20%,
with achievement favoring Second Edition students. These items deal with translating
from words to symbols, interpreting the meaning of slope, finding area, finding angle
measures, finding probability, modeling compound interest, and using the Multiplication
Counting Principle. Among these nine items, six deal with uses, one with properties, and
two with representations.
Achievement on the Problem-Solving and Understanding Test
For neither sample is achievement on the Problem-Solving and Understanding
Test high. For the Second Edition and First Edition sample, mean achievement was
118
slightly more than 30% on the odd form and around 50% on the even form. There were
no overall differences in achievement between students studying from the Second Edition
or First Edition curriculum on either form of the test.
Both Second Edition and First Edition students had difficulty with the
constructed-response items on both forms of the test. The small sizes in the pairs due to
the use of two forms of the test make it difficult to draw reliable conclusions about
achievement on the PSU Test items.
For the Second Edition and non-UCSMP sample, the mean achievement on the
odd form was about 37% for Second Edition students and 19% for non-UCSMP students;
on the even form, mean achievement was roughly 46% and 26%, respectively. However,
OTL likely explains many of the differences. Although Second Edition teachers reported
teaching the content for 100% of the items on both forms, non-UCSMP teachers reported
teaching the content for 50-75% of the items on the odd form and 25-75% of the items on
the even form. Again, the numbers of students taking each form at the pair level, together
with the limited OTL for non-UCSMP students, make it difficult to draw any reliable
conclusions about item achievement.
In general, students in all four groups had difficulty with these constructedresponse items on which they had to justify their thinking and explain their solution,
suggesting that all students need more practice with tasks for which they need to explain
their thinking.
Summary
Overall, these results suggest that there are no significant differences in
achievement between students studying from the Second Edition or First Edition
curricula, regardless of how the data are analyzed, on any of the posttest measures. For
the Second Edition and non-UCSMP sample, there are no significant differences on a
standardized test that primarily assesses skill proficiency with algebra. On a UCSMPconstructed Algebra Test, there were differences in achievement when OTL was not
controlled but these overall differences dissipated when OTL is considered. Students in
neither sample were particularly successful on constructed-response items requiring them
to explain their thinking, indicating a need for teachers to focus on such items if students
are expected to be proficient with them.
Attitudes
Students’ Attitudes
In the Second Edition and First Edition sample, about 60% of the students in both
groups thought mathematics was an interesting subject. The majority of students in both
groups (63% and 66%, respectively) disagreed with a statement suggesting that
mathematics is mostly memorizing, an encouraging result for the curriculum developers
given the emphasis on applications.
119
About half of the students in each group reported mathematics as confusing to
them. However, 60% of Second Edition and 57% of First Edition students reported being
good at mathematics; 51% and 54%, respectively, reported liking mathematics.
Among both Second Edition and First Edition students, the majority (74% and
76%, respectively) reported that a calculator helped them learn mathematics. However,
46% of the students in both groups thought too much use of a calculator would make
them forget how to do mathematics.
Over 80% of the students in both groups of this sample found most of the material
of the course to be new to them, not surprising given the focus on algebra content. About
a third of the students in both groups reported not having enough review in the course. In
terms of pace, 41% of Second Edition and 32% of First Edition students reported the pace
of the course as too fast for them to keep up.
UCSMP students, whether Second Edition or First Edition, recognized the
importance of reading the textbook in order to understand. However, students did not
necessarily find the textbook problems interesting, even though they found the textbook
easy to understand.
Among the Second Edition and non-UCSMP sample, Second Edition and nonUCSMP students reported mathematics as interesting (74% and 71%, respectively) and
44% and 35%, respectively, reported mathematics as mostly memorizing formulas and
things. The majority of students in both groups disagreed that mathematics was primarily
for scientists and engineers (83% and 71%, respectively).
Both groups responded comparably to the items dealing with confidence toward
mathematics, with no overall differences of at least 15% on these items. Overall, 53%
and 40% of Second Edition and non-UCSMP students reported mathematics as confusing
to them. Nevertheless, 46% and 55% of Second Edition and non-UCSMP students
reported being good at mathematics, with 60% and 59%, respectively, reporting liking
mathematics.
Second Edition students were more likely than their non-UCSMP counterparts
(80% vs. 49%) to report a calculator as helping them learn mathematics; 35% vs. 45%,
respectively, were concerned that using a calculator would cause them to forget how to
do mathematics. These differences are likely attributed to differences in frequency of
calculator use in the two curricula. Only for using a calculator to learn mathematics were
the overall differences in the response patterns significant.
Most students in the Second Edition and non-UCSMP sample reported the content
of the course as new to them (76% and 69%, respectively), not surprising given the
course was likely their first exposure to significant amounts of algebra content. Both
groups of students generally considered their respective text as having sufficient review
and their course as at an appropriate pace.
Second Edition students perceived the importance of reading the textbook to a
greater extent than their non-UCSMP counterparts (80% vs. 62%). Neither group of
students reported their textbook problems as interesting. However, over half of the
students in each group reported their textbook as easy to understand and as useful to fill
in gaps in understanding that occurred during class.
120
Summary of Students’ Attitudes
The results in this section suggest the Second Edition and First Edition students
generally had comparable attitudes, with Second Edition students slightly less likely to
view themselves as good at mathematics. For students in the Second Edition and nonUCSMP sample, the attitudes of the two groups differed in terms of viewing mathematics
as mostly memorizing (Second Edition students more negative), the usefulness of
calculators in learning mathematics (Second Edition students more positive), and the
importance of reading (Second Edition students more positive).
Teachers’ Attitudes
Teachers were not explicitly surveyed about their attitudes toward mathematics.
However, Second Edition teachers did have an opportunity during the teacher interview
to comment about the book. In general, Second Edition teachers in both samples liked the
book, including the review and the applications. Nevertheless, some teachers perceived
the text as not having enough skill problems or depth in comparison to a traditional
algebra text.
Changes Made for Commercial Publication
As indicated earlier, the study involved both formative aspects to aid the
curriculum developers and summative aspects to assess the effectiveness of the materials.
As a result of preliminary results, conversations with Second Edition teachers at the two
meetings during the school year, and comments on the Chapter Evaluation forms, some
minor changes were made in the Field-Trial Version in preparation for commercial
publication.
As the Algebra text was being field-tested and then prepared for commercial
publication, other texts in the UCSMP Secondary Component were also being revised. In
those other texts, In-Class Activities were written in which students engaged in some
mathematical investigation related to upcoming lessons, often in a small group. Teachers
of those texts responded positively to these activities, and so, activities were written for
Algebra as well. Hence, the commercial version has a number of such activities in
various chapters.
A few minor changes were made in the sequence of lessons. For instance, some
minor reordering occurred in Chapter 1 with lessons on expressions, formulas, and square
roots and variables. In Chapter 3, the distributive property was moved later in the chapter
and introduced after the solving of equations. In Chapters 4 and 5, lessons were added on
automatic graphers. In Chapter 6, a lesson on weighted averages was inserted.
The biggest changes occurred related to the work with polynomials in Chapters 10
and 12. Lessons within these chapters were reordered to have Chapter 10 focus on
multiplying polynomials and Chapter 12 focus on factoring.
121
Conclusions and Discussions
It is important that such research on the effectiveness of curricula be conducted.
Too often, materials are developed, published, and used by teachers and students with no
prior knowledge that such materials are effective. The traditional curriculum in place has
often been assumed to be effective, but without evidence to support that assumption.
This study compared instructional approaches, achievement, and attitudes of
students and teachers in two samples: First Edition or Second Edition of UCSMP
Algebra; and Second Edition of UCSMP Algebra or the non-UCSMP course text
currently at the school. Teachers’ reported opportunity-to-learn measures for the items on
the posttest instruments suggest there were some differences in the curricula as
implemented in the schools, for both samples.
On the standardized achievement measure, there were no significant differences
in achievement between the UCSMP Second Edition students and the non-UCSMP
students, regardless of how the data were analyzed. Hence, concerns that students using a
Standards-based curriculum will not be competitive with students using more traditional
curriculum are not justified, at least in this case. UCSMP students were able to study an
algebra curriculum that incorporated applications and geometry without sacrificing their
ability with algebra skills.
On the UCSMP-constructed tests, there were some significant differences in
achievement between Second Edition and non-UCSMP students when opportunity-tolearn was not controlled. However, these differences dissipated when OTL was
controlled from the perspective of the teacher.
Results on the various achievement measures could certainly have been higher for
all students. However, assessments for the purpose of such curriculum research do not
influence grades. In some schools, the ethos is such that students will give their best
effort regardless; in other schools, such assessments are not taken seriously. So,
achievement results in such curriculum research efforts are perhaps underestimates of
what students may really be capable of achieving.
There is always the issue of fairness of tests. Although a standardized test was
used as one means of assessment, the evaluator, in consultation with project staff, did not
view this standardized measure as sufficient to assess the achievement of students using
UCSMP Algebra. More assessment of algebra, particularly applications and
representations, was also needed. Hence, project personnel developed a second multiplechoice assessment as well as an assessment containing constructed responses.
Critics may claim that such project-developed tests are inherently unfair to
comparison students. Every effort was made to write the majority of items that would be
fair to both groups. By controlling for the opportunity to learn through the use of Fair
Tests and a Conservative Test based on teachers’ OTL responses, we have attempted to
be as fair and upfront as possible in making comparisons. The classroom teacher is in the
best position to know whether items are fair or not; by using the classroom teacher’s
perspective, we have controlled for content knowledge in a way that is often done even
on standardized tests.
122
In addition to differences in content coverage, what other factors might account
for the achievement differences? More research is needed to understand how the
curriculum was implemented in the various schools. How does support for activities such
as small cooperative groups influence the teachers’ use of such practices and what are the
subsequent links to achievement? More research is also needed on the longitudinal
impact of such materials. Follow-up is needed to determine how students might achieve
after using UCSMP materials for several years.
There were some clear limitations to the study. In particular, the number of pairs
in the Second Edition and non-UCSMP sample is small. Despite the limitations, this
study illustrates that students studying from a rich curriculum can maintain a level of
performance on a standardized algebra test comparable to that of students using more
traditional materials while simultaneously developing greater facility with applications,
representations, and properties of mathematics. The study provides some evidence that
the reforms recommended for mathematics at the secondary level are feasible.
123
124
REFERENCES
American Testronics. High-School Subject Tests: Algebra. Glenview, IL: Author, 1988.
College Board. Academic Preparation for College: What Students Need to Know and Be
Able to Do. New York: College Board, 1983.
Countryman, Joan. Writing to Learn Mathematics: Strategies that Work K-12.
Portsmouth, NH: Heinemann Publishers, 1992.
Dolciani, Mary P., Richard G. Brown, Frank Ebos, and William L. Cole. Algebra:
Structure and Method, Book 1 (new edition). Boston: Houghton Mifflin, 1984.
Fair, Jan and Sadie C. Bragg. Algebra 1. Englewood Cliffs, NJ: Prentice Hall, 1990.
Gravetter, Frederick J., and Larry B. Wallnau. Statistics for the Behavioral Sciences. St.
Paul, MN: West Publishing Company, 1985.
Hedges, Larry V., Susan S. Stodolsky, Sandra Mathison, and Penelope V. Flores.
Transition Mathematics Field Study. Chicago, IL: University of Chicago School
Mathematics Project, 1986.
Hirschhorn, Daniel B. “A Longitudinal Study of Students Completing Four Years of
UCSMP Mathematics.” Journal for Research in Mathematics Education, 24
(1993): 136-158.
Malone, J. A., G. A. Douglas, Barry V. Kissane, and R. S. Mortlock. "Measuring
Problem-Solving Ability." In Problem Solving in School Mathematics, edited by
S. Krulik and R. E. Reys, pp. 204-215. Reston, VA: National Council of Teachers
of Mathematics, 1980.
Mathematical Sciences Education Board. Reshaping School Mathematics: A Philosophy
and Framework for Curriculum. Washington, D.C.: National Academy Press,
1990.
Mathison, Sandra, Larry V. Hedges, Susan S. Stodolsky, Penelope Flores, and Catherine
Sarther. Teaching and Learning Algebra: An Evaluation of UCSMP Algebra.
Chicago, IL: University of Chicago School Mathematics Project, 1989.
McConnell, John, Susan Brown, Susan Eddins, Margaret Hackworth, Leroy Sachs,
Ernest Woodward, James Flanders, Daniel Hirschhorn, Cathy Hynes, Lydia
Polonsky, and Zalman Usiskin. Algebra (Teacher’s Edition). Glenview, IL:
ScotForesman, 1990.
125
McConnell, John W., Susan Brown, Susan Eddins, Margaret Hackworth, Leroy Sachs,
Ernest Woodward, James Flanders, Daniel Hirschhorn, Cathy Hynes, Lydia
Polonsky, and Zalman Usiskin. Algebra: Teacher’s Edition. Glenview, IL:
ScotForesman, 1990.
McConnell, John W., Susan Brown, Zalman Usiskin, Sharon L. Senk, Ted Widerski,
Scott Anderson, Susan Eddins, Cathy Hynes Feldman, James Flanders, Margaret
Hackworth, Daniel Hirschhorn, Lydia Polonsky, Leroy Sachs, and Ernest
Woodward. Algebra: Teacher’s Edition. (Second Edition) Glenview, IL:
ScottForesman, 1996.
McConnell, John W., Susan Brown, Zalman Usiskin, Sharon L. Senk, Ted Widerski,
Susan Eddins, Cathy Hynes Feldman, James Flanders, Margaret Hackworth,
Daniel Hirschhorn, Lydia Polonsky, Leroy Sachs, and Ernest Woodward. Algebra
(Second Edition), Field-Trial Version. Chicago, IL: University of Chicago School
Mathematics Project.
National Advisory Committee on Mathematical Education, Conference Board of the
Mathematical Sciences. Overview and Analysis of School Mathematics Grades K12. Washington, D.C.: National Council of Teachers of Mathematics, 1975.
National Commission on Excellence in Education. A Nation at Risk: The Full Account.
Cambridge, MA: USA Research, 1984.
National Council of Teachers of Mathematics. An Agenda for Action: Recommendations
for School Mathematics of the 1980s. Reston, VA: National Council of Teachers
of Mathematics, 1980.
_____. Assessment Standards for School Mathematics. Reston, VA: National Council of
Teachers of Mathematics, 1995.
_____. Curriculum and Evaluation Standards for School Mathematics. Reston, VA:
National Council of Teachers of Mathematics, 1989.
_____. Professional Standards for Teaching Mathematics. Reston, VA: National Council
of Teachers of Mathematics, 1991.
National Research Council. Everybody Counts: A Report to the Nation on the Future of
Mathematics Education. Washington, D.C.: National Academy Press, 1989.
National Science Board Commission on Precollege Education in Mathematics, Science,
and Technology. Educating Americans for the 21st Century: A plan for action for
improving mathematics, science and technology education for all American
elementary and secondary students so that their achievement is the best in the
world by 1995. Washington, D.C.: National Science Board, 1983.
126
Saxon, John H., Jr. Algebra 1: An Incremental Development. Norman, OK: Grassdale
Publishers, Inc.
Senk, Sharon L. "Van Hiele Levels and Achievement in Writing Geometry Proofs."
Journal for Research in Mathematics Education, 20 (1989): 309-321.
Stenmark, Jean Kerr (Ed.). Mathematics Assessment: Myths, Models, Good Questions,
and Practical Suggestions. Reston, VA: National Council of Teachers of
Mathematics, 1991.
Thompson, Denisse R. and Sharon L. Senk. "Assessing Reasoning and Proof in High
School." In Assessment in the Mathematics Classroom, edited by Norman L.
Webb and Arthur F. Coxford, pp. 167-176. Reston, VA: National Council of
Teachers of Mathematics, 1993.
University of Chicago School Mathematics Project. Project Brochure. Chicago, IL:
University of Chicago School Mathematics Project, 1997.
Usiskin, Zalman. "A Personal History of the UCSMP Secondary School Curriculum,
1960-1999." In George M. A. Stanic and Jeremy Kilpatrick (Eds.), A History of
School Mathematics, pp. 673-736. Reston, VA: National Council of Teachers of
Mathematics, 2003.
Usiskin, Zalman. “The UCSMP: Translating 7-12 Mathematics Recommendations into
Reality.” Educational Leadership, 44 (Dec.-Jan., 1986-87): 30-35.
Weiss, Iris R., Michael C. Matti, and P. Sean Smith. Report of the 1993 National Survey
of Science and Mathematics Education. Chapel Hill, NC: Horizon Research, Inc.,
1994.
127
D- 1
Appendix D
Rubrics and Sample Student Responses
D- 2
D- 3
Problem-Solving and Understanding Test (Odd Form): Item 1 and Rubric
Item. a.
Make up a question about a real situation that can be answered by solving
the system
 x + y = 300

 y = 3x
b.
Answer your question.
Rubric
4
The student writes a correct question dealing with a real situation and solves the
question correctly.
3
The student writes a generally correct question dealing with a real situation.
However, there is a minor problem in the statement of the question or in the
completion of the solution.
2
The student writes an appropriate question dealing with a real situation but does
not solve the question or does not solve the question completely and correctly.
OR
The student solves the system correctly and makes some reasonable attempt at
writing a question dealing with a real situation.
1
The student makes some meaningful entry into the problem.
OR
The student solves the system correctly but makes no reasonable attempt at a
question related to a real situation.
0
There is nothing mathematically correct.
D- 4
Item. Here is a list of the winning times in the Olympic 800-meter women’s swimming
free-style race since 1972. Explain how you could use this information to estimate
the winning time in the 1996 Olympics. (You do not have to find an estimate.)
1972
1976
1980
1984
1988
1992
Kena Rothhammer, U. S.
Petra Thuemer, E. Germany
Michelle Ford, Australia
Tiffany Cohen, U. S.
Janet Evans, U. S.
Janet Evans, U. S.
8 minutes, 53.68 seconds
Rubric
2
The student gives a clear description of an appropriate method for finding an
estimate. If an individual followed this method, a reasonable estimate would be
obtained.
1
The student makes a beginning toward a description of a method that could be
used to find an estimate. However, there are not enough details to be able to
obtain a reasonable estimate.
0
The student writes nonsense or does not provide enough information to
understand the method being described.
OR
The student does not provide enough information to know if the method will lead
to an estimate.
The response was also coded with the strategy used.
1
2
3
4
5
6
7
8
9
10
find an average (mean) of the times
find the median of the times
use a graph (line of best fit)
use slope
find the average increase/decrease
mixed methods
look for patterns (increase/decrease with trends)
no response
guess
use range
D- 5
Item. a.
b.
For all numbers x and y, is it true that x2 + y2 = (x + y)2? Yes
No
Imagine that someone does not know the answer to part a. Explain how
you would convince that person that your answer to part a is correct.
Rubric
4
The student provides a correct solution with a good explanation. The student
shows one or more counterexamples or shows (x + y)2 = x2 + 2xy + y2 with the
notion that x2 + 2xy + y2 is not equal to x2 + y2.
3
The student indicates that they would substitute values into the two sides of the
equation but the student doesn’t show the results to indicate that the values make
the two sides unequal.
OR
The student makes some minor error in working out counterexamples.
2
The student begins to show a counterexample but does not bring the work to a
conclusion.
OR
The student shows a counterexample but there is a major conceptual error in the
evaluation.
1
The student responds NO but without an appropriate justification.
OR
The student responds NO and indicates a need for a counterexample but has no
idea how to provide one.
OR
The student answers YES and provides some clear indication of the need to
substitute numbers, providing values for x and y but without proceeding.
OR
The student answers YES and shows (x + y)2 = (x + y)(x + y).
0
The student responds YES.
D- 6
Item. a.
b.
On the axes below, sketch the graph of 3x + 2y < 12.
Is the point (100, -145) on the graph? Yes
Explain how you know.
No
Rubric
Each part was scored separately using a rubric with scores of 0, 1, and 2.
Part A.
2
The student correctly sketches 3x + 2y < 12, shading the appropriate half-pane.
There is no penalty for having a solid line as the boundary.
1
The student plots 3x + 2y = 12 correctly but fails to shade a half plane.
OR
The student makes an error in plotting the line but shades the correct half-plane
for the plotted line.
0
The student has no appropriate graph or there are major difficulties with the plot.
Part B.
2
The student answers YES and provides a valid explanation, probably by
substituting the ordered pair into the inequality.
1
The student answers YES but makes an error in evaluation, although the response
is in the proper direction.
OR
The student answers NO but provides work to show some knowledge toward
evaluation.
0
The student answers YES with no further response.
OR
The student answers NO but provides no meaningful work.
D- 7
Problem-Solving and Understanding Test (Even Form): Item 1 and Rubric
Item. a.
Make up a question about a real situation that can be answered by solving
the equation
5x + 100 = 7x + 75.
Be sure to tell what x represents.
b.
Answer the question you asked in part a.
Rubric
4
The student writes an appropriate question dealing with a real situation and solves
it correctly.
3
The student writes a generally correct question dealing with a real situation.
However, there is a minor problem in the statement of the question or in the
completion of the solution.
2
The student writes an appropriate question dealing with a real situation but does
not solve the question or does not solve the question completely and correctly.
OR
The student solves the equation correctly and makes some reasonable attempt at
writing a question dealing with a real situation.
1
The student makes some meaningful entry into the problem.
OR
The student solves the equation correctly but makes no reasonable attempt at a
question related to a real situation.
0
There is nothing mathematically correct.
D- 8
Item. a.
For all numbers m, x, and y, is it true that m(x + y) = mx + my?
Yes
b.
No
Imagine that someone does not know the answer to part a. Explain how
you would convince that person that your answer to part a is correct.
Rubric
4
The student answers YES and then provides a justification with an example or
possibly with a general explanation of testing with examples.
3
The student answers YES and then appeals to authority, such as stating the
statement is true because of the distributive property (actually naming the
property).
2
The student answers YES and then explains what the property means.
1
The student responds NO but the argument shows some understanding of the
distributive property (examples might have arithmetic errors that lead to a false
statement).
OR
The student responds YES but provides no appropriate justification.
0
The student responds NO with no argument suggesting any understanding of the
distributive property.
D- 9
Item. a.
When an item is on sale at 20% off, you can find the cost of the item by
multiplying its original (non-sale) price by .80.
True
b.
False
If you marked True, explain why this works. If you marked False, show
that the statement is false.
Rubric
4
The student answers TRUE and provides a convincing argument, such as an
example with numbers or some use of properties to get 100% - 20% = 80%.
3
The student answers TRUE and then attempts a convincing argument but makes
some minor error in the example.
OR
The student uses division of the sale price by 80% rather than multiplying the
original price by 80%, but without showing the relationship between the two.
2
The student answers TRUE and then attempts some argument that is partially
correct (e.g., the student only does 20% off the cost without showing 80% or vice
versa).
OR
The student answers TRUE and writes about the process in a very general way.
1
The student answers TRUE with nothing correct for a justification or with no
attempt at a justification.
OR
The student answers FALSE but the attempt at justification suggests some entry
into the problem.
0
The student answers FALSE and there is no glimmer of understanding.
D- 10
Item. a.
b.
On the axes below, sketch the graph of y = x2 – 5.
Give the coordinates of the points where the graph intersects the x-axis.
Show your work or explain how you got your answer.
Rubric
Each part was scored separately using a rubric with scores of 0, 1, and 2.
Part A.
2
The student correctly sketches y = x2 – 5. The y-intercept is correctly plotted at
(0, -5) and the x-intercepts at ( ± 5 , 0), or other points are plotted so that the
graph is reasonably accurate.
1
The student makes a quick sketch with the proper shape and in a reasonable
position but without much care being given to correct values.
OR
The student plots points correctly for only one side of the parabola with work
clear.
0
The student has nothing mathematically correct. The graph has a totally incorrect
shape or there are such errors that it is not clear how the student determined the
graph.
Part B.
2
The student correctly determines the x-intercepts and provides work or
appropriate justification.
1
The student estimates the x-intercepts from the graph, possibly resulting in (±2, 0)
with an explanation.
OR
The student obtains ( ± 5 , 0) with no work.
OR
The student obtains just one intercept.
D- 11
0
The student obtains grossly incorrect values or has nothing mathematically
correct.
OR
The student confuses the x-intercept and the y-intercept.
D- 12
Sample Student Responses to Problem-Solving and Understanding Test (Even
Form): Item 1
Score of 4
D- 13
Score of 3
D- 14
Score of 2
D- 15
Score of 1
D- 16
Score of 0

An Evaluation of the Second Edition of UCSMP Algebra

Transcription

Similar documents

Sonia Kovalevsky Math Day Poster

here

Drawing Math Stories: Linking Language, Art, and

What Did the Baby Porcupine Say When It Backed Into a Cactus?

Counting to Calculus

Mathematical Modeling as a Means to Demonstrate the

NCCTM Central Region Conference - NC Mathematics

Linear Algebra Done Right

The perspectograph - For the Learning of Mathematics

Algebra - Clinical Assessment