A Look Inside: Scoring the GED

Transcription

A Look Inside: Scoring the GED
Language Arts, Writing Essay
March 2011
March,
Marjorie Wine
Associate Director, Assessment Services
Introduction
This presentation will walk you through common questions
related to scoring the GED Language Arts
Arts, Writing Essay
Essay,
including:
– What can we learn about scoring the GED Writing Essay?
– What does “holistic scoring” really mean?
– How do essay readers apply the GED Essay Scoring Rubric?
Do they use it differently than teachers do in their classrooms?
– Is there any one thing that can make a test-taker’s response
ineligible for the full range of scores?
In this presentation:
•
•
•
•
•
•
•
•
•
A glossary of scoring terminology
The Essay Prompt
The Language Arts Writing Test
Responding to the Prompt
Scoring the Essay
The 2002 Series GED Test Essay Scoring Rubric
A deeper look into score points and dimensions of writing
Sample Prompt Alpha
Annotations for responses in the Sample Prompt Alpha
Guide Set
A Look Inside: Scoring the GED Language Arts, Writing Essay
Glossary of Scoring Terminology
To help you understand the following discussion, please
review the scoring terminology as it is used here:
• Prompt
A stimulus or topic designed to generate a written response from examinees.
• Response
Any writing generated by an examinee in response to a prompt.
• Rangefinding
The process by which the range of score points is determined for each individual
prompt. For each prompt, the Writing Advisory Committee reviews roughly 100
responses, comparing each response to a Guide Set (see Training Prompt
Alpha Guide Set) that is created by the GEDTS Language Arts Test Specialists.
Based on the Guide Set, in conjunction with the Rubric, the WAC will define the
range of each scorepoint (e.g., a high 1 vs. a low 2 or a typical 3 vs. a high 3).
Th rangefinding
The
fi di process iis iintended
t d d tto d
determine
t
i T
True S
Scores on each
h
response in the 100-response sample.
• True Score
The consensus score given to a response
response, as determined by the WAC
during rangefinding. If consensus on an individual response cannot be
reached, the paper will be marked “Do Not Use” and will not be used in
reader training.
• Rubric
A scale developed to provide holistic scoring guidelines to rangefinders and
response readers. The rubric should never be used alone to score a
response, however. The rubric can only be used only in conjunction with
the Guide Set.
• Training Papers
Several sets of responses that have already been given True Scores
Scores.
These sets are used for reader training. Readers will have access to all of
their training papers throughout live scoring. They will be encouraged to
compare the responses to the training papers in order to determine the
correct score. Because it is easier to compare a one response to another
response, the readers are likely to have better agreement rates if they use
their training papers to help them score, alongside the rubric, rather than
using the rubric alone
alone.
• Guide Set
A selection,
selection chosen from the previously rangefound responses
responses, of
representative responses that reflect the range of all score points. The
Guide Set will include 4-12 responses, with 1-3 responses per score point,
demonstrating the range of each. Papers in the Guide Set are sometimes
referred to as Anchor Papers, as they “anchor” scorers to the score points
as described in the rubric.
The Essay Prompt: A Definition
The GED Essay Topic is really a
prompt— i.e. it is a group of
prompt
sentences that provides the
examinee with a starting point
for writing a response.
No specialized knowledge is needed to respond. In order to provide
guidance each prompt contains a sentence similar to the following:
guidance,
“In your essay, use your personal observations,
experience, and knowledge to support your essay.”
The Essay Prompt:
g Process
The Vetting
• Presents the test-taker with a
single topic
topic.
• Is based on general knowledge.
g to both
• Is chosen to be interesting
writer and reader.
• Is the same rhetorical type and
format as all other operational
topics—expository.
• Has been field tested.
• Has been analyzed by
psychometricians.
The Essay Prompt & Scoring
A response
p
that shows any
y evidence that the
examinee has read the prompt designated in
his or her answer document will not be
deemed non-scorable and is eligible
g
for the
full range of scores.
The Essay Prompt & Scoring
• Examinees need not answer each and every word of a
promptt in
i order
d ffor their
th i essays tto receive
i scores.
• The point of the prompt is to get examinees writing, not to
assess how well they read the prompt.
– Reading comprehension skills are tested elsewhere (i.e.,in the
Language Arts, Reading Test).
• While the ability to communicate ideas effectively is a key
factor in the score a response receives, effectiveness at
addressing the prompt is not one of the dimensions upon
which they are scored.
Language Arts, Writing Test, Part I
Test-takers must begin the test with the multiple-choice items. They
will have 75 mins
mins. to work through the same processes they will use
in their essays to make corrections, revision, and construction
shifts:
– Surface Editing
– Organizing
– Revising
Language Arts, Writing Test, Part II
After they complete the 75 minutes allotted for the multiple-choice
section the test-takers
section,
test takers are directed to begin Part II
II, The Essay.
Essay
They will have 45 minutes to:
– Plan
– Draft
– Revise
their responses to the prompt.
How Readers Score
Responses to the Prompt
•
Readers acknowledge that an examinee’s response may take into
account multiple definitions of words
words.
•
The reader understands that the writer had only 45 minutes to plan,
d ft and
draft,
d revise
i th
the response.
…Therefore, the test-taker’s response is not a finished product, but
a “good first draft.”
•
Some Considerations for Responding
to the Prompt
•
Occasionally, examinees misread or misinterpret the wording of the
prompt Any response that displays that the examinee is,
prompt.
is in fact
fact,
responding to the prompt designated on his/her answer document
will be considered a valid attempt to respond to the prompt. Such
misinterpretations will not be marked “off-topic.”
off topic.
•
Although some prompts state, “Identify one opinion, problem,
incident etc
incident,
etc.,” a response in which an examinee develops several
opinions will be considered a valid response.
•
Relatedly,
R
l t dl a response in
i which
hi h an examinee
i
responds
d tto only
l one
or two sentences in a three-sentence prompt will also be
considered valid.
Some Considerations for Responding
to the Prompt
However, each
H
h off the
th examinee’s
i
’ points
i t mustt b
be
developed with specific and relevant detail so that the
reader can understand and follow his/her ideas.
What is holistic scoring?
“Holistic Scoring is a method of evaluating writing in which a reader
judges a piece of writing for its overall effectiveness. No one
factor—neither grammar, organization, spelling, etc.—is weighted
more than any other in determining the overall effectiveness of the
piece of writing (Conlan1976).”
The GED Essay Scoring
• Two certified GED® essay readers read and score each
examinee’s
examinee
s response
response.
• The two scores are averaged with the possibility of .5-point
scores (e
(e.g.,
g a response that is scored as a 2 and a 3 by two
readers respectively receives a final score of 2.5).
• Th
The Chief
Chi f R
Reader
d resolves
l
scores on responses receiving
i i
two non-adjacent scores (e.g., one reader gives the response
a 2 and another gives it a 4).
How Readers Score – The Holistic
Approach
•
The GED Testing Service-trained reader reads the response to get an
overall impression of an examinee’s response, not to tally specific errors in
sentence
t
structure,
t t
usage, grammar, and
d mechanics.
h i
•
The examinee must “build the bridges of communication” to the reader
effective word choice, sentence, and paragraphing.
•
The reader’s job is not to “build bridges” to the writer by making inferences
or judgments about the writer’s intent. The reader must score the response
that is in front of him/her.
This means that…
• The reader’s impression is based solely on the examinee’s words on the
page(s).
• The reader reads only the two pages of the answer sheet—no
additions.
The 2002 GED Test Series
Essay Scoring Rubric
Characteristics of Each Score Point
Each score has a one-word descriptor:
1-Score is INADEQUATE
2S
2-Score
iis MARGINAL
3-Score is ADEQUATE
4-Score is EFFECTIVE
…and a brief description of the overall impression responses at each
score-point give:
1-Score: Reader has difficulty identifying or following the writer’s ideas.
2S
2-Score:
R
Reader
d occasionally
i
ll h
has diffi
difficulty
lt understanding
d t di or ffollowing
ll i th
the
writer’s ideas.
3-Score: Reader understands the writer’s ideas.
4-Score: Reader understands and easily follows the writer’s expression of
ideas.
id
*Keep in mind that each score point describes a range of responses and can
represent quite a wide variety of “writer-ly” approaches.
Classroom Scoring vs. Standardized
Scoring
“It doesn’t
have to be an
A+ to get a
A
4!”
Scores
*Approximate
range of
traditional letter
grades
Nonscorable
Invalid response (blank,
off-topic, non-English,
illegible etc.)
illegible,
etc )
1
Not passing
2
D- to C-
3
C to B
4
B+ to A+
*The ranges
g at each GED Essay
y score p
point do not
map to traditional letter grades exactly.
The Five Dimensions
The Scoring Rubric also breaks down each score level
into five dimensions that contribute to the reader’s
reader s overall
impression.
*Readers are trained to ensure that no one dimension is weighted more heavily than
another.
•
•
•
•
•
Response to the Prompt: How well does the examinee establish a main
idea and maintain focus throughout the response?
g
How effectivelyy does the examinee order sentences and group
g p
Organization:
ideas into paragraphs to create a cohesive whole?
Development and Details: Do the details and examples the examinee
chooses to include support idea or further an argument?
Conventions of Edited American English:
g
How well does the examinee
control sentence structure and demonstrate mastery of English language
conventions?
Word Choice: How varied, precise and appropriate are the words the
examinee has chosen to include in the response?
The 4-point Response: Effective
Reader understands and easily follows the writer’s expression of ideas.
Response:
Organization:
Development:
Conventions:
Word Choice:
Presents a clearly focused main idea
that shows evidence of addressing
th prompt.
the
t
Establishes a clear and logical
organization.
Achieves coherent development with
specific and relevant details and
examples.
Consistently controls sentence
structure and the conventions of
Edited American English (EAE).
precise word
Exhibits varied and p
choice.
The 3-point Response: Adequate
Reader understands the writer’s ideas.
Response:
Organization:
Development:
p
Conventions:
Word Choice:
Establishes a main idea that shows
evidence of addressing the prompt.
Uses an identifiable organizational
plan.
Has focused but occasionally
y uneven
development; incorporates some
specific details.
Generally controls sentence structure
and the conventions of EAE.
Exhibits appropriate word choice.
The 2-point Response: Marginal
Reader occasionally has difficulty understanding or following the writer’s ideas.
Response:
Organization:
Development:
Conventions:
Word Choice:
Shows evidence of addressing the prompt
but the focus may shift or deviate from a
central idea.
Shows some evidence of an
organizational plan.
Has some development but lacks
specific details; may be limited to
listing, repetitions, or generalizations.
May demonstrate inconsistent control
of sentence structure and the
conventions of EAE.
Exhibits a narrow range of word choice,
often including inappropriate or vague
selections.
The 1-point Response: Inadequate
Reader has difficulty identifying or following the writer’s ideas.
Response:
Organization:
Development:
examples or
information.
Conventions:
conventions
Word Choice:
Attempts to address the prompt but with
little or no success in establishing a
focus.
Fails to organize ideas.
Demonstrates little or no development;
usually lacks details or
presents irrelevant
May exhibit minimal or no control of
sentence structure and the
of EAE.
g ,g
general and/or
Exhibits vague,
inappropriate words choice
The Guide Set: Sample Topic Alpha
In this section:
• Sample Prompt Alpha
• Examples of test-taker responses written to Sample
Prompt
p Alpha
p
• Scoring annotations about each response
Sample Prompt Alpha
Score point 1
Response 1
Response 1 Annotation
True Score: 1

Although
Alth
h this
thi response demonstrates
d
t t that
th t its
it writer
it read
d the
th prompt,
t it does
d
not sustain focus for its duration. Rather, it jumps from one idea (“…I would
want someone to aknowlege [sic] me…”) to another (“…discrimination is
another…”).

The writing is not sustained long enough to establish much of an
organizational structure or a grouping of ideas.

Included details are minimal and the writing sample contains several
mechanical and spelling errors.

Although
g the writer has included a few appropriately
pp p
y chosen words ((e.g.,
g
“aknowlege [sic],” “discrimination”), most of the response is composed of
common, general words and, again, the writing is not sustained long enough
to display the writer’s command of written English language.
This inadequate response received a score of 1.
Response 2
True Score: 1

This response clearly misinterprets the prompt, but because it demonstrates
that the writer has, in fact, read the prompt and sustains the focus of the
response based on that misinterpretation, it still received a score.

The strength of this response lies in its appropriate grouping of ideas and
well-paragraphed organizational structure.

However, each idea is minimally developed and contains only vague
allusions to examples (e.g., “I’ve
I ve worked as a waitress for a restaurant...”
restaurant... and
“I’ve went [sic] to all kinds of businesses…”).

Although the response contains a few mistakes in conventions, these
mistakes do not impede the reader’s
reader s overall understanding
understanding.

The word choice is repetitive and lacks precision.
This inadequate response received a score of 1.
1
Score point 2
Response 3
True Score: 2

Although this writer does not sustain the response for very long, each
sentence builds logically from the one before it, adding new ideas and
information as it proceeds in order to make a brief but coherent argument.

It contains both an introductory sentence and a summative concluding
sentence that provide a skeletal organizational structure.

Although it contains a few mechanical errors, they do not impede meaning.

The word choice is appropriate and occasionally quite precise (e.g.,
outragous [sic]”
[sic] and “appreciative”)
appreciative ), the sample is not sustained long
“outragous
enough to demonstrate the writer’s fluency.
This marginal response received a score of 2.
Response 4
True Score: 2
 Thi
This response sustains
t i its
it main
i idea
id for
f its
it duration
d ti with
ith little
littl or no
irrelevant information.
 Each sentence brings in new information, building upon the idea(s)
expressed in the last. The grouping of ideas is mostly appropriate.
 Although this response contains some errors in grammatical
conventions they are not so severe as to prevent the reader from
conventions,
understanding the writer’s ideas.
 This response contains a mixture of general, repetitive terminology
(e.g.,“…better communication leads to bigger and better things.”)
and more specific word choice (e.g., “…dental insurance its [sic] a
win win situation…”).
Response 5
True Score: 2

In this response,
response we see a clear thesis statement
statement. The writer maintains acute
focus on this thesis for the duration of the essay, thereby showing elements
of higher-scoring responses.

The response makes good,
good if rigid
rigid, use of the “five
five-paragraph
paragraph essay”
essay format
in order to organize ideas into paragraphs.

Although each paragraph develops unique ideas with relevant details, they
are communicated using a mixture of general and specific language
language, with
little variance in sentence structure (e.g., “This is…,” “That is…” of “Those
are…” begin 8 different sentences).

There are relati
relatively
el fe
few errors in spelling or mechanics thro
throughout
gho t this
response, but it lacks the precision necessary for a higher score.
Score point 3
Response 6
True Score: 3

One strength of this response is its clarity and sustained train of thought
thought. It does
not deviate from its central premise at all.

Another strength is the correct application and precision of its word choice (e.g.,
“ it l tto a positive
“vital
iti working
ki environment,”
i
t ” “i
“irritable
it bl + d
defiant,”
fi t ” ““newfound
f
d gratitude
tit d
and respect,” etc.), particularly in light of a small number of errors in conventions
that we see here.

However, although the details included are relevant, there is only one
substantively developed point (e.g., paragraph 2). This writer’s skill in
elaboration is sufficient, but not advanced.

Relatedly, even though we can see a clear introduction, body paragraph and
conclusion, this response lacks enough development for this organizational
structure to be employed to its full benefit.
This adequate response received a score of 3.
Response 7 (page 1)
Response 7 (page 2)
True Score: 3

This response is well organized with an introduction, a conclusion and two
body paragraphs that each include some detail and elaboration upon each
of the writer’s two main ideas.

Although most of the information included maintains focus on the
response’s thesis statement (“One of these benefits that I would like to
recieve[sic] from an employer would be health insurance.”), it expresses its
id
ideas
iin a mixture
i t
off specific
ifi ((e.g., ““…spared
d th
the struggles
t
l of…”)
f ”) and
d more
general, repetitive language (e.g., “…to keep a good relationship between
employer…”).

The grammar usage and spelling are generally correct throughout the
response.
This adequate response received a score of 3.
3
Response 8 (page 1)
Response 8 (page 2)
True Score: 3

This response brings in four key, focused ideas and organizes them effectively
in service to its one main idea: “…there is more to a job than just the salary they
receive.”

Although each of the four body paragraphs contain some elaboration supporting
the paragraphs’ main ideas, the elaboration is general and does not contain
specific examples or details.

Similarly, the word choice throughout this response is clear and easily
understandable but is occasionally repetitive and lacks some precision.

Any usage or spelling errors are negligible and do not impede understanding.
This adequate response received a score of 3
3.
Score point 4
Response 9 (page 1)
Response 9 (page 2)
True Score: 4

The organizational
Th
i ti
l strategy
t t
off thi
this response uses the
th five-paragraph-essay
fi
h
f
format
t to
t
good effect. Each body paragraph contains well-grouped ideas that progress from
sentence to sentence, and the introductory and concluding paragraphs create an
effective frame to the response.

Each sentence brings in new information that builds on the sentence previous. Each
paragraph contains adequate elaboration and some specific detail.

Although the language is not consistently exact throughout the response, it contains
some specific and effective phrasings (e.g., “not likely a computer will ever deliver a
baby,” “excellent incentive,” etc.).

Also, sentence structure is varied throughout.

Grammatical errors are few and do not hinder understanding.
This effective response received a score of 4.
Response 10 (page 1)
Response 10 (page 2)
True Score: 4

This response maintains clear focus on its central message and awareness of its
audience
di
for
f its
it duration,
d ti
creating
ti a compelling
lli argument.
t

Also, this response provides a strong example of how a writer can deviate from
the five-paragraph-essay format without compromising organizational structure
structure.
Rather, this writer’s effective argument brings in new details and ideas with every
sentence in order to build to its central point.

Fluency of language is another strength of this response. Although the chosen
vocabulary isn’t “flashy,” it is correctly applied and mostly precise. The sentences
are fluid and lead logically into each other.

Errors in conventions are negligible and do not impair the reader’s understanding.
This effective response received a score of 4.
Conclusion and Contact Information
Conclusion and Contact Information
• Discussion and Questions
• Contact for further information:
–
–
–
–
–
Marjorie Wine
Associate Director, Assessment Services
marjorie wine@ace nche edu
[email protected]
Office: (202)939-9474
Mobile: (202)870-1469

A Look Inside: Scoring the GED

Transcription

Similar documents

Katie Browne - School of Humanities and Social Sciences

are factory farms hog-tying people`s

Dr. Jeanne Tunks, Director of the Core Barbara Stone, Graduate

Adult Education Program - Henderson Community College

SAMPLE GED PRACTICE TEST All sample GED Questions taken

Going Further 4 PERSONALITY TYPE

HSPA Example - www.ramsey.k12.nj.us.

ON`S Industry Germany - ON`S Industry Germany

America the Beautiful Essay Contest

MFCU Scholarship Application - Marine Federal Credit Union