Document 6539630

Performance On The 1973
Revised
Work
Philadelphia JEVS
Sample Battery
DON
FLENNIKEN
previous study by Flenniken (1974) descri, bed a hi-modal distribution of tinge and quality scores on the Philadelphia JEVS work sample
battery. Questions were raised at that time concerning the validity o1•
the Philadelphia JEVS work sample battery as an evaluation tool for a
disabled population.
In an earlier paper, Plax (1972) reported a highly skewed distribution of scores in his study and evaluation of the Philadelphia JEVS work
sample battery. Plax pointed out that:
A
hi.ghly
skewed distribution o1• scores was particularly
unsettling s•nce one of the main strengths o1• the sampling
approach was thought to be its effectiveness in discriminatin G
the employableclient from the unemployable. Further investigation of the method used-to develop the Philadelphia JEVS
The first
norms suggest two possible bases for this inadequacy.
is that the Philadelphia JEVS norms were developed on preyoung, black, disadvan.taged clients and may not be
applicable to a wider range .of St. Louis clients.
The second, more serious problem, is that since the Philadelphia JEVS norms were not developed on an industrial population or usi.ng industrial standards, their relationship to
employability in industry has not been su.bstantiated and may
be nonexistant. (p.2)
It should be noted that these conclusions were reached .prior to the
1973 revision of the Philadelphia JEVS work sample battery. According
(1973), this revision,
to the Jewish Employment and Vocational Service
in part, establishes norms based on:
(3) point scale, developed from a norming study
a three
enrollees. The emon the performance ot• over two hundred
pirically derived distributions were examined and divided into
This
domir•antly
Don Ftenniken is the Director of Rehabilitation Services at Goodwill Industries
in Zanesville, Ohio. He was previously employed as a Vocational Evaluator and as
the Supervisor of Vocational Evaluation at the Zanesvtlle Goodwill Industries. Mr.
Flenniken received a bachelor's degree from Marietta College in 1962 and a master's
degree from the University of Northern Iowa in 1971.
VOLUME
8, NUMBER 4, DECEMBER,
1975
three sections
corresponding
to
the
following percentile
rank
ranges:
PERC.F_.NT•LE RANKS
Sco•
99-61
60-40
3
2
39-1
1
A rating
performance
of 3 for performance indicates that an enrollee's
falls into the interval between 61 and 99 percent
of all those who took .the sample.
A rating of 2 for performance indicates that the enrollee's
performance falls into the interval between 40 and 60 percent
of all .those who took the sample.
A rating of 1 for performance indicates that the enrollee's
performance falls into the interval between 1 and 39 percent
of all those who took the sample.
This new scoring system supersedes the previous one based
(p.v.).
on a f-point scale
The purpose of this study was, therefore, to describe the performance
of disabled referrals on the Philadelphia JEVS work sample battery
using the revised format. Since this is an initial investigation of the
revised JEVS battery, no attempt was made to formally compare the
distribution of scores obtained on a disabled population with the revised
JEVS norms. Rather, this study focused primarily on a comparison of
the performance of five disability groups oft the JEVS work sample
ba.ttery.
METHOD
Subjects
provide a more accura-te evaluation of client performance on the
JEVS work: samples, the total sample of 195 clients was divided into
five major groups according to the nature of their disability. The groupTo
ings
those found on the standard referral forms received from the
Ohio Bureau of Vocational Rehabilitation and the Ohio Bureau of
•;ervices for .the Blind. They were as follows: (1) Group A: Menta.lly
l•etarded, (2) Group B: Emotional Problems, (3) Group C: Behavioral
Problems, (4) Group D: Epileptic Disorders, and (5) Group E: Other
Medical Disorders. Many of the referrals were reported as having multiple disabilities and, as such, had their performances included in more
than one group. For example, a referral with epilepsy might also be
listed as •being mentally retarded and appear in the resul•ts of both
Group D and Group A. Thus, multiple disabilities accounted for the
total sum of the group subects being 251 rather than the actual number
of. 195 clients. The number of cases in each disa'bility category are shown
by sex in Table 1.
•6
were
VOCATIONAL
EVALIJATION
AND
WORK
ADJUSTMENT BULLETIN
Total
TABLE 1
of Clients in Each Group,
Number
Group
A
Mentally
•tetarded
Group
Mentally
A
Retarded
Group
Emotional
B
Problems
Group
Behavior
C
Problems
Group D
Epileptic
Group E
Other
Medical*
Total
*Exclusive
of
arm,
Group
Group C
Behavior
Problems
B
Emotional
Problems
29 Male
19 Female
6 Male
3 Female
6 Male
3 Female
14 Male
10 Female
According
]
I
6 Male
6 Female
Male
4
2 Female
Group
to Sex
D
Epl/eptie
1 Male
Group
•
Other
Medle•J*
3 Female
9 Male
5 Female
0 Male
0 Female
0 Male
1 Female
2 Female
9 Male
5 Female
/ 1 Male
1 .0 Female
I Male
3 Female
0 Male
0 Female
1 Male
0 Female
6 Male
5 Female
2 Male
9 Male
5 Female
0 Male
1 Female
3 Male
1 Female
2 Male
$ Female
30 Male
12 Female
51 Male
36 Female
24 Male
16 Female
23 Male
14 Female
10 Male
11 Female
6 Male
6 Female
hand.
or
4 Male
• Male
1 Female
$ Female
44 Male
22 Female
finger disabilities.
Analysis
according to
of Table 1 shows the total number of clients in each group,
Group A (Men-tally l•etarded) contains the greatest
sex.
number of clients, while Group D (Epileptics) contains the least number
of clients. Males outnumbered females in all groups combined by about
2 to 1. Individuals groups, except for Group D (Epileptics), showed
nearly this same male/female ratio.
The division of the referrals into the five groups was done in an
arbi-trary manner and is considered a convenient classification. I.t is
felt, however, that such a division may provide a better insight into
client performance than simply combining the performance of all clients
into one group.
PROCEDURE
All work
samples
Sample
were
administered and scored as .outlined in the
Handbook (Jewish Employment and Voca-
Evaluator's
tional Service, 1973). Time and quality rating scores were kept separated
by disability and sex. Data was collected from mid-July, 1973 through
mid-October, 1974. Work sample numbers that appear in this s.tudy are
those assigned by the Philadelphia JEVS for the work sample battery.
JEVS has assigned Work Sample #80 to the blouse-making work sa.mple
for female evaluation and Work Sample #80a to the vest making work
sample for male evaluation; however, since male and female time and
quality rating scores were kept separate in this study, Work Samples
Work
VOLUME
8, NUMBER
4, DECEMBER,
1975
37
Listing
Work
Sample
TABLE 2
of Philadelphia JEVS Work Samples
Number and Title
l•umber
Sample
Nut, Bolt, and Washer
Rubber Stamping
Work
1
2
3
4
Assembly
Grommet Assembly
Union Assembly
Belt Assembly
Ladder Assembly
Metal Square Fabrication
Hardware Assembly
34
35
36
4O
41
50
51
Telephone Assembly
Lock Assembly
Filing by Numbers
Proofreading
Filing by Letters
52
Adding
53
54
Payroll Computation
Computing Postage
60
Resistor
Nail
70
and
Title
Budgette Assembly
Sign Making
Tile Sorting
Nut Packing
Collating Leather Samples
I2
20
30
31
32
80
to
Washer Threading
5
10
11
#80
According
(•a)
#80a
have
been
combined
and
Screw
(Soldering)
Sort
Machine
Reading
Pipe Assembly
Blouse]Vest Making
Condensing Principle
under a single number
(#80--
listing of the Philaa complete
and work sample title.
Not all subjects were administered .the entire work sample battery.
Of the 27 work samples in the JEVS bat.tery (W/S #80 and #80a being
combined), an average of 16.97 work samples were administered per
Blouse/Vest Making). Table 2 presents
delphia JEVS work samples by number
those subjects who possessed sufficient reading and physical abilities and who performed adequately on the more elementary
tasks were admir•istered the more complex tasks. The results of this
selectivity is reflected in Table 3, showing the actual number of work
samples completed by each of the five groups in this study.
It should be noted that many of the work samples contain a sampling
However,
too small to be considered valid for most predictive purposes.
enough data is available to describe the performance of certain disability
groups on the revised format of the JEVS battery when analyzed with
respect to time and quality factors.
subject. Only
VOCATIONAL
EVALUATION
AND
WORK
ADIUSTMENT
BULLETIN
The
Number
of
According
No.
Subjects
to
Sex
TABLE 3
that Completed Each Work Sample
Within Each Disability Group
'X'ttle
Group
Group
36
40
4.1
50
Assembly
Stamping
Threading
Budgette Assembly
Sign Making
Tile Sorting
Nut Packing
Collating Leather Samples
Grommet Assembly
Union Assembly
Belt Assembly
Ladder Assembly
Metal Square Fabrication
Hardware Assembly
Telephone Assembly
Lock Assembly
Filing by Numbers
Proofreading
Filing by Letters
51
Nail
52
53
54
60
Adding Machine
Payroll Computation
Computing Postage
Resistor Reading
Pipe Assembly
Blouse/Vest Making
Condensing Principle
I
2
3
4
5
10
11
12
20
30
31
32
33
34
35
70
80
90
Nut, Bolt, Washer
Rubber
Washer
and
Screw
Sort
Group
Group
Group
M
F
M
F
bf
F
M
F
50
45
50
49
32
35
14
I4
14
14
21
19
19
8
10
8
51
15
18
22
11
12
12
lI
12
13
13
13
12
33
20
20
19
I9
23
13"
51
35
33
35
31
31
24
24
23
15
22
15
22
15
24
.23
14
15
22
22
22
12
12
12
30
0
0
31
•
24
19
2
24
9
15
0
0
15
3
2
•0
22
7
20
17
2
15
51
51
51
51
49
30
1
49
11
5
36
19
•3
•0
22
18
17
7
35
39
10
44
19
13
9
44
13
8
10
9
9
44
44
44
44
20
19
20
17
19
8
0
1
44
31
4
41
21
13
12
12
13
13
13
12
13
0
9
22
13
I0
3
3
2
3
10
5
I0
19
17
9'
12
12
13
10
3
8
13
9
23
17
20
11
21
21
28
2
30
24
2
13
18
5
4
0
8
13
14
2
2
23
3
12
1
1
0
3
0
1
1
0
0
15
41
2
12
18
4
44
1
9
2
1
8
9
3
6
6
11
12
16
18
6
8
19
12
11
5
10
7
1
2
1
1
3
3
3
6
2
1
2
2
0
1
0
0
0
6
38
32
16
32
15
18
0
2
18
15
3
18
18
16
18
26
•6
18
6
11
19
26
15
6
2
12
4
0
19
RESULTS
Many of the subjects completing the respective work samples did
not perform in a successful manner as estabilshed by the Philadelphia
JEVS time and quality norms. A "successful performance" on the JEVS
work samples results from a client receiving either a 2 or 3 rating in
both time and quality as established by-the JEVS norms. The results
of quality will be examined first.
Table 4 shows the number (N) and percentage (%) of subjects
completing each of the respective work samples in a successful manner
for quality as compared to the total number of subjects administered
the work samples. The results are separated by disability group and sex.
VOLUME
8, NU•IBER 4, DECEMBER,
1975
39
VOCATIONAL
EVALUATION
AND
WORK
ADJUSTMENT
BULI...ETIN
Group A (Mental Retardation), only .two work samples were
completed successfully by 75•o or more of the Subjects and seven different work samples were successfully completed by 50% or more of the
subjects. In Group B (Emotional Problems), six dif£erent work samples
successfully completed by 75% or more of the subjects, while
were
twenty-one work samples were successfully completed by 50% or more
of the subjects. In Group C (Behavioral Problems), six differen.t work
samples were successfully completed by 75•o or more of the subjects,
while twenty-three differen-t work samples were successfully completed by
50C•o or more of the subjects. In Group D (Epileptics) eight different
work samples were successfully completed by 75C•o or more of the subjec-ts, while seventeen different work samples were successfully completed
by 50% or more of the su,bjects. In Group E (Medical Disorders)•
thirteen di.fferen.t work samples were successfully completed by 75•o or
samples were sucmore of the subjects, while twenty-five different work
cessfully completed by 50% or more of the subjects.
In general, the performance of males was better than that of females
in Group A (Mental Retardation), Group B (E.motional Problems), and
Group C (Behavioral Problems). On the other hand, ,the per£ormance
of females in Group D (Epileptic Disorders), and Group E (Other Medical Disorders), was better than that of males.
The best overall performance for quali.ty on .the entire JEVS work
sample battery was earned by the subjects in Group E (Medical Disorders), respectively followed by Group C (Behavioral Problems), Group
B (Emotional Problems), Group D (Epileptic), and Group A (Men.tal
Retardation).
With respect to the time variable, Table 5 shows the number (N)
and percentage (%) of subjects completing each of the respective work
sax•aples within the tinge allotment necessary to meet successful performThese results are separated by disability
ance on the JEVS work samples.
In
group
and
sex.
Group A (Mental l•etardation), three work samples were completed successfully by 75C7o or more of .the subjects and ten work samples
completed successfully by 50•o or more of the subjects. In Group
were
B (Emotional Problems), and Group C (Behavioral Problems), nine were
st•ccessfully completed by 75% or more of the subjects, while nineteen
clitferent work samples were successfully completed by 50% or more o£
tl•e subjects. In Grot•p D (Epileptic), none of .the work samples were
completed successfully by 75•o or more of the subjects, but five different
work samples were completed successfully by 50% or more of the subjects. In Group E (Medical Disorders) ten different work samples were
completed successfully by 75% or more of the subjects, while twenty-two
In
VOLUME
8, NL:MBER 4, DECEMBER,
1975
41
42
VOCATIONAL
EVALUATION
AND
WORK
ADJUSTIVIENT
BULLETIN
different work
samples
were
successfully completed by 50•7o
or
more
of
subjects.
In general, a greater percentage of males than females received sarisfactory time ratings in Group A (Mental Retardation), Group B (Emotional Problems), and Group E (Other Medical-Disorders). The rime
rating percentages were about equal for males and females in Group C
(Behavioral Problems), and Group D (Epileptic Disorders).
The best overall performance for success in time was earned by
Group E (Medical Disorders), and respectively foil-owed by Group C
(Behavioral Problems), Group B (Emotional Problems), Group A
(Mental Retardation), and Group D (Epileptic Disorders).
As indica-ted above, successful performance on the JEVS System is
based upon the ratings received for both ti-me and quality on each work
sample. The results of these two combined factors are presented in Table
6, separated by disability group and sex.
In Group A (Mental Retardation), none o£ the work samples were
completed successfully in both time and quality by 50Wo or more of the
subjects. Only three work samples were completed successt:ully by 25¢•o
(all of these subjects were male). In Grou.p B
or more of the subjects
(Emotional Problems), none of the work samples were completed successfully, for both time and quality by 75% or more o£ .the subjects, and
only one work sample was completed successfully by 50• or .more o[
the subjects. This work sample was completed by a male subject.
Eighteen work samples were successfully completed by 25¢•o or more of
the males, while eight work samples were successfully completed by 25•o
of the females. In Group C (Behavioral Problems), only one
or more
work sample was successfully completed in both time and quality by
75% or more of the subjects (a male), and only one additional work
sample was so completed by 50% or more of the subjects (a female).
Seventeen work samples were successfully completed by 25•o or more
of the male subjects, whiIe ten work samples were successfully completed
by 25% or more of the female subjects. In Group D (Epileptic), none
of the work samples were successfully completed, for both time and
quality by 50% or more of either male or female subjec-ts. Only four.
work samples were successfully completed by 25°/o or more of the sub-jects, all of whom were male. In Group E (Medical Disorders), none of
the work samples were successfully completed for both time and quality
by 75% or more of the subjects. Five work samples were successfully
completed by 50•o or more of the male subjects, and six by 50•o or more
of the female subjects. Fifteen work samples were successfully completed
by 25% or more of both the male and the female subjects.
Except for Group E (Other Medical Disorders), the con•bined time
and quality performance of males on the JEVS work samples was conthe
VOLUME
8, NUMBER
4, DECEMBER,
1975
43
44
VOCATIONAL
EVALUATION
AND
WORK
ADJUSTMENT
BULLETIN
sistently superior
that of females. The best overall percentage of
time and quality standards was earned by Group E
success in combined
(Medical Disorders), and respectively followed by Group C (Behavioral
Problems), Group B (Emotional Problems), Group A (Mental Retardation), and Group D (Epileptic Disorders).
to
DISCUSSION
Further
examina-tion
of Table
6
shows
that
Disability Groups
B
(Emotional Problems), C (Behavior Problems), and E (Medical Disorders), achieved a greater degree of successful performance than did
subjects in Group A (Mental Retardation) and Group D (Epileptics).
However, exceptions did occur. For example, in work sample number
2 (Rubber Stamping), female subjects in Group A (Mental Retardation),
did better than female subjects in Group B (Emotional Problems), while
this same work sample, male subjects in both Group A (Mental
on
Retardation), and Group D (Epileptics), did better than female subjec.ts
in Groups B and C (Emotional and Behavioral Problems), respectively.
In work sample number 4 (Budgette Assembly) male and female subjects
in Group A (Mental Retardation), did better than male subjects in
Group B (Emotional Problems) and Group C (Behavioral Problems).
In work sample number 30 (Union Assembly), male subjects in Group
A (Mental Retardation), and Group D
(Epileptics), did better than
males in Group B (Emotional Problems). In work sample number 36
(Lock Assembly), male subjects in Group D (Epileptics), did better than
males in Group B (E-motional Problems), while in work sample number
50 (Filing by Letters), female subjects in Group D (Epileptics), did
better than females in Group B (Emotional Problems). These are just a
few
of the many
samples,
exceptions
noted which show .that, on specific work
mental retardation and epilepsy performed
individuals with
better than those from other disabili.ty groups.
With respect to the variable of sex, male
subjects did better than
female subjects in seventy-nine instances (excluding work sample number
32--Ladder Assembly--as it was administered only to male subjects),
while female subjects did better than male subjects in thirty-three inThese results are indicative that-some work samples may be
stances.
favorable to one sex than the other. Those work samples that
more
appear to be more favorable to female subjects include #4: Budgette
Assembly, #41:. Proofreading, #52: Adding Machine, #60: Resistor
Reading, and, to a lesser degree, #35: Telephone Assembly, #-53: Payroll Computation, and #80: Blouse/Vest Mraking. Those work samples
that appear to be more favorable to male subjects include #1: Nut,
Bolt, and Washer Assembly, #5: Sign Making, #20: Grommet Assembly,
VOLUME
8, NUMBER
4, DECEMBER,
1975
45
Assembly, #70: Pipe Assembly, and,
Rubber Stamping, .#3: Washer Threading, #31:
Lock Assembly, #40: Filing by Numbers, and #51
#30:
Union
SUM•VIARY
AND
to
a
degree, #2:
Assembly, #36:
lesser
Belt
Nail and Screw Sort.
CONCLUSIONS
The primary function on this study was to analyze the results of
time and quality performance of 195 subjects from a disabled population
JEVS Work Sample Battery. Subjects
on the 1973 revised Philadelphia
divided into five disability groups that included: (1) mental rewere
tardation, (2) emotional problems, (3) behavioral problems, (4) epilepsy,
and (5) other medical disorders. The results of -time and quality per-
formance
were
compared
the established JEVS norms (1973 refor each of the five disability groups, as well
with
vision) and kept separate
for male and female subjects.
The results indicate that subjects in the disability groups labeled
emotional problems, behavioral problems, and other medical disorders
achieved a greater degree of success, on the JEVS work sample battery
than did subjects in the_ disability groups labeled mental retardation and
epilepsy. Addi.tionally, male subjects performed much better than female
subjects on .the work sample battery as a whole.
This investigation has led to some conclusions that might be of use
in future studies of the JEVS System when applied with a disabled population. The following two conclusions deserve notation:
(1) Subjects
with mental retardation and epileptic disabilkies should not be expected
to meet successful performance standards with any degree of consistency
in either ti.me or quali.ty when compared with the established Philadelphia JEVS norms. (2) Female subjects can be expected to do less
satisfactorily than male subjects with the same disability on many-of
the same work samples. Exceptions to this are noted and it is, .therefore,
concluded that some of the JEVS work samples may favor one sex over
the other.
This study also raises questions concerning the appropriateness of
the norms that accompany .the Philadelphia JEVS Work Sample Battery,
in its revised format, when applied to certain disabled populations. Can
these norms be used to predict success in employment for subjects considered mentally retarded or those having epilepsy? Should separate
to females?
norms be established [or use in evaluating males as opposed
Is the work sample battery, as a whole, or in part, male or female
as,
discriminatory?
The
the
46
answers
performance
to
these
of other
VOCATIONAL
questions
disabled
EVALUATION
must
await
subjects.
AND
further
Follow-up
WORK
research
studies
on
into
the
ADJUSTMENT BULLETIN
degree of success in employment as a result
the Philadelphia JEVS Work Sample Battery
validation of the JEVS System.
of
predictions
seem
essential
made
to
the
from
true
I•EFERENCES
Time-quality performance of Goodwill clients evaluated by the JEvs
Work Sample Battery.
Vocational Evaluation and
Work Adjustraent Bulletin,
1974, 7 (3),
Jewish Employment and Vocational Service, Work Sample Evaluator's Handbook,
(Manpower Administration Contract No. 99-2-0419-000). Philadelphia, Pa.: Jewish
Employment and Vocational Service, August, 1973.
Plax, K. (Ed.). An evaluation of the Philadelphia JEVS Work Sample Battery.
search Bulletin, St. Louis Jewish Employment and Vocational Service, August, 1972.
Flenniken, D.
Winter Man. Child
JOHN
Then
as
Dripped
H.
PAUL
before .the neon lights
in night haze
an attitude
walking
Street sounds reverbera.ted,
The endless uselessness
being
Finger-printed
Faceless option
window
to
thundering
mirrors
discipline
asking
That shattered, battered, crippled heart
Brutalized by shadow voice and form
realizing
technology is i.tself
Your personal halloween mask hiding
feeling
From what he is wanting to be
Not knowing .the parade is yesterday
Be walking the lines on pidgeon pavements
Touching tomorrow with tears
That
your
Children's Center, a
John H. Paul is a Vocational •.valuator at the Maryland
correctional facility of the Department of Juvenile Services in Baltimore, Maryland.
This poem is an attempt to describe the inner experience of the universal ]uvenlle
client at the Maryland Children's Center.
VOLUME
8, NUI•IBER 4, DECEMBER,
1975
47