THE RELATIONSHIP OF GRE GENERAL-TEST

THE RELATIONSHIP OF GRE GENERAL-TEST SCORES
TO FIRST-YEAR GRADES FOR FOREIGN GRADUATESTUDENTS:
REPORT OF A COOPERATIVE STUDY
Kenneth M. Wilson
GRE Board Professional
ETS Research
Report
Report
GREB No. 82-1lP
86-44
December 1986
This report presents the findings
of a
research project
funded by and carried
out under the auspices of the Graduate
Record Examinations
Board.
0
ES
EDUCATIONAL
^
v
. .
.
.
.
.*
‘___A.
._.^*
_
TESTING
Q
SERVICE, PRINCETON,
NJ
The Relationship
of GRE General Test Scores to First-Year
Report of a Cooperative
for Foreign Graduate Students:
Grades
Study
Kenneth M. Wilson
GRE Board Professional
Report
GREB No. 82-11
December 1986
Copyright,
@
1986 by Educational
Testing
Service.
All
rights
reserved.
Acknowledgmnts
This study wasmade possible by the support and encouragement
of the
Graduate Record Examinations Board and the cooperation of participating
graduate schoolsand departments.
Richard Harrison helped plan for data collection
Ka Ling Ghanhelped in data analysis.
and file
developrvsnt.
HenryBraun, NancyBurton, and Charles Seckolskymadehelpful reviews of
the manuscript.
Thesecontrilxtions are gratefully acknowledged.The writer, however, is
solely responsible for the contents of this report.
i
Abstract
The present studywasdesigned as a GEUZ
population validity study for
foreign students enrolled in U.S. graduateschools. Primary interest ms in
validity for students for whomEnglish is a secondlanguage(foreign ESL students). The objectives of the studywere to obtain information regarding the
typical within-departmentrelationship of scores on the QREGeneral Test to
first-year averagegrades (FYA) (a) in samplesof foreign ESLstudents that
are heterogeneous
with respect to linguistic-cultural-educational background,
(b) in subgroups that are homgeneouswith respect to country of origin and
associated background
variables, and (c) in subgroupsclassified according to
relative level of English proficiency, as masured by scores on the Test of
mglish as a ForeignLanguage(IOEFL), GEU3
verbal and analytical performance
relative to quantitative performance, and self-reported English language
conmunicationstatus.
The study wasbasedon data for a total
of 1,353 foreign ESLstudents and
42 foreign DTL (English-native-language)students from 97 graduate departnmts
in 23 graduate schools. Eighty-six departmnts were primarily quantitative-
either engineering, math/science,or economics;six were bioscience departmants; and five wre primarily verbal-four education and one political
science. More than 90 different countries were representedin the Sample. The
The three largest national contingents
majority of students were from Asia.
(students from India, Taiwan,and Korea) accountedfor about one half of all
students. Studentswere highly selected both in terms of quantitative ability
and in terms of English proficiency as measured
by TOEFL.
In order to
The departnmt-level samples were small (mdian N = 12).
obtain reliable estimates of within-departmmtGRE/Fw relationships, data
for similar departmentswere pooled. GREand Fw variables were z-scaled
by departmnt before pooling-that is, scores were expressed as deviations
frcundepamnt-level means in department standarddeviation units. Primary
emphasiswas given to analyses basedon pooleddata for the primarily quantitative departmnts, because representation of departmnts from the other
fields was limited.
Averagelevels and patterns of GRE/FYA
correlations basedon pooled data
for foreign ESLstudentsfrom the 86 quantitative departments and the five
"verbal" departmnts were foundto be comparableto levels and patterns of
coefficients that have beenreported by the GREValidity Study Senrice (VSS)
for a samplecomposed
primarily of U.S. citizens; results for the small sample
of bioscience departmentswere anmalous, due probably to sampling effects. In
quantitative fields, quantitative and analytical scoreswere the strongest
predictors; in the verbal fields, verbal scoreswre strongest. GREverbal and
TOEF'L
total scoreshad parallel patterns of validity.
Study findings suggested that inferences basedon GEUZ
scores regarding
the subsequentacademic performanceof applicants, especially those applying
for admissionto primarily quantitative departments,are likely to be as valid
for foreign ESLapplicants as for U.S. applicants. Questionsregarding the
comparativeacademicperformance
of U.S. and foreign students with comparable
GREscores, not addresseddirectly in this study, call for further research.
iii
Table of Contents
Acknowledgmnts
..........................
Abstract .............................
sumnary
i
iii
S-l
..............................
Section I:
Background.....................
1
ThePresentStudy ..................
3
Distribution of Studentsby Schooland Department.
.
National Origin and English-LanguageBackground
ofStudentsintheSample..............
Section II:
7
Description of StudyVariables and SamplePerformance
0ntheVariables . . . . . . . . . . . . . . . . . .
11
Variables Related to English Proficiency
Performnce on the Study Variables
Section III:
4
.
. . . . .
12
.
14
Intercorrelations of Selected Variables in the Total
ESLSample . . . . . . . . . . . . . . . . . . . .
18
StudyMethodsandProcedures.............
21
GeneralMeth~ological Rationale
.........
21
Application in the Present Study
.........
23
. . . . . . . .
GRE/FYA
Relationships for Foreign ESLStudents,
byAcademic=ea ...................
27
Interpretive Perspective
. . . .
27
QiEFYAValidity Coefficients for Foreign ESL
Samples . . . . . . . . . . . . . . . . . . . . .
28
Multiple RegressionResults for Foreign ESLStudents
byAcademicA.rea . . . . . . . . . . . . . . . . .
30
Validity of a StandardGREPredictive Composite
forQuantitativeDepartments . . . . . . . . . .
.
32
SimpleCorrelation of Selected Non-GRE
Variables
WithFYA . . . . . . . . . . . . . . . . . . . . .
32
Differing in
Section V: Zgre/Zfya Relationships for Subgroups
Performanceon Selected mglish-Proficiency-Related
Measures-PooledZ-scaled Data for Quantitative
Departmnts....................
37
Section IV:
.
V
. .
. .
. . . .
Section VI:
Procedures Involved in Defining Subgroups. . . . .
31
Results......................
38
Coqxxative
40
Performance of Subgroups . . . . . . .
Correlation of a Standard z(gre) Predictive Composite
with Z(fya) for Foreign ESL Students Classified by
National Origin and Graduate Major Area-Data for
QuantitativeDepartmznts
. . . . . . . . . . . . .
Section VII:
Exploratory Analysis of the Comparative Performance
of Regional Subgroupson FyA and (;REVariablesStudents from All Quantitative Departmnts . . . .
45
An
Findings
. . . .
. . . .
.
.
.
. . .
. . . .
. . . .
AppendixASumaryofDepartm?nt-Level
47
. . . . . . . . . . . . .
Observed versus Predicted Criterion
RegionalGroups..................
References
41
Standing of
. . . . . . . . . . . . .
Data
. . . . . . . . . . .
49
.
53
55
List of Tables and Figures
Table 1
Table 2
Table 3
Table 4
Table 5
Table 6
Table 7
Table 8
Table 9
Table 10
Table 11
Distribution of Foreign ESL (English SecondLanguage)
StudentsbySchoolandDepartmnt...........
5
Distribution of 97 Study Departmentsby Size of ESL
Sample . . . . . . . . . . . . . . . . . . . . . . . .
6
Distribution of Studentslq Countryof Citizenship and
Academic
Area for 39 Countries F&presentedby Five or
MoreStudents. . . . . . . . . . . . . . . . . . . . .
8
Distribution of Foreign ESLStudentsby Cuuntryof
Citizenship within "Regional" Classifications Basedon
Geographicand ~glish-Background-RelatedVariables . .
10
Sumary Statistics for the ESLSampleon Selected Study
Variables, by BroadAcademicAreas, with Comparative
DataforallEM;Students...............
15
Intercorrelations of Selected StudyVariables in the
TotalESLSample . . . . . . . . . . . . . . . . . .
19
.
SimpleCorrelation of GREGeneral Test Scoreswith
FYAfor Foreign ESLStudents: Weighted(Size-Adjusted)
Meansof Department-LevelCoefficients for Departmnts
Groupedby GraduateField and Area . . . . . . . . . .
29
RegressionResults for ESLStudentsUsing Departnmtally StandardizedData Pooledby GraduateMajor
Area . . . . . . . . . . . . . . . . . . . . . . . . .
31
Validity Coefficients for a StandardCompositeof QIE
Quantitative and Analytical Scoresversus Coefficients
for Individual GEEScores: Size-AdjustedAveragesof
Department-IevelCoefficients for Quantitative DepartmentsGroupedby Fields and Major Areas . . . . . . .
33
SimpleCorrelations of Selected Background
Variables
with FYAfor Foreign ESLStudents: Size-AdjustedMeans
of Departnmt-Level Coefficients for Departmnts Grouped
by GraduateField and Area . . . . . . . . . . . . . . .
34
Relationships for Subgroups
Defined by Melative Standingon Selected FSL-Proficiency-Related
Test or Self-Report Variables in AnalysesUsing Departmentally StandardizedPredictor/Criterion Data
PooledAcrossQuantitative Departments . . . . . . . . .
39
GI~E/FY?I
Table 12
Table 13
Table 14
Table 15
Table 16
Correlation of a Corpsite of Z-ScaledGREQuantitative (Zq) and Analytical (Za) Scoreswith Z-Scaled
FYA(Zfya) for Subgroups
Definedby CountryandRegion:
All Quantitative Fields
. . . . . . . . . . . . . . .
Correlation of a Cmposite of Z-ScaledGEIE
Quantitative (Zq) and Analytical (Za) Scoreswith Z-Scaled
FYA (Zfya) for Subgroups
Defined by Regionand Major
GraduateArea . . . . . . . . . . . . . . . . . . .
42
. .
42
Meansof RegionalGroupson Regularly ScaledFw and
GFBScores: PooledData for 1,213 Studentsin 86
Engineering, Math/Science,and Economics
Departments .
.
48
Meansof Foreign Studentson FYAand GEIE
Variables,
by Region, Ekpressed(a) as Deviations from the Means
of Their RespectiveDepartmentsof Enrollmentand
(b) as Deviations from GrandMeansfor all Students:
Data for All Quantitative Departmnts . . . . . . . . .
.
48
GrandMeanFYRandMeanFYA' as CcanparedtoWithinDepartmnt
Figure 1
.
for
MeanZfyaandMeanZ~fya
Regional
Groups: PooledData for All Quantitative
Departmnts.......................
50
Findings basedon analysis of data acrossdepartmmts
versus findings basedon analysis of data within
departments.......................
51
...
VI.11
The examineepopulation taking the GEXE
GeneralTest and the samples used
in standardization and calibration (scaling) are madeup predcaninantlyof U.S.
citizens towardwhomthe test is oriented &cationally, culturally, and linguistically.
However,the test is also taken by foreign nationals, the majority of whm speakmglish as a secondlanguage (ESL) and whosecultural and
educational backgrounds
differ from those of U.S. examinees.
The average GRE quantitative performanceof foreign ESL examinees is
fully comparableto that of U.S. examineesin similar fields of study, lmt
their average performanceon the GE?E
verbal andanalytical ability masures is
markedly lower, due primarily to factors associatedwith their less-thawnative level of English proficiency. For foreign examineesfor whom
English is
the native language(ENL), verbal and analytical averagesare cmsistent with
average performanceon the quantitative test.
Study d3jectives
The present study was designed to obtain information regarding the
typical within-departmentrelationship of GREscoresand certain ESlglish-proficiency-related variables to first-year averagegrade (Fw) in selected graduate fields
(a) in Samplesof foreign ESL studentsthat are heterogeneous
with respect to linguistic-cultural+ducational backgroundvariables, (b) in
subgroupsthat are hcmgeneous
with respect to country of origin and associated backgroundvariables, and (c) in subgroups
classified accordingto relative level of "English proficiency," as defined by the following variables:
0 Total score of individual studentson the XEFL, the Test of mglish as
a Foreign Language(available for about two-thirds of the students)
o Relative verbal PerformanceIndex (Rvp~)-the discrepancybetin
ob
served GE verbal score and that expectedfor U.S. GREexarninees
with given
quantitative scores, thought of as reflecting a type of general mglish proficiency deficit
o Relative Analytical PerformanceIndex (=I)-the
discrepancy be-n
obsenredanalytical score and that expectedfor U.S. GREexaminees
with given
different type of Enquantitative scores, thought of as reflecting a somewhat
glish proficiency than that indexedby the KVPI
0 Self-reported better cmmmication in English (BCE) status versus
better comnuni
cation in someother language
StuQSaqleandIBta
The study analyzed data for cohorts of foreign students, without regard
tovisa
status, who entered their respective departmmts in 1982-83 and
1983-84 as full-tim
students, whoearneda first year average, and for whom
GREscores and information regarding country of citizenship were available.
s-1
-
Studentswith GREscoresfrom a test administration prior to October 1981 were
not included in the study becausethe analytical test changedin October1981.
Data were obtainedfor 97 departmentsfrom 23 graduate schools. The cooperating schoolsand departxwntssupplied first-year average (FYA) grades and
ICEFLtotal scores (as available).
Theyalso supplied information on country
of citizenship, undergraduateorigin (U.S. versus other), sex, and date of
birth, if this informationwasnot supplied by the student whenregistering to
t.aketheGRE.
Most of the samples(86 of 97) were from primarily quantitative departments: engineering (electrical, civil, chemical, industrial, and mechanical),
mathematics,statistics, chemistry, physics, coquter science, or econcunics.
Six were from biological sciencedeparwnts, and 5 were from either education
or political sciencedepartments(see text, Table 1, for detail).
The department-levelsamples typically were quite small: llledianN = 12
(see Table 2).
Howver, acrossall departrrrents,data were available for a
total of 1,353 ESLstudents: 702 from co&k& engineering departments, 353
from math-science departments,and 138 from econtics departnwtnts, plus 55
from biosciencedepartmentsand 76 frcansocial science departments (primarily
education). Data were also available for 42 W students.
'Ihe samplewas extremelyheterogeneouswith respect to national origin
(see Table 3 and Table 4). Over 90 countries were representedby at least one
student, but about 90 percent of all the ESL students ware accountedfor by 39
countries with five or mDrenationals in the study -le.
Morethan twwthirds of the students (67.8 percent) were fram Asia, 11
percent were from Europe,9 percent from the Artwicas (excluding Canada),
7 percent frcanthe Mideast, and about 5 percent from Africa.
The three largest national contingentswere fram Taiwan, India, and Korea.
Thesecontingentsaccountedfor 666 of the 1,353 foreign ESLstudents.
'Ihe studentswere highly selected in terms of quantitative ability
(see
Table 5 for detailed swnary of samplecharacteristics). For both ESLand ENL
students, the GEEquantitative meanwas 684-above the eightieth
percentile in
the score distribution for all GREexaminees. For ESL students in quantitative departmentsthe wan was 698. Verbal and analytical meansfor ESL students were 382 and 486; for the m students, correspondingvalues were 546
and 592.
Quantitative nwns were lower for bioscience and social science students
than for those in the primarily quantitative departments. However,for ESL
students in all major areas the pattern of relative performanceon the respective ability ma&es JEGthe sa& highest on qua&i-btive,
lowest an verbal,
with analytical in between.
The ESLstudentswere highly selected in terms of English proficiency as
s-2
~asuredbyTOEFL: the total score man, 567, for the sa@e was at
mately the 84th percentile for all graduate-level TOEFLexaminees.
approx-
MeanFYAsfor the ESLstudents, except for those in the small samplefram
biosciencedepartments,were ccanparable
to those reported by the GRE validity
StudyService (VSS) for samplescosnposed
prehninantly of U.S. citizens.
o The meanfirst-year average (F’YA) grade for all ESLstudentsMIS 3.45 (or
approxiroatelyEt). For 1,213 students from quantitative departments, the
meanwas 3.49. The -11 samplesof students from bioscience and social
sciencedepartmentshad RA meansof 3.18 and 3.44, respectively.
In order to obtain general estimates of GRE/FYA
relationships for foreign
ESLstudents (data were not available for U.S. students), the principal analyses were basedon data pooled across departments within particular fields.
Since the study was primarily concernedwith general trends in GRE/FYArelationships, and not with the development of operational prediction equations,
it was cwenient to standardize the scores of students on each continuous
variable, including the GREpredictors. Predictor and criterion scores were
z-scaled-that is, they were expressedas deviations from the respective departmentmans in departmentstandarddeviation units prior to pooling. Data
for the small Sampleof ENL studentswere z-scaled using meansand standard
deviations for ESL students in their respective departments.
coefficients basedon the z-scaled variables are equivalent to avera
s of
v
corresponding
department-level coefficients weighted according to size of
sarqle. The coefficients reported in this study maybe thought of as estimates
of populationvalues for ESLstudents in the groupsof departmentsfor which
dataw~repooled.
Principal Fidings
GRJZ/FWI
RelationshipsFor General Samplesof Foreign ESLStudents
m determinethe typical levels of QIE/FyAcorrelations in general saw
ples of foreign EISLstudents in the respective fields, size-adjusted means of
department-levelGREvalidity coefficients were computedfor various groups of
deprtrntnts:
industrial, and mechanical engineering
a) chemical, civil, electrical,
departments,respectively; all engineeringdepartments
b) statistics, chemistry, physics, mathematics,and computer science
departraents,respectively; all I&h/science departrrrents
c) econcxnics
departments
d) all 86 quantitative deparmnts-(a)
e) biosciencedepartments(six)
s-3
+ (b) + (c)
f)
social science departmmts (five)
9~ size-adjusted mans of the resulting m
correlations (see text,
lable 7, for detail),
except those for the six biosciencedepartnmts, were
ccnparableto size-adjusted man coefficients that havebeen reportedby the
GEEValidity St&y
Setice (VSS) for Samples catgxx& predanimntly of U.S.
citizens.
o For all subgroupsof quantitative departruents,GZEquantitative scores and
GREanalytical scores were mre highly correlated with FyAthan were GRE
verbal scores.* For the pooled sampleof 1,213 studentsfrom the 86 quantitative departmnts, size-adjusted man coefficients were .3ll, ,275, and
,097 for quantitative, analytical, and verbal scores, respectively. Trends
were generally similar for engineering, math/science, and econcmics
samples.
o However,for the smll sampleof 85 students from five social science departments, verbal scores were mst valid and quantitative scores were
least valid.
Meancoefficients for verbal, analytical, and quantitative
scoreswere -253, ,184, and ,116, respectively.
o For the six bioscience samples,averagecoefficients for quantitative and
analytical scores were of about the sam magnitude(.061 and ,081, respectively), while the verbal coefficient was anomalously
negative.
titiple
regression analyseswere conductedusing z-scaled data aggregated for broader groupingsof departmnts. For the quantitative departments,
regression results for the combinedsampleand those for engineering, mathscience and economics subgroups
ware similar (see text, Table 8 and related
’
discussion).
o The standard partial regression (beta) coefficients for quantitative,
analytical, and verbal scores in the combinedquantitative sar@e were
* In evaluating the validity coefficients obtained in this study for GRE analytical ability scores it is important to knowthat prior to October 1985
users were advised by EI'S not to consider the analytical score in selective
admission. Assumingthat this advicems followed, 0nlyGREverbaland quantitative scores were considered directly in admitting the ESLsamples, and
their relationship with EYAwill tend to be reducedsmewhatby the resulting
restriction in range. For analytical scores, only same"incidental" restriction in range would be expected (they are positively correlated with the
verbal and quantitative scores). However,attenuating effects on validity due
to restriction of range in selection shouldtend to be greater for the verbal
and quantitative scores than for the analytical scores. Validity studies
basedon samples tested after October1985 will be necessaryin order to
evaluate the relative contribution of the three GREmeasures
under conditions
in which all three were eligible for consideration in selective admission.
-24, -18, and -.Ol, respectively. (The negative beta coefficient
al reflects a negligible suppressioneffect.)
0
for verb
Patterns of regression weights for the mall bioscience and social science
sampleswere consistent with the patterns of validity coefficients.
When
departmentally_z-scaled
FYA(Zfya) was regressedon departmentally
_ _
_z-scaled GRE(Zgre) variables (Zq, Za, and Zv) in the combinedquantitative
sanple, and in the engineering, math-science,and econmics Samples, respectively, only quantitative (Zq) and analytical (Za) scorescontributed significantly to prediction of Zfya. me best-weightedcomposite in the combined
quantitative samplewas ,238 Zq + ,176 Za = Z'fya (predicted Zfya).
This general equation wasused to cmte
Z'fya for studentsin each of
the 86 quantitative departmmts. Tne averagedepartmnt-level validity coefficient for this standard c-site
tended to be higher than the averagecoefficient for either quantitative or analytical ability considered separately
(see Table 9 and related discussion).
Effects of mglish Proficiency on GREValidity+
Regressionanalyses based on pooled departmentallyz-scaled data were
conductedfor subgroups of students classified accordingto level of English
proficiency, variously defined by (a) the relative verbal performance index or
RVPI, (b) the relative analytical performanceindex or MI,
(c)
selfreported better conmmicationin English (SR-BCE) versus other status, and (d)
TOEF%
total score, respectively (see Table 11 and related discussion).**
Interpretation of results for classifications basedon TOEEL
total score
was corqlicated by the fact (a) that manystudents did not haveXXFL scores
and (b) that differences in regressionassociated with *'availability versus
nonavailability" of the score were muchmre pronouncethan those associated
with differences in TOEFGscorelevel amng those with TWFL.
antrolling for level of I%qlish proficiency as
I3owever, m
balance,
definedbytksevariahlesdidnotagqear
tohave aclearmDderati.ngeffectm
the relationship
bebeen the z-scaledFYA criterim &the
z-scaled GRE
scoresforESLslzukr&sinanyofthequantitative
subgroups(seeW3lellaxrl
_
- relateddiscussim).
* In these and subsequentanalyses, only data for studentsfrcm the 86 quantitative departmentswere included.
**RVpKvpI=discrepancybe-n
observedverbal score, V, andpredicted verbal
score, V*, whereV' = .52 (GE?E4)+ 185, a regressionequationbasedon data
for a sample of U.S. examineestested during 1981-82. RANPI= discrepancy
betin analytical score, A, and predicted analytical score, A', whereA' =
,661 (GE-Q) + 202, basedon data for the sam general sample.
-f;
.....*_:
.-
Parallel validity for ?DEm,and GRE verbal. Regressionresults obtained
whenZfya was regressedon Zq, Za, and Zv generally paralleled those obtained
whenz-scaled 'IDEFLtotal score was substituted for Zv. Neither variable contributed significantly to prediction in the primarily quantitative samples.
GREValidity for NationalandRegional Subgroups
Simple correlations betweena standardcompositeof z-scaled quantitative
and analytical scores (predicted Zfya, or Z'fya, specified by the regression
of Zfya on Zq and Za scoresin the cmbined quantitative sample-that is, ,238
Zq + ,176 Za) wre computed
(a) for students frcunall quantitative departmnts
classified by country of citizenship (N > 9) and by regions defined for the
study, and (b) for students classified by both region and graduate major
area-that is, engineering, math/science,and economics (see Table 12 and
Table 13, and related discussion).
'Ihe correlation between this compositeand Zfya tended to be relatively
consistent for subgroupsdefined in terms of national origin and/or academic
area. Correlations did not tend to be higher for subgroupsthat were home
geneousthan for those that were heterogeneous
with respect to national origin.
o For 20 national contingents with N > 9, the mdian Z'fya/Zfya coefficient
was r= -36, and for nine regional contingentsthe mdian was I: = -34, as
comparedto the general quantitative samplecoefficient of I: = -35.
o For 22 subgroups(by region and quantitative area) the median coefficient
was II = .37Coqarative Performanceof RegionalSubgroups
on FYAand GREVariables
The study ms not designedspecifically to assessthe extent to tiich the
averageacademicperformanceof studentsfrom different countries or groupsof
countries (regions) tended to be consistent with their averageGRE perform
ante. Generally speaking, an assessment
along these lines is complicated by
the national diversity of the foreign student population, lack of consistency
across departmentsin the national and/or regional mix of enrolled students,
and differences in the fields of study selected by various national and regional groups. Members
of various national or regional subgroupsmaynot be
enrolled in departmentsthat are comparable
with respect to grading standards,
degree of selectivity, and so on.
Suchfactors complicate interpretation of obsemeddifferences in the
averagewithindepartment standingof national or regional contingents as refleeted in their mans on departmentallyz-scaled predictors, predictive cm
posites, and/or criterion variables.
Exploratory analyses were conducted to assess (a) differences in the
averageacademicperformanceof students in several regional classifications
defined for the study arkI (b) the extent to which the obsenred differences in
academic performnce tended to correspond to observed differences in QIE
performance. I‘bm parallel analyses were made, one involving coqarisons based
on departmntally z-scaled data (indicating average withindepartment standing
on predictor
and criterion variables)
and the other involving comparisons
based on original
FYAand GREscores (indicating average standing on the FYA
criterion and the GREscores without regard to departmnt of enrollment).
Results of the regression of FYA on GREscores for students in the quantitative
sample, treated without regard to their departments of enrollment,
paralleled almost identically
the results of the regression of Zfya on Zgre
scores in pooled data for the w
Sample. Means of predicted FYA values
(ETA’)
were ccdnputed using the general Sampleequation:
-0013 GREiQ+ .0006
GREXI. The corresponding predicted Zfya values (ZYya) were specified by:
,238 q + ,176 Za. Thus, regional comparisonswre based on tw sets of mean
observed criterion scores, =ly,
FYA and Zfya, and the corresponding man
predicted criterion scores, FYA’ and Z’fya.
For the mst part, patterns of findings regarding regional differences
based on the two sets of data were consistent. For example, with one exception
the ranking of regional groups in terms of meanFYAwas consistent with ranking based on m?anZfya (see Table 14 and Table 15 and related discussion) :
0
0
0
At one extrerrre, students from Europe had man FY& of approximtely
3.6,
and they also enjoyed relatively
high within-department Zfya standing,
averaging 0.2+ standard deviations above the foreign ESL meansfor their
At the other extreme, students from Africa and
respective departmnts.
the Mideast earned grades averaging approximately 3.3, and they were in
departments in which they tended to perform at a lower level than other
ESL students (below average by 0.2 standard deviations).
For students from Asia, who accounted for some 68 percent of all
foreign
students in the sample, meanFYA was approximately 3.5 ( corresponding to
the grand FYAman for all quantitative students without regard to department) and their Zfya mans were approximtely 0.0.
By inference, Asian
students tended to provide a substantial cmnon element in the regional
mix of the respective departmmt-level samples; both the department-level
(Zfya) means and the grand mean FYA reflect substantially the coqaratively high FYAsearned by the Asian students.
Effects associated with department of enrollment were illustrated in data
for students from South American countries and Wxico hose Fw mean of
3.4 was lower than the general man FYAof 3.5 but who were from departmznts in which they enjqed above average relative standing (man Zfya ms
+O.ll).
Patterns of relationships be-en mean FYAversus meanFYW were similar
to those for mean Zfya versus meanZ’ fya (see Table 16 and Figure 1, Section
VII, and related discussion). Again, the principal departure from parallelism
in the two sets of data was associated with the South Amrican contingent.
s-7
0
European students had s&t
higher n~an FYA and man Zfya than predicted fram the respective general ESL equations.
Students from Africa and
the Mideast, regional groups with the lowest ranking on GREvariables were
also were the lowest ranked on both mean FYA' andnreanz'fya.
Theacademic
standing of Asian students tended to be consistent with expectation based
on the general ESL equations.
The findings of these exploratory analyses suggest (a) that there are
systematic differences by world region in the average academic performance of
foreign students, and (b) that level of academic performance of regional subgroups tends to be generally consistent with expectation based on GFE scores.
However, the findings should be thought of primarily as suggesting directions
for research designed specifically to assess the possibility that mng
foreign ESL students, sorrenational or regional subgroupsmay tend to perform
better
(less well)
than other subgroups with cornparable GRE scores.
Implications of Firdings
The findings that have been reviewed permit relatively strong inferences
regarding the relationship be-en
GRE scores and first-year
average grades
for foreign ESL students in quantitative fields-students
enrolled in engineering, math/science, and economics programs. GRErelationships
for
general department-level Samplesof foreign ESL students in these quantitative
areas appear to be quite coqarable in both level and pattern to relationships
reported for samples of U.S. students in similar fields.
A standard c-site
of QIE quantitative and analytical scores had general validity
for predicting relative
within-department standing on the FW
criterion across all
the quantitative
areas represented:
the several engineering fields,
math-science fields, and economics. For students in these
fields, neither GREverbal scores nor TOEFLtotal scores contributed
significantly to predictian when included in a battery with QZE quantitative
and
analytical scores.
The validity of the standard coqosite was not affected by introducing
control for English proficiency, variously indexed, nor was validity
stronger
for groups that were horwgeneous than for groups that were heterogeneous with
respect to national origin.
In evaluating this finding, it is important to recall that the students
in these quantitative‘ departxwnts were highly selected in terms of quantitative ability as rwasured by the GRE, and in terms of English proficiency
as
Ihey were in fields emphasizing quantitative
reasoning
rr82asuredbytheTWFL.
abilities
and international symbols. These fields presumably require coxrparatively low levels of general English language verbal cm
ication
skill.
Ibe
opposite may be true for fields such as, say, education.
In the limited sample of ESL students from five "verbal" departments
(four &cation
and one political
science), the correlation between FYA and
GREverbal scores was stronger than that for either GRE quantitative
or
This is consistent with a priori
expectation based on
analytical scores.
s-8
1.
- ;
._
,.-.-.?a_
; -_:i
validity study findings for U.S. citizens. Accordingly, it seems reasonableto
hypothesizethat this pattern may tend to be typical for foreign ESL students
in primarily verbal fields.
Findings for the six bioscience departments were not consistent with
expectation and can be interpreted only as reflecting samplingeffects.
The averagescores of ESLstudentson the verbal and analytical ability
masures (382 and 486, respectively) were nwe than one standard deviation
lower than wouldbe expected for U.S. exaxninees
with the saw quantitative
ability-the 11y3an
quantitative score for foreign ESL students was 698, only
slightly higher than the quantitative meanof 687 reported by the GREvalidity
Study Service (VSS)at ETSfor r&h-science zzunples
ccqosed primarily of U.S.
citizens (with a GREverbal mean of 528). The mean Fw for foreign ESL students was approximately3.5 (or Bt on the grading scale employed), as compared
to a meanof 3.4 reportedby the GFEVSSfor U.S. math-scienceSamples.
This particular pattern of findings suggestsnot only that the foreign
EISLstudents in this Sample tendedto be "academically sucessful" but also
that they mayhave tendedto receive higher grades, on the average, than their
U.S. counterparts with ccaqarablescores on the GRE.Research is needed to
assess the validity of this inference as well as the possibility that the
average academicperformance
of sm regional or national contingents IMy not
be consistent with their averageperformanceon the GRE.
With regard to the analytical masure, it is again noted that scores an
this masure presumably
were not useddirectly in selecting the Samples under
consideration in this study, hereas scores on the quantitative and verbal
masures were used directly in selection. The contribution of the analytical
masure maybe overestirrrated
somewhat
in the current Samples.
On balance, the study findings suggest that inferences based on GRE
scores regarding the subsequentacademicperformance of foreign ESL applicants, especially those applying for admission to primarily quantitative departments, are likely to be as valid as those for U.S. applicants.
s-9
Section I:
Background
The Graduate Record Examinations (GRE) General Test is widely used for
evaluating the academic qualifications
of applicants for admission to graduate programs in the United States.
The General Test traditionally
has
provided masures of developed verbal and quantitative
reasoning abilities
(GREverbal or GRE-V and GRE quantitative or GRJZ4). In 1977, the test was
restructured (see Miller
& Wild, 1979) to include a masure of developed
analytical reasoning ability,
a revised version of which was introduced in
October 1981 (see, for example, EI'S, 1985). Ihere was some restructuring of
the verbal and quantitative masures in terms of format and time allotments,
but no change in test content was involved. Prior to the October 1985 test
administration, users were advised not to consider the analvtical score in
admission.
The verbal test includes antonyms, analogies,
sentence completion, and
reading passages or reading comprehension item-types, and the quantitative
test includes quantitative comparison, regular mathematics, and data interpretation item-types.
These verbal and quantitative
items Sample two wellestablished ability domains. Less is known regarding the "ability
d-in(s)"
sampled by the analytical
ability
measure, which includes 38 analytical
reasoning (AR) and 12 logical
reasoning (LR) itemtypes,
both of which call
for considerable verbal processing.
The examinee population taking the GREGeneral Test and the samples used
in standardization and calibration (scaling) are made up predcminantly of U.S.
citizens toward whomthe tests are oriented educationally,
culturally,
and
linguistically.
Hmever, the test is also taken by foreign nationals. During
1981-82, for example, non-US. citizens representing n-me than 140 different
countries, territories,
or other geopolitical
entities madeup approximtely
16 percent of the total GRE examinee population (Wilson, 1984a, 198413).
There are large differences beWeen the population of U.S. examinees and
the population of foreign examinees in average performance on the verbal and
analytical measures. The verbal and analytical performnce of foreign examinees, largely
individuals for whomEnglish is a second language (ESL exam
inees), is markedly lower than that of U.S. examinees or of foreign W
(English native language) examinees. The depressed verbal performance of foreign
ESL examinees is attributable,
primarily,
to their less-than-native levels of
proficiency in English.
However, performance on the GREquantitative
measure does not appear to
vary with mglish language background. U.S. examinees, foreign ESL examinees,
and foreign ENL examinees in the sam fields of study tend to have similar
quantitative rtmns. National contingents with very depressed verbal and analytical means frequently have very high quantitative mans.
In essence, it appears that for foreign ESL examinees GREverbal test
items (indeed, all English-language verbal test items) are masuring selected
aspects of "developing ESL proficiency" rather than level of "developed verbal
reasoning abilities,"
which the test measures in samples of native English
speakers. Thus, papulation differences in level of "verbal reasoning ability"
or "analytical reasoning ability" cannot be inferred from observed popvlation
differences between U.S. vs foreign ESL exam&es in test performance.
-2At the ~~IIXZ time, the population differences in verbal test performance
represent "real" population differences in "functional ability"
to perform
English-language tasks such as those represented by the verbal and analytical
test items under testing conditions, including time constraints.
Quantitative test items, on the other hand, appear to be rwasuring the
same underlying abilities
for foreign ESL examinees as for U.S. examinees.
Available evidence suggests that U.S. and foreign examinees with similar
GRE
quantitative scores have similar levels of quantitative ability.
Given evidence of systematic population differences both in background
and in test performance-evidence permitting the strong inference that the
verbal and analytical tests are not rwasuring the m
underlying "ability
constructs" for U.S. and foreign ESL examinees (prospective students)-questions naturally arise regarding the criterion-related
or predictive
validity
of GREscores for xwmbersof the foreign ESL student population. For example:
o For individuals in general samples of foreign students, how valid are the
GREscores for predicting subsequentperformance on somecriterion
of success in graduate school (say, first-year average grade, or FY?Q?Are the
GRE/Fw correlations (GRE/Fw validity coefficients)
for general Samples
of foreign ESL students camparable to those typically observed for general
Samples of U.S. students? Do GRE/FB validity coefficients tend to be cm+
parable for different national contingents of foreign students? Are GRE/
FYA correlations typically
stronger in samples that are relatively
hm
geneous with respect to linguistic-cultural-educational
background variables (for example, samples from specific countries or from countries
judged to be similar with respect to language, culture, or educational
systems) than in heterogeneous sa+es?
Is criterion-related
validity
typically higher in samples of foreign ESL students with "higher levels of
English proficiency"
than for those with "lower levels of
English
proficiency?"
o How closely does relative criterion
standing of various national continc-sites
gents correspond to their relative standing on GREpredictive
developed for general ESL samples?Do students with more "ESL proficienexample,
cy, " tend to outperform students with less "ESL proficiency"-for
do they tend to earn higher ~lleanFYA than would be expected for foreign
ESL examinees generally,
who have similar GREscores? Within the papulation of foreign ESL students, do s= national-linguistic
subgroups tend
to earn higher average grades than would be expected for foreign ESL students generally, who have similar GREscores?
ThePresentStu3y
?he present study was designed as a GREpopulation validity study for
foreign ESL students. It was prirtwily concernedwith obtaining and evaluating
evidence bearing on the first set of questions outlined above. More specifically, the study MS designed primarily to assess:
o the
level and pattern
of GRE/F'YAcorrelations
in
general samples of
-3-
foreign ESLstudents in selected graduatefields,
the possibility
of systematicdifferences in the level and pattern of
GRE/FB car*relat i .onsfor subgroupsoA
F foreign ESLstudents, especially
subgroupsdiffering in English languagebackgroundas reflected by
national-linguistic origin, and by -level of -performance on vario%
Engliskproficiency-related test variables, including total score on
the Test of English as a ForeignLanguage
or ?DEm,(El& 1983), and
whether improved prediction of FYA for foreign ESLstudentsmight be
expected by considering information regarding national origin and
scores on the 'IDEFLin conjunctionwith GREscores.
The study wasnot designedto addressquestions regarding the ccqarative
performanceof various subgroups
of foreign examinees. However, exploratory
analyses were undertakenthat permitted limited inferences in this regard.
The study was conductedby ticational Testing Senrice (ETS) under the
100
auspices of the Graduate &cord Examinations-rd.
In March 1984,
graduate schoolsthat usually receive the largest nunberof GREscore reports
frc4nnon4J.S. citizens were invited to cooperatein a study with the foregoing
objectives by providing data for Samplesof non-KS. students, (a) regardless
of U.S. visa status, who (b) were first-tinut, full-t*
students during the
academicyears 1982-83 and 1983-84, respectively, (c) coqleted the acadexnic
year in which they initially
enrolled, and (d) earn& a first-year average
grade.
The graduate fields (departments)targeted for studywere primarily guak
titative fields knc3wto be mst popular aT[y3ng
international graduate students: engineering-chemical, civil, electrical, industrial, and mechanical;
mathematics,chemistry, physics, coIq3uterscience, statistics, and econamics.
Several fields that involve m3re verbal (less heavily quantitative) subject
matter were also targeted: educationand political science; agriculture, biology, biochemistry, and microbiology.
Over 100 departmentsprovidedIBE, sex, and date of birth identification
for the designated entering cohortsof foreign students. This identification
was used to locate records of the studentsin the GREcomputerized history
file-records containing GE?EGeneralTest scoresand the corresponding test
administration date(s) and other relevant information (if provided by individuals whenthey registered to take the GRE).
Basedon the file-matching procedures, data collection rosters were prewere asked (a) to
pared w EE and sent to the departments. Departments
supply a first-year average grade and, if available, TDEF'Ltotal score, (b) to
identify native English-speakers, and (c) to provide information regarding
country of citizenship and/or U.S. vs. non-U.S. undergraduateorigin (whenthe
roster indicated that this informationwasnot available in the GPEfile for a
student).
Basic requirementsfor inclusion of a departmentaldata-set in the study
were that the departmenthave a minimum
of five foreign ESLstudentswith:
-6
0 verbal, quantitative, and analytical ability scores from a
GREGeneral Test that ms administered after September 1981,
o a first-year
restructured
average grade, and
o data on country of citizenship.
Distribution
of students by School and DeparTtlmlt
A total of 97 departments from 23 graduate schools met these basic reThe departments (fields)
represented and their grouping by
quirements.
academic area, were as follows:
1.
2.
3.
4.
5.
6.
Engineering (chemical, civil, electrical,
industrial,
mxhanical)
&&h/Science (applied math, statistics,
chemistry, physics,
coqmter science)
Econmics
Quantitative total (engineering + math/science + economics)
Bioscience (microbiology, agriculture, biochemistry, biology)
Social science (education, political
science)
Table 1 shms the distribution of ESL students by school and graduate
departmnt. The 23 schools (identified
only by a Wigit
study code) ranged
from the top 10 percent through the bottom 10 percent amongthe 100 graduate
schools that typically
receive the largest number of GRE score reports from
n0n-U.S. citizens.
0
There were 86 departnmt-level ESL samples from primarily
quantitative
fields, with a total of 1,213 foreign ESL students: 40 engineering sa@es
with a total of 702 students, 36 mathematics and physical science sa@es
(373 students), and 10 economicssamples (138) students.
0
data were available for only 6 biological
science samples (55 students)
and 5 social science samples (4 education samples with a total
of 76
students, and 1 political science sample with 9 students).
Table 1 showsthat most of the departmnt-level
samples were quite small.
of Ns for the 97 foreign-ESL Samples, shownin Table
2, points up the positively skewednature of the sample-size distribution.
The actual distribution
A total of 32 samples included between 5 and 9 ESL students, 36 sa@es
includd between 10 and 14 students, 13 included between 15 and 19 students, and 16 included 20 or mre students. The median N was 12.
Bata were also available for a total of 42 non-US.
English-native-language students from major English-speaking countries (31 from the quantitative
fields, one frm a bioscience field, and 10 from the social science fields).
These data were e@oyed in the study primarily
to point up differences
between foreign EDIL students and foreign ESL students in patterns of average
performance .onthe GREGeneral Test.
Table
Distribution
of
Foreign
ESL
(English
Second
Physical
Sci
1
Language)
Department
Sch
Engine
65
66
Math/
ering
67
68
Tot
54
62
59
72
Students
by
Tot
78
and
Department
code
P.S.
Bioscience
Econ
76
School
84
Quan
07
31
34
35
Bio
85
92
sot
tot
-
02
07
17
32
-
12
30
33
12
7
94
9
10
13
14
19
14
31
-
-
4’:
6
-
10
14
31
-
9
-
14
20
-
37
57
10
24
28
5
42
19
13
24
30
32
13
13
12
-
10
103
0
48
33
34
-
I1
-
36
52
-
17
-
64
52
-
13
5
38
41
-
19
5
15
13
25
-
13
-
57
15
18
-
10
5
-
-
-
10
0
15
0
5
xs
11
-
-
18
-
Note.
Departments
15
-
12
7
10
26
15
12
8
9
21
-
10
15
12
24
10
6
-
87
10
58
38
I1
-
11
16
0
11
48
8
16
12
5
7
13
37
10
5
101
67
33
21
7
13
6
12
-
90
48
25
9
0
-
9
29
7
-
7
0
0
8
-
8
0
0
-
16
36
138
10
1213
86
11
-
6
9
7
0
-
0
80
8
162
256
8
10
Quant(itative)
and
89
6
total
codes:
59
Statistics,
62
ture,
34
Biodhemietry,
115
702
8
ie
sum
Engineering--64
Chemietry,
35
72
Biology;
54
40
16
111
5
of
Ne
for
Engineering,
Chemical,
Mathematics,
85
Education,
39
4
10
65
76
51
10
Math
Civil,
Physics,
92
Political
6
66
78
373
113
6
Physical
Computer
Science.
-
6
-
26
-
26
102
17
-
9
26
0
194
45
0
0
21
0
0
65
87
0
21
10
199
0
0
0
38
75
0
6
-
-
16
-
-
12
I
and
0
0
67
106
0
0
48
32
13
21
9
0
0
0
-
0
-
0
0
0
0
-
0
0
:
-
0
0
11
16
85
5
1353
97
11
-
11
0
23
2
19
2
7
55
76
4
6
9
1
Economics.
Industrial,
84
48
113
0
7
-
1
12
-
0”
6
-
-
0
16
7
-
-
0”
0
13
21
67
Science;
8
0
Science,
Electrical,
7
153
45
65
5
5
11
0
15
0
119
15
47
0
16
172
38
75
48
-
0
0
92
Depte
8
6
13
29
87
91
Total
11
0
49
60
73
82
9
I1
Total
SC1
Economics;
68
Mechanical
07
Microbiology
54
;
Math
31
9Applied
Agricull
29
4%
Table 2
Distribution
5
4*
4
3*
3
2*
2
1"
1
O*
of 97 Study Departments by Size of ESL Sample
2
8
2
6
01123
566
0114
55556 66777 899
00000 01111 11111 12222 22222 33333 33344 4
55555 55555 66666 77777 78888 89999 99
( 1)
( 1)
( 1)
( 1)
( 5)
( 3)
( 4)
(13)
(36)
(32)
Note. Sample sizes are specified by combining leading digits
xth
successive digits in the respective rows. For example,
52, 48, 42, 36, 30, 31, 31, 32, 33, and so on. Mdn N = 12.
-7-
In view of the limited numberof bioscience and social science samples
available, and their typically small size, the study focussedprixtwily on
data for ESL students in the 86 engineering, math-science, and econc&cs
departments.
National
Origin
and
Eraglish-Ianguage
Backgrand
of Students
in
the Sample
The samplewas extremelydiverse with respect to national origin. Over
95 of 140 countries (territories, protectorates, or other geopolitical entities) representedin the GREexamineepopulation were represented in the study
sample. However,IIy)re than 90 percent of the 1,353 foreign ESLstudents wre
accountedfor by 39 countries that were representedby at least five students.
Table 3 lists the 39 countries in order of total representation in the study
sa@e, and showsthe distribution of student-nationals by academicareas.
0
The largest contingents of students were fram Asian countries. More than
-thirds
of the students (67.8 percent) were froan&ian countries, 10.9
percent were from Europe, 8.6 percent from the Ilrnericas(excluding Canada), 7.4 percent fram the Mideast, and 4.9 percent from Africa.
English-Language
Background
and National Origin
The last colurrrn
of the table showsthe fl3Em, total meanreported by Glls
(1983, Table 10) for all U.S.-bound TCEFL,examineesfrom each country, tested
during 1980-82, without regard to educational level, designatedas XEE'L-LEVEL
to help reinforce the fact that they are not the TOEFLmans of students in
the Sample.
o Ihe national means('IIDEFGW
values) shownin Table 3 indicate typical
levels of mglish proficiency, as Easured by the ?DEFL,for contingents
of prospectiveU.S. students from the respective countries. Theymay be
thoughtof as reflecting, in part, backgrounddifferences associated with
national-linguistic origin (a) in the usual patterns of English language
acquistion andusage (for example, type, duraticm, intensity, and quality
of experiencein the use of English as a secondlanguage), and (b) degree
of overlap betwzennative language and ~glish.
U.S.-boundstudents from
India, Singapore,or the Philippines, for example, typically have had a
"richer" English languagebackground including mre experience in using
mglish in both general and academic settings than, say, students from
Taiwan,Japan,Thailand, or the Mideast, whose native languagesand English, rroreover,have relatively few cclm[y~n
elerwnts. Ihe degree of overlap betweennative languageand English varies markedlyand it z~y be seen
that TDEFGLEVEL
values also tend to be higher for examineesfrom western
Europeancountries than for Asian or Mideasternexaminees.
o Background
differences other than those involving mglisklanguage usage
are also partially reflected by differences in 'IDEm,swans. In a study of
TTXFLexamineestested during 1977-79 (Wilson, 1982a), Izloderate correlanwns and published
tions (approximately .5) were found between ?DEF'L
indicators of national developmentsuchas literacy rate, higher education
-8Table 3
Distribution of Students by Country of Citizenship and AcademicMea
for 39 Cuuntries Represented by Five or More Students
country
Engin
M&PS
Econ
auant
Biosci
Sot
Sci
total
188
108
68
50
26
Japan
PRepchina 17
28
Greece
Hong Kong 19
17
Thailand
12
Malaysia
13
Pakistan
7
Mexico
3
Chile
12
Turkey
3
wermany
7
Egypt
5
Colcanbia
Denmark
11
4
Spain
5
Brazil
5
Venezuela
11
Peru
3
2
Indonesia
Israel
6
6
Nigeria
3
Singapore
Philippines
1
Algeria
5
3
Yugoslavia
4
Jordan
2
Cyprus
0
IvoryCoast
2
Argentina
5
Nepal
4
Italy
1
Tanzania
Bangladesh 2
Vietnam
3
Subtotal
671
% of Total 95.6
Total
702
Taiwan
India
Korea
Iran
94
63
52
10
7
22
8
9
2
4
6
3
5
1
10
3
4
2
0
4
4
1
3
1
2
1
1
4
3
5
3
3
0
2
0
0
1
1
2
346
92.8
373
12
5
26
1
7
2
4
3
1
5
2
5
8
4
4
3
4
0
9
0
0
0
2
6
0
0
1
2
0
0
0
1
4
2
0
0
0
2
0
125
88.7
141
294
176
146
61
40
41
40
31
20
21
21
15
16
17
17
13
13
13
13
9
9
12
8
9
8
7
5
7
8
8
7
6
4
6
5
4
2
5
5
1142
94.1
1213
10
4
7
2
2
2
2
0
3
1
2
2
3
1
0
2
0
0
0
1
0
0
1
0
0
1
0
0
0
0
0
0
1
0
0
0
1
0
0
48
87.3
55
16
1
12
1
2
1
1
1
9
5
0
5
0
0
0
0
1
0
0
3
3
0
1
1
1
1
4
1
0
0
1
0
1
0
1
1
2
0
0
76
89.4
85
All
fields
320
181
165
64
44
44
43
32
32
27
23
22
19
18
17
15
14
13
13
13
12
12
10
10
9
9
9
8
8
8
8
6
6
6
6
5
5
5
5
1266
93.6
1353
mm
r
LIEVEL”
(493)
(555)
(504)
(484)
(487)
(473)
(502)
(508)
(473)
(534)
(518)
(514)
(520)
(500)
(576)
(485)
(508)
(580)
(543)
(513)
(479)
(487)
(513)
(479)
(528)
(509)
(563)
(547)
(511)
(530)
(458)
(482)
(498)
(543)
(523)
(552)
(542)
(483)
(497)
3cToEFGLEvELis the TOEFLTotal mean for all .TOEFGtakers fromthe country
during the period 1980-82, as reported by EIS (1983, Table lo), not the mean
for student-nationals
in the present study. The grand man for all IDEm,
exammees during 1980-82 t(ms 503, S.D. = 66.
._~. .
..~, :‘.
. ..
. ;A-
-9-
enrollrclentrate, per capita expenditures for education, and so on, in
analyses involving rrreansfor sm 100 different countries. Students from
higher TOEFGLEVEL
countries as compared to those fromlcxr
TOEFGLEVEL
countries maytend to enjay sameeducationaladvantagesas well as advantages in the acquisition of general ESL proficiency.
Mean'IDEFGLEVEL
differences appearto be quite stable wer time. In the
study cited above, a correlation of .94 was found betweenmans of 129 national contingents of U.S.-boundTWFIXakers in two different testing years.
A classification of countries of citizenship that takes into accountboth
geographiclocation (world region) and characteristic differences in English
languagebackgroundis provided in Table 4. U.S.-boundstudents (whotake U.S.
admission-related examinations)from CategoryI countries (Asia I, Europe I,
Africa I, AmericaI), as compared
to their CategoryII counterparts, typically, (a) have had moreextensive experiencein using English as a second language, and/or either (b) earn higher averagescoreson the TCEET.,,
and on other
English-languageverbal treasures suchas GREVerbaland GMATVerbal,or (c)
earn verbal scores that are relatively mDreconsistentwith expectation based
on their quantitative scores (see, I?& 1983; Powers,1980; Wilson 1982a,
1982b, 1984a, 1984b, 1985, 1986b). Nowithin-region subgroupingws judged to
be feasible for the Mideastern countries, whichtend tohave Category II
characteristics.
The last column of the table shws for each regional classification the
average of the TDEFGIEVEL
values for the respectiveI[ymbercountries MoWn
in Table 3) weight& accordingto the numberof students representedin the
present sample, including students fram "other" countries. To reiterate,
these TOEFGLENEL
values are not the T0EFZmeansof students in the sample.
that the backgrounds
of U.S.As a working proposition, it ws as&
boundstudents from CategoryI countries as compared
to CategoryII countries,
typically, were I[y)reconducive to develvnt
of general ESL conwnmicative
coqetence.
Table 4, page 2 of 2 pages
Table 4
Academicarea
Distribution of Foreign ESLStudents by Country of Citizenship
within “Regimal” Classificatio&Based oh Geographic
and hglish-Backqrmrxl-Related Variables
RegiW
Academicarea
MbPs
Biosci
Eka-l
Sot Sci
LEvEz*
Asia II
(374)
(203)
Taiwan
188
Korea
68
Japan
E%epulina f!
94
EE?
Malaysia
Pakistan
Indonesia
Nepal
:;
;
ii
(lm
108
India
Singapore
3
Philippines 1
0
Other
&mica
2;
9
2
:
:
(76)
:
3
::
6
4
5
II
Mexico
Chile
Colanbia
Brazil
Venezuela
PeLlJ
Argentim
Other
294
10
(48)
(720)
ii
165
320
-4%
:
:
1
9
(35)
s
s
0
:
13
(41
(6)
(1%)
1
181
4
fi
:
0
z
5
176
5
:
0
:
0
;
i
0
0
(2)
(96)
(2)
(4)
(104)
i
61
12
8
ii
ii
:
64
12
9
8
0
1
1:
f
(27)
10
6
8
63
:
i
f
(188)
(20)
2:
i:
(8)
:
1
170
(24)
(86)
(7)
(16)
(109)
e5
:Fi
13
i
0
ii
1
i
fi
14
13
;
ii
t
10
i
0
ti
3
:;
6
13
7
:
i
B
:
5
s
2
3
(0)
(3)
(3)
(6)
mropfi II
(49)
(20)
(9)
(78)
-483
-5l2
lbtal
(7)
-555
(83)
507
:
:
8
:
(15)
10
(64)
(62)
t;
0
0
13
x
x
::
13
1;
0
:
1:
(3)
(42)
(16)
(9)
(8)
(33)
(6)
7
@wet
Algeria
5
Ivory Coast 0
4
Other
3
:
13
8
2
0
11
;
:
7
(01
lml?L
LEVEW
43
18
:
Africa II
-.556
(1)
:
9
0
2
Demrk
Spain
Italy
other
Sci
4
28
GCl?WX!
12
m
‘ rkey
Yugoslavia 3
2
4
%Y
W&W
Sfx
QLLant
Aumica I
(18)
::
32
32
0"
M&PS
tow
;
5:
9
(68)
10
Iran
IAhrm
Israel
Jordan
OthfX
(29)
146
40
41
31
20
52
::
2
ni&ast
(643)
1’
Vietnam
-lade*
other
Asia1
(66)
Ehgin
iT
3
;
Africa I
(10)
(5)
(2)
Nigeria
muania
Other
6
1
3
1
1
3
:
2
702
373
141
-561
497
15
x
t
ii
13
(24)
-539
9
1:
1353
-510
Note. Classification of countries within several of the wrld regions (Asia I
and so cm) takes into account
versus Asia II, mrcpe I versus EUope II,
differences in mglish-language backgrand associated with mtional-linguistic
origin. Contingents of U.S.-bound students fran Category I countries as cmpared to those fran Category II countries are assumd to have backgroumisthat
are mre cmducive to the aquistion of ESLproficiency. Consider, for example, typical patterns of ESL aaquisitiar and usage in America II (e.g., Mexico
and South America) versus America I, a classificatitn that includes OuyaM,
Haiti, Horhras, Jamaica, Trinidad aK1 lbbago (none of which was represented
by five or mre nationals in the St*
sample), ccuntries with strmg Ehglishspeaking traditions.
* Ihe regianal IrDEFLJ_5JEX
values shcxmin the last colum are averages of the
mher-country X~%-LEVEL values weighted according to the mmber of students
fran those countries (see Table 3 for country values and definitiaul
notes).
-llSection II: Description of StudyVariables and
on the Variables
SamplePerformance
GREGeneralTest verbal, quantitative, and analytical ability scores and
a first-year averagegradewere available for all students. Data on undergraduate origin (U.S. versus other), age, and sex were available for mst
students. Several variables thoughtof as reflecting different aspects of
"acquired ESLproficiency" were also employed:
total score on the Test of English as a Foreign Language(TCEFL)
the discrepancybetweenobsenred GREverbal scores and the scores that
wuuldbe predicted for U.S. examineeswith given quantitative scores
(a relative verbal performanceindex, or RVPI)
a similarly derived discrepancyindex for the analytical score relative to
the quantitative score (a relative analytical performance index, or
_I),
self-reported mglish languageconmmicationstatus (better ccmmmication
in English (BCE)than in any other languageversus the opposite status).
More detailed information regarding these mgliskproficiency-related
variables and their role(s) in the study, findings regarding the performance
of the Sampleon these and other study variables, and the intercorrelaticms of
study variables in the total Samplesampleare provided later in this section.
Scaled-scores(200 - 800) for the GREverbal, quantitative, and analytical rrreasures
were available frm testing after Septeber 1981. The predictive
properties of the traditional verbal and quantitative nrreasures
are well known.
Less is knownregardingpatterns of predictive validity in different
academic
areas for the analytical ability masure that was introduced in October 1981.
Positive correlations are expecteda priori for QIE scores and academic criteria. Negative validity coefficients for the GREare theoretically ancmlous
and, if found, usually may be explained in terms of sampling error or
selection effects.
The FYAcriterion was reported on the saw numerical scale by all departments, namzly,A = 4, B = 3, C = 2, D= 1, and F = 0. As is well known, the
FYAmetric is grading-contextspecific- that is, the Yevel" of academic performanceassociatedwith a given Fmcannot bebe assumdtobe comparable
In assessing GRE/FEJvalidity,
therefore,
across different departnmts.
attentim is focussed primarily on analysis of withitiepartmnt
relationships.
At the same tim, grades generally have the sam relative significance
across all grading contexts-for example, in departmntally heterogeneous
saqles, studentswith meanFw of 3.8 maybe as&
to be faring relatively
better academically,on the average, within their respective departments than
Thus, even whenconsidered without
studentswith meanFYA averaging3.0.
regard to departmnt of enrollment (grading context), the FYAcomeys useful
-12information regarding the academic progress and accmplishmnt of students.
It is assumedthat within-department grading standards mre comparable for all
students regardless of citizenship status.
Variables
Related to Ehglish Proficiency
TOEFLTotal Score
The TOEFLtotal
score (reported on a scale with an effective range between 213 and 677) reflects
performance on three separate measures: English
vocabulary and reading comprehension (with items that are mre general in nature and considerably less difficult
than the vocabulary and reading cqrehension items in the gre verbal test), knowledge of rules governing English
language structure and written expression, and a measure of mglish language
listening comprehension (m,
1983). In the present study, TOEFLtotal score
was treated both as a potential predictor of FYA and as a basis for classifying students according to general levels of mglish proficiency.
?DEFLtotal score is mderately highly correlated with GRZverbal score
in general samples of TDEFL/GRE test-takers. For a general sample of 3,808
ToEFL/GREexaminees tested during 1977-79 (Wilson, 1982b), the correlation
between these two scores was approximately .7O. Given this amunt of cwerlap,
the correlations of the respective measures with academic criteria might be
expected to be approximately equal. There is evidence suggesting that this is
a reasonable working hypothesis (see Sharon [1972] for evidence of parallel
patterns of criterion-related
validity for GE?E-Vand TQEFL; see also, Wilson
[1985] for evidence of parallel levels of validity for scores on the verbal
section of the Graduate Management Admission Test [G@YXT]
and ?DEFL, respectively, in samples of foreign ESL MBAstudents).
TOEFLtotal scores were supplied for only about 60 percent of all foreign
ESL students (N = 818). my
68 of the 97 department-level samples supplied
?DEFLscores for at least
five students.
Information was not available on
departmental 'IDEFL requirements, bases for exemption, and so on. Several departments with relatively
large numbers of students indicated that because of
clerical problems they were not providing 'IDEFL scores. Ihe uneven availability of ?DE~, scores across departments results in some interpretive
coqlications in analyses involving XXFL scores.
TOEFLscore estimated from --Verbal
score ('IDEFGEST). To provide an
estimte of ?DEFL performance (I0EFL-EST) for students for whcmTOEFL total
scores were not available, a regression equation for predicting 'IDEFL total
score (T') frcnn GREverbal (V) score was derived using data for the 1977-79
GRE/'IQEFLsarrrplecited abwe (GRE-V, man = 360, S.D. = 108; ?DEFLTotal man
= 559, S.D. = 63; r = -70): T' = 7IEFMBT(~)
= ,406 (V) + 413.
ToEFL/vERBAT,I.
For exploratory purposes, .a variable
was created by imputing TOEFLAZSTscores for individuals
scores.
called ToEFL/VEEB?%
without T0EFL Mzal
-13GPEVerbal Performance
Relative to Quantitative Performance
The discrepancybetweenobservedGEE-V score, V, and the predicted GRGV
score, V', that wouldbe predicted from GE?E+ score using a regression quation basedon data for U.S. examinees(called a Relative Verbal Performance
Index
or MI),
maybe thoughtof as indexing "degreeof English proficiency
I .
deficit" in the verbal performance
of foreign GREexaminees(Wilson, 1984a,
1986b).
For national contingents of foreign ESL examinees,meanIWPI values (a)
are systematically negative, (b) are typically relatively large in absolute
value, and (c) tend to vary with English-languagebackground.For ccmtingents
of foreign ENL(English-native language)GRE examinees,on the other hand,
man Kvp~values tend to vary aroundzero, as would be the case for Samplesof
U.S. examinees.
The predicted GE?E
verbal score used in calculating the PVPI was coqmted
using a regression equationbased on data for U.S. mathematicsand physical
scienceexamineestested during 1981-82 (GEE-V,mean= 520, S.D. = 109; GREiQ
= 645, SD. 104; estimated r = .50), described in detail elsewhere (Wilson,
1984a):
Predicted verbal = v' = -52 (Q) + 185, ad
RVPI =V-V~,*reV=absemedverbal
score.
In the present study, the FWPIbiastreated as a potential moderatorof GEErelationships.*
GEEMalvtical Performance
Relative to Quantitative Performance
Proceduresanalogousto those used in deriving the PVPI wre employed to
derive an index of the discrepancybetween observedGREanalytical score, A,
andthepredictedanalytical score, A' (the score that wouldbe predicted for
a U.S. examineewith a given quantitative score). This discrepancyindex was
called the Relative Analytical Performance Index, or RANPI.The analytical ability measurecalls for relatively extensive verbal processing, albeit of a
* In a study involving GM&Tverbal and quantitative scores for MBA students
frcm 59 schools(Wilson, 1985), a comparablyderived PVPI measure (reflecting
GNRT-V
score relative to -T-V
score predicted from @IAT using an equation
for U.S. GMAT
examinees)was analyzedas a potential mderator variable. When
ESLMBA studentswere classified accordingto roughly the top, middle, and
bottmthirds on the RVPI index, the GMA!l?/FYA
correlations were found to be
highest for examineesin the top third on the GRT II
(assumed
to have the
lowest for those in the bottm third
least English proficiency deficit),
(assumed
to have the greatest English proficiency deficit, and in between for
This
students in the middle third (with mediumEnglish proficiency deficit).
relationships in
finding indicated that the GMATRVPI"tierated" CXWT,/FYA
samplesof foreign ESLMBAstudents.
-14mre specialized character than that involved in the GRE verbal test. The
MI,
which has not been used previously, was thought of as possibly indexing
a scarrewhat
different type of "English proficiency deficit" for the foreign ESL
population than that indexedby the RVPI. In the present study R?WI was
treated as a potential tierator of GRE/FYA
relationships.
The equation specified belclwwas used to obtain predicted analytical
score (A').
The equation was basedon data for the sa@e of 1981-82 U.S.
examineescited above, without regard to-field of study: analytical mean =
520, standard deviation = 124; quantitative 11y3a.n
= 521, standarddeviation =
132; estimated Q,A correlation = .65.
Predicted analytical
= A’ = .611(Q) +
RANPI =A-A’,
wkreA=ohervedanalytical
202, and
score.
Self-Reported English LanguageConwnication Status
GREbackgroundquestions include, "Doyou coamMlicatebetter in mglish
than in any other language?" For study purposes"Yes"= SR-ECE(self-reported
better c-ication
in English) status = 1; "No" and "no response" = 0. Foreign ESLexamineeswho report BCEstatus typically are frm nonnative-English
speakingcountries with a strong mglish speaking tradition. Ihey typically
are not natively proficient in mglish but tend to be rrroreproficient than
their ESL counterparts who report better corrnnmi
cation in a language other
than English, as indicated by higher averagescores on the ?DEm,and the GRE
Verbal Test, for example. However,the ESGBCEexamineeseamlower average
verbal scores than foreign m
(mglish native language)examinees (Wilson,
1982b).
Other Variables
The I~EFL total n~an reported by EI'S (1983) for all 1980-82 -takers
from a given country was ascribed to each student from that country in the
present study -le.
This variable, called TOEFGLEVEL
(see Tables 3 and 4
and related discussion), was thought
of as reflecting differences in Englishlanguage and educational backgroundassociated with national-linguistic
origin.
Information regarding sex, age in years (as of October1982), and undergraduate origin (U.S. versus other location) ws available for rt0st students.
Rxfonnance
m the St&y
Variables
Table 5 provides sunwry statistics (nwns and standarddeviations) for
the variables described in the precedingsection for foreign ESLstudents, by
broad academicarea. For perspective, statistics are also shawnfor the total
sa@e (N= 42) of foreign ENL (English native language)students.
-15
Table 5
for the ESLSaI@e on Selected StudyVariables, by
BroadAcademicAreas, with Cmparative Data for All EI\TL,
Students*
Summy Statistics
Variable
ESLsaqle
Total
Quant
Biosci
Sot Sci
R% sample
Total
3.18
0.52
3.44
0.42
3.55
0.44
total
FYA tfQ=d
(Standarddeviation)
3.45
0.42
3.49
0.41
GREVerbal
382.0
105.6
384.4
106.1
375.5
88.1
351.5
103.3
546.0
131.4
GRE Analytical
485.6
107.0
492.1
105.6
450.5
108.1
414.5
94.9
591.7
126.8
GREQuantitative
684.3
98.7
698.4
595.8
81.3
127.3
541.3
148.3
684.5
113.4
Relative Verbal
-158.9
PerformanceIndex (RVPI) 106.7
-163.8
104.7
-119.4
106.3
-114.9
118.9
5.0
126.7
Relative Analytical Per- -134.2
formanceIndex (RZWPI)
92.4
-136.2
91.7
-115.2
94.2
-117.9
99.0
-28.2
106.7
mEFLTotal.,
566.9(a)
44.7
568.0(b)
44.7
549.7(c)
43.0
549.6(d
40.7
TCEF'LEstimated
568.1
42.9
569.1
43.1
565.4
35.8
555.7
42.0
from GREVerbal
N.A.
634.7
53.3
‘
TCEmvL(country~ans
ascribed to students)
510.2
28.7
510.7
29.0
504.0
20.4
506.8
28.2
N.A.
0.16
0.20
0.16
N.A.
AttendedU.S. Undergrad-
0.16
0.16
0.22
0.14
0.12
0.17
0.13
0*X?-
0.61
0.31
uateSchool=l
Sex (M = 0, F = 1)
Age (in years, as of
Rtober
1982)
25.8
4.0
25.4
3.7
27.2
4.5
29.8
5.3
26.3
6.4
* Meansin first row for all variables, standarddeviations in row 2 except for nominally coded (1,O) variables. Ns for groupswere, from left to
right, 1353, 1213, 55, 85, and 42, except for Age (total ESLN = 12861,
Sex(N= 1307), and T0EF'LTotal: (a) N = 818, (b) N = 771, (c) N = 22, and
td) N = 25. N.A. indicates not applicable for English-native-language
(m) students.
Diskette 546.84, working copy of 2, Docmmt 2
.:
. . _,
Self-Reported Better Com- 0.16
mnicator in English = 1
.
:‘I.
>
._
.*
-5
First-Year Averaqe Grade
For the total saqle of foreign ESL students, meanFw was 3.45;
for
those in quantitative
fields, the meanwas 3.49. For the foreign ENL students, man FYAtams 3.55. Such meanFw levelsare
cmsistentwith
those
reported by ETS for saqles
amposed pmdanhantly of U.S. citizens. The mzan
of 3.18 for students in bioscience departments is based on data for only 55
students from six departments.
o The GRE Validity Study Service (VSS) at ETS reported naedian first-year
averages of 3.39 for 49 physical science departments, 3.53 for 33
bioscience deparbnts,
and 3.51 for 102 social science deparwnt-level
samples, respectively (Burton & Turner, 1983). Although very few of the
deparents participating
in this study were included arwng those that
participated in the GREVSS, the comparison suggests that these foreign
ESL students, especially those in quantitative fields,
probably earned
first year grades c-r-able
to those of their U.S. counterparts.
GREGeneral Test Performance
Several trends are noteworthy (see Appendix A for department-level data) :
0
The quantitative
ability meanfor all foreign ESL students and for those
in quantitative departments was above the eightieth percentile for basic
GREreference groups (e.g.,
EI’S, 1985, p. 17). For all major areas,
including the social sciences, the basic pattern of abilities
was the
same: highest on quantitative,
lowest on verbal, with analytical
in
between.
0
Foreign ESL students and foreign ET\JL
students had identical -meanson the
GREquantitative
measure, but the verbal and analytical mans of ESL
students were muchlower than those of ENL students.
0
The large negative KVPI and RAM?1means (-158.9 and -134.2,
respectively)
performfor foreign ESL students indicate average verbal and analytical
ance markedly lower than would be expected for U.S. examinees with GRE
quantitative scores averaging 684. Note that the verbal and analytical
mans for ENL students were quite consistent with expectation for U.S.
examinees: mean XVPI =5.0
andnEanRANP1 = -28.2, discrepancies that may
be accounted for by sanpling effects.
For perspective, the &ians of the distributions of meansof GRE verbal
quantitative scores for 57 physical science departrrrentsparticipating
in
the GREVSS through June 1982 were verbal = 528 and quantitative = 687 (Burton
Turner, 1983, p. A-l) as cornparedto 384 and 698 for ESL students in the
present study. The median analytical IIED for GREVSS participants was 588
(for scores on the pndktabe r 1981 analytical test only).
-17TCEFL
Performance
&out 60 percent of the total sample but proportionately fewer of those
in bioscience and social science Sampleshad 'IDEm scores. TheXEFL total
meanof 567 indicates that the studentswith TWFL scoreswere highly selected
in terms of the aspectsof English proficiency measured by this test-the
average student was at approximately the eigh*fcmrth percentile for all
graduate-level TOEFL
examinees(EI'S, 1983).
The TWFL-EST mean, 568, indicates the man of the distribution of
estimated XEFL scoresfor studentswith the sameGRE verbal meanas that of
the present Sample.The close agreementbetweenthe TWFL and TOEEGESTmeans
indicates that studentswithout ?DEFL scoreshad GEEverbal scores camparable
to those with XEFL scores. This suggests that, on the average, students
without TDEFL scoreswere roughly camparableto those with XEFL scores in
tern of aspectsof "mglish proficiency" tappadby the c;REverbal measure.It
is of incidental interest to note that the estimted 'IDEm,score for the ENL
students was approximately635, a value higher than the TOEZ'L
meanattained by
any contingent of U.S.-boundTWFLAakers.
The high TEFL man for the foreign ESLsamplereflects in part the fact
that foreign ESLapplicants typically are screenedfor "English proficiency."
Presumably,those admittedare judged to have either (a) at least a minimally
adequatelevel of proficiency in English to begin academicstudy full-tirlle or
part-timz and/or (b) a reasonablelikelihood of being able to acquire a
minimally adequatelevel given a period of intensive ESLinstruction.
However,the high TOEFL
man is also attributable in part to the fact
that lDEFLexaminees
whotake the GRE,on the average, are muchmore proficient in those aspectsof mglish proficiency masured by the %XFL than T0EFL
examineesgenerally.
o For scm 3,808 YCDmm examineestested during 1977-79, for example, the
TOEFLtotal meanwas 559 (Wilson, 1982b) as ccaupared
to a meanof 508 for
all graduate-level 'IDEm,examineestested during 1980432 (GI5, 1983). The
lDEFLman of 559 correspondsto the eightieth percentile rank in the
distribution of scores for all graduate-level TOEFL examinees tested
during 1980-82.
Results of surveysof institutions to determinescore levels on the TOEFZ
that are associatedwith various types of admissions decisions (e.g., FI5,
1983), indicate that scoresin the 550 range frequently are judged to be
sufficient for unconditionaladmissionto academicprogramsfor JZSLapplicants
judg@ to be academicallyqualified-that
is, for beginning academic work
without concomitant participation in intensive ESL instruction or other
activities designedto improvemglish proficiency. Thus, it is possible that
mny if not mst of the ESLstudentsin the present Samplemayhave surpassed
a "threshold" level of proficiency required for academic functioning in an
English-languageenvironmnt-especially as students in fields that emphasize
primarily quantitative abilities, internationally employednotations, specialized English vocabulary,and so on.
-18Other Variables
The TWFL-m
~llean for the
ESL sample was 510-close to the grand mean
for all U.S.-bound TWF'L examinees as reported by ETS (1983). To reiterate,
TOEF'L-LEVEL
was derived by ascribing to each student in the Sample frcun a given country the 'IDEF~total
mean of individuals fromthatcountrywhotook
the
ToEFL during 1980-82, as reported by E'I'S (1983).
About 16 percent of the ESL students reported BCE status (better commmication in English than any other language), and the samepercentage reported
attending a U.S. undergraduate school.*
Sme 17 percent of the students were female. Average age for all ESL
education)
students was 25.8 years; students in social sciences (primarily
and those in biosciences were
were considerably older (mean = 29.8 years),
somewhatolder (mean = 27.2 years) than those in the quantitative fields (man
= 25.4 years).
Intercorrelations of Selected Variables in the Total ESLSaqle
Table 6 shows intercorrelations
of designated study variables
in the
total ESL sample; intercorrelations
of GEE scores and the two measures derived
from GREscores (RvpI and RANPI) are shownfor the total foreign EM., Sample.
The correlation between GREverbal and TEFL in the sample of ESL students
(.71) was almost identical with that (.70) found in the general sample of
'IDEFL/GR examinees referred to earlier.
The a.nalyticalItleasure and the RANPI as compared to theverbalmeasure and
the RvpI had substantially lower correlations
with KEFL total
score,
T0EETAEVEL, and self-reported BCE status. mis suggests that the type of
ESL facility
masured by the verbal test is not very similar to that
RVPI and RANPI were not strongly
rcleasured by the analytical
test.
correlated (r = .41).
Differences on other study variables betwen students with T0EF'L scores
and those without TWFL scores are pointed up by coefficients
for a
variable called yEs-ToEFL( (1 = student had a score versus 0 = no score
available). Students without T0EFL scores, on the average, were (a) quite
similar to those with TOEFLscores in terms of mean score on the verbal
andanalyticalmeasures
(I: = -00 and r= .03), (b) slightly lower, on the
bt
average, with respect to KVPI and FMPI (low negative correlations),
(I: = .12).
(c) somwhat higher in quanti- tative ability
straxpt single correlate of YEG-m was location of udergrdde
Students not reporting a U.S. undergraduate school
school (r = -36).
The
*Both of these are underestimates of the actual percentages of students in the
respective statuses since they represent the percentage of all students, including somefor whomdata wre not available.
-19Table 6
Intercorrelations of Selected StudyVariables in the Total ESLSample
lbtal
V
Variable
GRE47
E
A
Q MI
correlation matrix
RAN-ToEFLYEsm~m-
PI
Age
total TFL Level BCE
-.07 -.03
-.05 -.04
-.27 -.16
-52 -26 -.05
.25 .04 -.24
-06 .05
.12 .06
.90 -26 -.15
-39 -m-.01
RVPI
RANPI
U.S.
SCh
.46 .22 .a8 .39 -71 .OO .52 .26 -.la
-21 .a3 .4a .03 .22 .03 -.35
--51 -.26
-.07 .25 .12 -.Ol -.Ol -,26
,54
so 3
sex
.41
-42 .60 -.06
.41 -.05
TmFLTotal
YES-TFL=1*
9Dm-LEVELI
SR-BCE=l
Sex (M = 0, F = 1)
U.S. undergraduateschool= 1
-
.oo* .50 .2a
.oa -.ol
.35
-
-.20
-02
-.13
-.ll
-.04 -.05
-.06 -.36
-.04 -.12
-.03 .Ol
-.Ol -.20
-.Ol
-
N = 1353 for coefficients above diagonal except those involving Age
(maximumN=1286), Sex (N = l307), U.S. undergraduateschool (N = 1291), and
XEFZ Total (N = 818). -tries below the diagonal are coefficients involving
GREscores, or variables derived using GREscores, for the total sa@e of
EM1,(English-native-language)students (N = 42).
Note.
*YES-TFL=
YES-TOEFL = TOEFL TWal.
score available = 1; not available = 0.
-2o-
were mre likely to have TOEFLscores and vice versa. This is consistent
with the fact that a record of successful performance in a U.S. school is
frequent- ly accepted as evidence of adequate ESL proficiency-applicants
with such a record may be exempted frcnn taking TOEFL(FTS, 1983).
0 It
is noteworthy that FSL students who reported a U.S. undergraduate
school tended to have lower GREscores, especially GRE quantitative scores
(r = - -16) than their counterparts who did not do so.
0 Correlations in the .5 range for GREverbal, RVPI, and TCEFL total,
respectively, with ToEFLrLEvELindicate that classification of students with
respect to level of performance on these measuresof verbal skills
would
result in substantial-incidental
sorting in terms of background variables
that are linked to country of citizenship through the TOEFGLEVELscore.
0 The pattern of coefficients for self-reported EKE status with th verbal
test measures was like that for TOEFTAEVEL, but relationships were
considerably weaker.
-21Section III:
StudyN&hodsand Procedures
Data were available for 97 small samples of ESLstudents predcminantly
frm quantitative fields:
engineering, mathematics, chemistry, physics,
statistics, computerscience, and economics.Cmplete GE?Em data were also
available for small Samplesfrom five bioscience departmnts and six social
sciencedeparmnts (five education departmmtsand one political science
departnmt). Also available on a ccmpletf+databasis were -LEVEL
scores
and scores on ti nominally coded(1,O) background
variables: U.S. undergraduate school versus other status, and self-reported better cmnunication in
English (SR-BCE) versus other status. ?DEFLscores were available for at
least five stw dents in only 68 departmmts.
The present st&y was concerned primarily with assessing the typical
levels of within-departmnt GREP relationships for foreign ESL students
generally and in various subgroups. Estimates of GREvalidity coefficients
basedm-single small samplessuchas those available for study wouldnot be
reliable. Homver, by pooling information from all the small saqles, reliable
estimtes of the -ical.
withitiepartmnt
correlation beimeen GRE scores,
Fw, and other variables can be obtained by analyzing interrelationships amng
departmentally standardizedpredictor and criterion scores in pooled sa@es
of students from "similar" departmnts-that is, by analyzing pooled withindepartmmtdata matrices. Coefficients in these rrratrices sumarize basic
trends in the distributions of departmmt-level GEE- coefficients.
It is convenient to standardize the predictor variables as well as the
criterion variable whenattention is focussedprimarily on assessing trends in
within-departmmt predictor/criterion correlations (as in the present study)
*
rather
than on the develapnent of operational predictive equationsfor use m
I
particular departnmts. In the present study, data for U.S. studentswere not
available for analysis.
CeneralMethmMogicalRatimale
Given ccmmn GRE,/FyA(predictor/criterion) data sets for a relatively
large numberof small Samples of individuals in "similar" graduate programs
being offered by departmmts in different graduate schools, it is possible to
drawmeaningfulinferences regarding general levels and patterns of GRE/Fw
(predictor/criterion) correlation and the relative weighting of predictors by
pooling data from all the "similar" settings. ?he estimates derived from the
pooleddata maybe thought of as approximatingpoprlation estimates.
Om useful pooling procedure involves standardizatim of the predictor
and criterion variables of interest within each departmnt-level sa@e prior
to pooling (see, for example, Wilson, 1979, 1982c, 1985, 1986a, 1986b). In
this approach,original or raw scores on the respective variables are subjected to a z-scale transformation-raw scores are expressedas deviations from
departmentmeansin deparmnt standard deviation units and are thus transfonmd to a ccmmnscale with meanof zero and standarddeviatim of unity in
all sarqles.
-22o The intercorrelations amongthe departmentally-standardized
variables for
combinedsamples from several departmmts constitute a pooled, withindepartmentcorrelation matrix. Validity (or other coefficients) based on
pooled departmentally standardizeddata for several departmentsare quivalent to size-adjusted meansof corresponding
coefficients for the smaller, department-level samples. Theuse of size-adjusted averagesof correlation coefficients to sumnarizeresults of comparableanalyses that
have been conductedin different samplesis a well-established iota-analytic technique (e.g., P&teller & Bush,1954). The contribution of a particular departmntal data set to pooledwithin-departmentvalidity estimtion is a function of sa@e size.
o There is reasonto believe that nwh of the variability in observedvalidity coefficients for cmmn predictors and criteria across "similar" settings is due to statistical artifacts rather than setting-specific differences in "criterion content." For exaxtple, in an analysis of 726 lawschooLvalidity studies, Linn, Harnisch, & Dunbar (1981) estimated that
about 70 percent of the variation in school-level validity coefficients
across studies PEGaccountedfor by differences in samplestandard deviations, estimated criterion reliability,
and sax@e size, respectively.
Similar findings have been reported for validity studies involving commn
selection tests and job performancecriteria in occupationalsettings (for
exa@e, Pearlman,Schmidt, & Hunter, 1980). In the &operative Validity
Studies Project (Wilson, 1979), in analysesinvolving over 40 departments
in five disciplines the majority of departxtmt-level regression coefficients for GRE predictors were fomd not to differ significantly from the
pooled within-department coefficients.
Statistically significant deviations could be accountedfor by clear outlier effects in small samples.
Thus, predictor/criterion correlation coefficients (validity coefficients) based on departmentally standardizeddata pooled across -departmnts
within various disciplines- that is, coefficients in pooled, withikdepartmnt
data matrices-may be thoughtof as approximating populationvalues, around
which department-level coefficients maybe expectedto vary, due primarily to
statistical and sampling considerations(sample size, degree of selection,
criterion reliability, and so on) rather than context-specific validity-related considerations such as real differences in the content of the criterion.
For example,economicsdepartmentsmay differ with respect to the amunt of
coursework in quantitative methodologytypically required during the first
year of graduate study.
westions regarding the relative contribution of GREscores and other
variables in predictive corqositesmaybe addressedby applying mltiple
regression methodsto pooled, within-departmentdata matrices. Standard partial
regressionweights for GREscores and other independentvariables, and mltiple correlation coefficients, for exaqle, as well as simple correlations,
basedon departmentally standardized data for several departments,
within various disciplines, maybe thought of as approximating popE%
values. If one is concerned with developing regressionequations applicable
for small department-level samples,it has been foundthat tien departmentlevel regression coefficients are adjusted towardcorresponding '@@ation"
values, the resulting equationsgeneratemre reliable predictions in subse-
-23-
quent samples than equationsbased solely on local data
(see, for
exanple,
Braun and Jones, 1985) .
The foregoing interpretative rationale for pooled, within-department
estimates of correlation or regressioncoefficients rests on an assumption
that the departints for whichdata are pooled are generally similar with
respect to the nature of the academictasks that students are required to
complete. The programsof study offered by the deparwnts for which data are
pooled should require the exercise of generally similar patterns of skills,
abilities, and so on--especially the types of skills and abilities represented
b the predictor variables of interest.
For academic departments, a logical initial
criterion for pooling is
academicdiscipline or field.
A nwe cqrehensive pooling rationale wculd
involve the reasonable asswption that tasks required of students by departments representingdifferent fields or specializations within the same general
academicarea are generally similar in relative dm
on, say, verbal as
opposedto quantitative skills. Academic
programsin English, history, political science, and so on, maybe as&
a priori to makegreater demandson
verbal abilities than on mathematical or quantitative abilities; for programs
in mathematics,chemistry, physics, and so on, the opposite is true.
This line of reasoningleads to the a priori expectationof higher validity for quantitative scores than for verbal scores in the primarily quantitative fields and the opposite pattern of validity for these masures in the
for
primarily verbal fields. Observeddifferences in patterns of validity
verbal and quantitative scoresare consistentwith a priori expectation (see,
for example, Willingham, 1974; Wilson, 1979, 1982c, 1986a, 1986b; Burton 61
kss
is known regarding patterns of predictive validity
Turner, 1983).*
across disciplines for the analytical ability masure.
pgplicatim in
the
Present Study
Sumnarystatistics (rtwns, standarddeviations, and intercorrelations)
were computedfor study variables within each of the 97 deparmnt-level
Samplesof ESLstudents. The EYAcriterion and original scoreson the GREand
other continuousvariables were z-scaled by department-the resulting departmnt-level distributions had equal means(zero) and standarddeviations (1.0).
me original scores of foreign ENLstudentswere z-scaled using parametersfor
the foreign JISLstudents.
* Braunand Jones (1985) reported that clustering deparbents empirically on
the basis of patterns of samplerrwns on the respective GEEmeasures provided
a useful basis for aggregatingdata for the purposeof estimating regression
coefficients for the GEEscoresfor Embers of a cluster. Clusters thus fommed
empirically mayinclude samplesfram different disciplinary areas, but will
tend to correspondbasically to a priori clusters based on disciplinary affiliation that differ primarily along a verbal-relatiw+toquantitative-er@asis
dinwrsion.
.
-241. To evaluate levels
and patterns
of GRE/EYA correlations
in general
samples of ESL students,
means- of departint-level
validity
coefficients
(weighted by sample size, or size-adjusted,
unless otherwise specified)
were
computed for each of the departments and classifications
of departments designated below:
0
Chemical, civil,
electrical,
Engineering, total
0
Statistics,
chemistry,
physics,
mathematics,
Mathematics and Physical Science, total
industrial,
mechanical,
engineering,
computer
science,
and
and
Economics
Quantitative,
total
(Engineering
+ Math/Science
+ Econtics)
Biosciences (total)
Social sciences ( total)
.: .
In view of the very limited representation
of departrrrents from bioscience
and social science fields,
primary eqhasis was placed on analysis of data for
the 86 departments from primarily quantitative
fields.
2. To obtain general (population)
estimates of the relative weighting
of
GRE scores in composites for predicting EYA, multiple
regression
analyses were
conducted using pooled matrices of departmentally
z-scaled Fw (zfya) and GRE
(zgre) data for several groups of depar-nts:
(a) engineering
(b) mathematics
and physical sciences, (c) economics, (d) quantitative
total,
(e) biosciences,
and (f) social
sciences. The simple correlation
between a standard c-site
of zgre predictors,
based on a regression
equation developed for the total
quantitative
Sample, and Z@a was c-ted
for each department-level
Sample.
Means of validity
coefficients
for
the composite predictor were compared with
mans for the individual GRE predictors to assess the potential
for incremental validity
in a uniformly weighted general coqosite.
The TOEFIAEWL score ws included as a supplemental predictor
in certain
of the titiple
regression analyses to assess the possibility
that this
variable, tiich was thought of as reflecting
differences
in English-language background linked to country of citizenship,
might have incrmntal
validity
when
included with GE?Escores.
3. Multiple regression analysis
of departmentally
z-scaled data,
pooled
for subgroups of students from quantitative
departints,
was employed to
explore the possibility
that GRE/FyA relationships
might tend to vary across
subgroups of foreign
ESL students judgd to differ
in “level of ESL proficiency” and Bnglish-language background, as defined by:
a. score level on the Relative
b. score level on the Relative
c. TOEPL total-score
level,
Verbal Performance Index (RVPI),
Analytical
Performance Index (RANPI),
-25
d. self-reported
BCE status versus other status.
These analyses, incidentally,
provided information regarding the average
relative withindepartmnt
standing of students in various subgroups as reflected in the mans on z-scaled QV3 (Zgre) scores and the z-scaled J?YA(Zfya)
criterion-that
is, mans on Zq, Za, Zv, and Zfya, Observed man Zfya for the
proficiency subgroups involved were compared with estimated or expected means
(z'fya) based on the departmentally z-scaled zgre scores, using general regression equations based on pooled z-scaled data for all departments involved
in the analysis. These cmparisons provided a basis for tentative
inferences
regarding the coqarative performance of various subgroups.
4. Are within-department predictor/criterion
relationships stronger in
groups that are homgeneouswith respect to national origin than in samples
that are heterogeneous with respect to national origin?
To evaluate this
and a stanquestion, coefficients reflecting
the relationship
between Zfya
site
of
Zq
and
Za
scores
were
computed
for
samples
of
student%&a
by country of citizenship and world region and for Samples classified
by world region and type of quantitative
department--engineering,
mathscience, economics-as well as for the combined sample of foreign ESL students
from all quantitative departments.
Consistent with the primary objectives
of the study, the foregoing
analyses were designed to provide evidence regarding the typical levels of
withikdepartmnt
relationships between the GRE and F&Avariables for foreign
ESL students and the effect on these relationships of intrticing
controls for
"levels of mglish proficiency," variously defined, and/or country of citizenship. These analyses are described in detail in Sections IV through VI.
'I8nestudy was not designed to address questions regarding the comparative
academic performance of students frm
different countries or regional groups,
or the extent to which level of academic performance was consistent with level
of GREperformance. Limited analyses related to these questions were possible,
however. These analyses and related
findings are described in detail
in
Section VII of this report.
-26-
-27Section IV: GRE-
Relationships For ForeignESLStudents,
by Academic
Area
This section presents findings regardingCRR/TAcorrelations for foreign
ESLstudents in departmnts classified by field and in broaderarea classifications. Size-adjusted nw.nsof department-levelCR.E/FYA
correlation coefficients are shownfor various classifications. Results of regressionanalyses of
z-scaled FYA (Z
) on the z-scaled GRE(*e)
predictors (Zq, ZQ Zv), using
data
departmentswithin broadacademicareas, are' presented.
Also reported are trends in departmmt-level correlations between FYA and
three nonGRE variables:
U.S. versus other uder~b
origin,
self-reported better cammicatim i.nEnglish(ECE) versusotkrstabs,
ad
YJEEZFLS
InterpretivePerspective
me of the principal questions implicitly at issue in this study is
whether GRE/TYAcorrelations for samplesof foreign ESLstudentsare similar
to those typically obsenredfor samplesof U.S. students. Since data for U.S.
students were not collected for the departmentsin this study, it is useful to
review briefly findings regarding typical levels and patterns of GRE validity
for predicting academic criteria in samples composed
exclusively or predaminant1y of U.S. students.
There is a substantial bodyof evidenceregarding the relationship of
scores on the well-established verbal and quantitative ability measures to
performancein graduate study, typically measured
by first-year average grades
(for example, Willingham, 1974; Wilson, 1979, 1982c; Burton& Turner, 1983).
Evidence regarding the predictive validity of the analytical ability measure
introduced in October 1981, and not madeoperational until October1985, is
muchmrelimited.
midence regarding tyqical levels and patterns of validity for the verbal
and quantitative ability measuresis basedon cumulativefindings for several
hundreddepartments. Ccmsistentwith a priori expectation, GRE quantitative
scores are mre valid predictors than GREverbal scores in quantitatively oriented fields suchas chemistry, engineering, mathematics,and so on, while the
opposite pattern typically hoids for verbal fields suchas English, history,
sociology, political science, education, and so on.
o In w cooperative studies, each of which involveda total of 100 or mre
departmantsranging from highly verbal to highly quantitative, median
validity coefficients for verbal and quantitative scores, respectively, in
the mre verbal departments were -31 and -25 (Wilson, 1979) and -27 and
-25 (Wilson, 1982c). For "quantitative" departments in the respective
studies, typical verbal. and quantitative coefficients were .20 and -31 in
the earlier study and -18 and -28 in the later one. The earlier study
involved scores on tests administeredprior to October 1977, while the
later study involved scores on the "restructured" test introduced in
October 1977.
-280
For 118 social sciencedepartmntsthat participatedintheGRE
Validity
~tudy~ervice (VSS) atm
through&me 1982,thepooled wiupartmnt validity coefficients (size-adjusted averages) for GREV and GE?EiQ
were .26and -23, respectively; for 56 mathematicsand physical science
departments,typical coefficients for Q&V and GF&Q were J2 and -27,
respectively (Burton& mrner, 1983).
A ccqarablebody of evidence is notyetatibble
for the revised GEE
analytical ability masure introducedin October1981. Analytical scores have
been found to be positively correlated with first-year graduate schml grades
(Kingston, 1985; Swintcn, 1985) and with UncEergraduate
grades (Wilson, 1984c,
1986a) in a variety of fields,.
o Basedm findings reported by Kingston (1985), for graduate departmnts
sendingpost-September
1981 GREGeneral Test scores and Fw data to the
GKEValidity StudyService (VSS) atEYE, theamlytical
scorewas more
heavilyb&hted
than the verbal. score in predictive canposites for
aduate engineering and mathematicaJ.sciencesdepartmnts.*
Iiowever,
validityof
the
questions regarding* theincremntal and/ differential
analytical ability measure (for predicthEr of grades in quantitative as
opposedto verbal areas of study, for example) for U.S. or other student
groups remainunresolved.
Table 7 showssize-iadjusted meansof departnmt-level GRE/FYAcorrelations for foreign ESLstudentsin designated groupsof departments. For each
grouping,thenu&er
ofdepartmnts(samples)is
shown,alongwiththe total
numberof students an hich the coefficient is based (see Appe&ix A for distrihtims
of department--levelcoefficients and other department-level data).
*In
emluatingKingston% findingsregarding theanalytical
ability masure, ard in evaluating the coefficients
obtained
in
the
present
study
for this
.
I
.
*
StW
measure, it should bekeptm mmdthatdurmgthe period mbmhthe
ZiiEhvolved
were admittedtograduate school, testuserswereadvised
not
toconsiderthe
analytical score in admittingstudents.Thus, the predictI&
value of the analytxal score presumably
was not affected by restriction of
range due to direct selection, whereasrestrictIEi due to direct selection was
a factor affecting the c0ntriImtj.m ofboththe verbal and the quantitative
scores. The fact that it was not used directly in selection theoretically
shouldenhance the observedpredictive validity of the analytical ability
measureY%EiEve to thatof theverbal andquantitative ability IIy3asures.
Studies involving sa@es admittedafter O&ober 1985 will be neededto resolvequestions regarding the incremental contribution 0ftheGRE analytical
ability measureto prediction under conditims in which all three GRE scores
have been consideredin selecting students.
-29-
Table 7
Simple Correlation of GREGeneralTest Scoreswith FYA:Weighted
(Size Adjusted) Meansof Department-Level
Coefficientsfor DepartmentsGroupedby GraduateField andArea
GraduateArea
No.
No.
depts. students
QIE-Q
r
GEB-A
r
GRE-V
r
Ehsineerins
40
702
a9
-278
,114
Chemical
Civil
Electrical
Industrial
Mechanical
8
8
10
6
8
80
162
256
89
115
.315
.163
,296
.400
.347
.362
.088
.340
.342
.300
.173
-.047
.185
,165
,104
Matl@chce
36"
373
-351
,268
,023
Statistics
chemistry
Mathematics
Physics
Coquter Sci
5
10
4
6
10
54
111
39
51
113
.330
-353
-602
.452
.249
.425
.190
.266
.452
.220
.012
TO13
.Oll
-.013
10
138
-313
-282
,210
86"
1213
.311
,275
,097
Ecaxmics
AllQuantitatiw?
Bioscience
6
55
A61
-081
Sac Sci
5
85
-116
-184
.189
-.023
,253
Note. Ihe coefficients shownare "size adjust& averagesof departmentlevel coefficients (i.e., weighted according to samplesize) for samples
of five or mre nonnativeEnglish-speakingstudents.
* Includes data for one applied mathematics
department(N = 5).
Table Diskette 546.84 Dot. 18 page 1
-3oFor foreign ESLstudents in all subgroups
of "quantitative" departnts,
GRE quantitative scores and GRE analytical scores were IIy)re highly
correlated with FYAthan were GREverbal scores.
The basic pattern of GREvalidity across the respective types of quantitatively oriented fields is suggestedby coefficients based on the total
quantitative Sampleof 1,213 students, namely, ,311, ,275, & ,097 for
quantitative, analytical, and verbal scores, respectively,
notwithstanding
the fact that coefficients for analytical scores were higher than those
for quantitative scores in several fields.
The pattern of corqaratively stronger validity of the analytical
ability
scores relative to the validity
of the verbal scores in these ESL Samples
is consistent with Kingston’s (1985) findings for samplesfromwhich ESL
students (U.S. as well as non-U.S. citizens) were excluded.
For the swll saqle of 85 studentsfram five social science departments
(four frcan education, one from political
science), verbal scores were m>st
valid, and quantitative scores were least valid: coefficients
for verbal,
analytical, aJndquantitative scoreswere, respectively, ,253, ,184, ti
,116.
The pattern of higher validity for verbal than for quantitative scores in this
ESL sample in which education students predcaninated is consistent with that
reported by the GREVSS for 17 educationsamples (-26 and -19 for verbal and
quantitative respectively--Burton & mmer , 1983) .
For the six bioscience Sampleswith a total of 55 students, coefficients
for quantitative scoresand analytical scoresTniFere
positive but atypically low
andofaboutthe
samerqnitude (.061as c-red
to .081),while the verbal
coefficient wasanomalously
negative.
I‘he GRE/Fw correlations in Table 7, except for those obtained in the
limited biosciencesarqle, appearto be comparableto coefficients that have
been found for U.S. students, as reviewedat the beginning of this section.
Multiple RegressionResults for ForeignESLStudentsby AcademicArea
Table 8 showsselected findings of rcailtiple regression analyses based on
departmentally standardizeddata aggregatedfor broader groupingsof deprtIIy3nts:engineering, mathematicsand physical sciences, economics,all quantitative, biosciences,and social sciences, respectively.
These results indicate the limited contribution
of verbal scores to
prediction of FyA in the primarily quantitative
fields, with the possible
exception of economics.
The patterns of regressioncoefficients tended to be quite similar
the respective quantitative areas.
Coefficients based on aggregated data
for
for
those
the engineering, math/science, and econtics departments were similar to
for the combinedquantitative-department sample of 1,213 foreign ESLstudents.
-31Table 8
RegressionResults for ESLStudents Using Departmntally Standardized
Data Pooledby GraduateMajor Area
Correlation with FYA
Field/Area
N
R
Beta weight*
Q
A
V
Q
A
V
Engineering 702
Math/Science 373
Economics138
.29
.35
.32
-28
.27
.28
.11
.03
.22
-21
.29
-x
.19
-XT
-YE
.Ol
-.08**
.12
.33
.38
.38
All quark 1213
.31
.27
-10
2 24
.18
-.01**
.35
Bioscience 55
.06
.08
.03
-09
-.02
JO
Social Sci
.12
.18
.05
.08
85
-.02
* Standard partial regression coefficient.
statistically
significant at p < .05.
.25
underscored coefficients
** Negative weight indicates suppression; note positive
.27
.21
are
simple correlation.
-32V&lidityofaStar&rdGRE , Predictive
for Qmntitative Dfw--’
,
,
mite
The similarity in regression results for engineering, math-science,and
economics
departmentssuggested the potential utility
of a standardC%E predictive compositeincluding only quantitative and analytical ability scores,
with weights specified by regression results for the combinedsample of
students from all quantitative departments.
A coqosite of departmkally z-scaled quantitative and analytical ability scores (Zq and Za) weighted to predict z-scaled EYAin the combined
quantitative sanple was c-ted
for each student: ,238 Zq + ,176 Za. 'Ihe siqle
correlation betmen this standard compositeand the criterion was determined
for each departmnt-level sample.
Table 9 showsthe nreans(size-adjusted) of the department-level coefficients for this standard composite(last column of table) for quantitative departmnts classified by field and major area. man coefficients for the individual GREpredictors (from Table 7) are included for perspective.
Ihe resultsin Table 9 indicate that the standard c-site
had general
validity for predicting relative withirkiepartrtmt standing for departments in
the three general quantitative areas: engineering, math/science,and econmits. For these three areas, coefficients for the standard ccqosite were
sorrrewhat
higher than coefficients for the highest single predictor (typically
quantitative ability).
Obsenred differences between coefficient for the
compositepredictor and that for the best single predictor maybe thought of
as reflecting reasonable estimates of the amount of incremental validity
involved in using a compositeof m GREscores. (Again, it is1 important to
recall that the analytical score probablywas not used directly in selection).
Siqle Correlation of Selected NakGEEVariableswithFW
Wle
10 shows size-adjusted coefficients sumnarizing trends across
departmentsin the relative academic perfomanceof studentstie (a) reported
BCEstatus (better cmnunication in mglish than in any other language)versus
other status and (b) reported attending a U.S. undergraduateschool versus
Positive coefficientsir&catehigkr
manFyAforthose with
other status.
ECEstatus and for those reporting a U.S. umkrgraduate school. Size-adjusted
coefficients reflecting trends in the relationship be-n
-LEVEL
scores
and FyAare also shownin the table. The numberof departmentsinvolved in
the respective analyses varied; in several departments, no student reported
BCEstatus or no student reported a U.S. undergraduateschool.
The variable called IWFL-LEVELwas essentially unrelated to FYAin the
samplesstudied. In analyses not reported in Table 10, it was found that
TO&LEV& did not have incremental validity whenincluded in a battery with
the GREpredictors.
._.
-33Table
9
Validity Coefficients for a Standard Compositeof GREQuantitative and
Analytical Scores versus Coefficients for Individual GKEScores:
Size-Adjusted Averages of Department-Imel Coefficients for
Quantitative Departments Grouped by Fields and Major Areas
Graduate Area
No.
No.
depts. students
40
Chemical
702
GRJS-Q
GRE-A
GRE-V Carpsite*
r
r
r
r
-289
-278
-114
-330
Civil
Electricai
Industrial
Mechanical
8
8
10
6
8
80
162
256
89
115
.315
.163
.296
,400
.347
.362
.088
.340
.342
.300
.173
-.047
.185
.165
.104
,402
.155
-372
.430
.354
Pht?J/zience
36**
373
-351
-268
,023
-370
Statistics
Chemistry
Mathenmtics
Physics
CcmputerSci
5
10
4
6
10
54
111
39
51
113
.330
.353
.602
.452
.249
.425
.190
.266
.452
-220
.189
-012
-.013
.Oll
-.013
.422
.340
.521
.522
.283
10
138
,313
,282
,210
-370
,311
,275
,097
,347
Ekmanics
All(&antitative
86""
12l3
..
.
Note. The coefficients
shownare “size adjusted” averages of departmentlevel coefficients (i.e., weighted according to Sample size ) for samples of
five or mre foreign Eish-second-language
(ESL) students.
* The entries in this columnare size-adjusted averagesof departmnt-level
simple correlation coefficients between~(fya) and a starrbrd tmpsite
of
bpartmntallyz-scalea
GEE quantitative,
Z(q),scores
and analytical,
Z(a), scores, weighted according to results of the regression of Z( fya) on
Z(q) and Z(a) in the cabined sample of students (N = 1,213) from all quantitative departmnts: Predicted Z(fya) = Z'(fya) = ,238 Z(q) + ,176 Z(a).
** Includes one applied mthemtics
not shownseparately.
department (N = 5) for which data
are
Table 10
SimpleCorrelation of SelectedBackground
Variables with FyA: Size-Adjusted
Meansof Department-Level
Coefficients for Departmnts Grouped
by GraduateField andArea
!EEEbuwEL
Field
NO. (N)
depts.
40
chemical
8
Civil
Electrical
Industrial
Mechanical
8
10
6
8
702
r
self-Reporbd
BCE=l
No. (N) r
depts.
-035
37
679 -.005
37
606 -226
80 -.073
162 -.133
256 .152
89 .lll
115 .028
6
8
10
5
8
66 -.090
162 -.098
256 .048
80 .122
115 -.138
8
8
10
2
5
80
162
256
30
75
Ma-em?
36* 373 -.078
Statistics
chemistry
Mathematics
Physics
ComputerSci
5
10
4
6
10
54
111
39
51
113
-.052
-.117
.037
-A35
-.039
5
8
2
4
9
Ecmabs
10
138
-030
6
AllQlmntitative
U.S. undeqraduate
sdmol-1
No. (N) r
depts.
86*l2l3
A01
29* 301 -A94
54
90
26
27
104
-042
-.202
-.026
-.155
-.017
-.116
-.204
-.236
-.285
-.300
23* 243 -2l.O
3
5
3
4
7
36
50
34
38
85
-.030
-.420
-.288
-.188
-.093
98 -.207
6
99 --A71
71 1078 ~046
65
948 -a9
Bioscienaz
6
55 -.017
4
41
Al5
5
49 7029
SocSci
5
a5 -.017
5
a5
,101
5
85 -A42
Note. The coefficients shownare "size adjusted" averagesof departmentlevel coefficients (i.e., weightedaccording to samplesize) for samples
of five or mre nonnative English-speakingstudents. EEFbLIWEL= TOEFL
mans of U.S.-bound'IDEn examinees
by country ascribed to citizens of the
Self-reported KE = better
respective countries in the present sa@e.
ccmnmicationin mglish than in any other languageversus other status.
U.S. umbgraduate school= designateda U.S. schoolversus other.
* Includes one applied mathematicsdepartmnt (N = 5) for which data are
not shwn separately.
Size-adjusted mans of TOEFTLEVEL/FVA
coefficients were .035 (all
engineering samples), -.078 (mathematics and physical sciences), ,030 (econcnw
quantitative), -.027
(biosciences), and -.017
(social
its), -.OOl (all
sciences).
These coefficients indicate no systematic tendencies across departrwnts
for academic standing to vary with national origin as indexed by historic
country-level mans on the TOEFL. In evaluating this outccmEz, it should be
recognized (a) that only a very limited numberof countries could be rep
resented in the very small department-level Samples, (b) that the range of
background differences indexed by this variable was correspondingly restricted, and (c) that the particular mix of countries was not constant
over departznts.
Coefficients for BCEstatus indicate that there was no systematic tendency for students reporting better coxwwnication in English to earn better
the opposite pattern
grades than others. In fact, for quantitative fields,
tended to be slightly mDre prevalent as indicated by the negative coefficients. BCE coefficients were positive, but lo& for biosciences and social
sciences.
The size-adjusted meancorrelation for U.S. versus other undergraduate
This indicates
school with FyA was negative for every academic classification.
a relatively strong tendency across departments for foreign ESL students tie
reported attending a U.S. undergraduate school to earn lower grades, on the
average, than their counterparts who did not do so.
(fram Table 6) that
0 In evaluating this finding, it is useful to recall
students with U.S. undergraduate origins had lower average scores on all
three GFE variables,
especially on GRE quantitative, than their
counterparts who ccqleted their undergraduate studies elsewhere.
-36-
-37-
Section V: Zgre/Zfya Relationships for SubgroupsDiffering in
Performance on Selected English-Proficiency-Related Measures:
Pooled Z-scaled Data for Quantitative Departments
This section presents findings of regression analyses based on pooled
z-scaled data for subgroupsof foreign ESL students fram quantitative
departItlents only. Analyses were not conducted for students in bioscience and social
science departments.
Ihe analyses were concerned with trends in Zgre/ZQa relationships
across
subgroups differing in relative levels of "ESL proficiency" as measured by (a)
the RS7PI(relative
verbl performance irxkx),
(b) the RlsNpI (relative
analyL
tid.
perfonE3me inkx),
(2) self-reported better ~CationinEhglish
or
BCE status versus other stabs, and (d) scores m the XEFL, The question
generally at issue was tiether validity coefficients
would tend to be higher
for subgroups with higher levels of English proficiency
than for those with
lower levels, as indexed by the respective masures.
With regard to the three test variables, the coqoaratively low intercorrelations reported previously (Table 6) indicate that the aspects of "ESL
proficiency" involved in processing the verbal content of the analytical
ability items are not very similar to those involved in processing GRE verbal
or 'IDEXL items.
Procdures InvolvedinLkfiningSubgr~
For the INPI, the RANPI, and the TUEEX, respectively, score levels for
setin
such away thatif
the
"N#b " "nEdiun," and"lclw" subgraqswere
respectivescoredistribtions
wxemrmal,
abcxtone-third
of the stzu3ent.s
adbe
ineachgroq
This was approximately true for RANPI and TOEFL.
However, the distribution
of RVPI scores was skew3 posi~t~y~abuu~ce~
percent of all students were in the "low" category as compar
in the "high" category; all deparmnt-level
samples were heterogeneous with
respect to scores on the RVPI and RANPImeasures and sa
representation
in
each score-level was as&
for the majority of the samples.
With respect to the self-reported RCE stabs variable, fourteen of the 86
department-level samples included no student who reported better
comwnication in mglish.
This was taken into account by forming three subgroups for
analysis: an "English better" subgroupand an "other language better" subgroup
for students from deparwnts
with at least one RCE student, plus a third
subgroup including "other language better" students from 14 deparmnts
with
only no*BCE students enrolled.
TOEFLscores were unavailable for a substantial proportion of the students. Three TOEFGscore subgroupswere formed for 769 students from 75 departments represented by at least five students with GRE/FyAscores, at least
one of whomalso had a 'IDEm, score. For 10 departwnts in which at least one
but fewer than five students had TOEFLscores, the available ?DEFL scores were
-38-
z-scaled using paraxmters for TWFL/VEXBAL
(WEFL score or TCEFLscore estimatedfromGREverbal score). A total of 198 students frm the 75 departments
hadGREand~data,htno'IDEn
scores.
Results
Table 12 summrizes the principal findings of the regression analyses:
meansof Zgre scores and meanZfya for each subroup,sir@e Zgrefifya correlations basedon pooled z-scaled data for the subgroups,and mltiple
correlations for Zgre ccarposites,one involving only quantitative (ZQ) and analytical @a) scores and the other including all three GEB scores (ZQ, Za, Zv).
ZtoefljZ@a
coefficients as well as Zv/zfya coefficients are shownfor the
?DEFLsample. Findingsbasedon the total quantitative saqle are also shown.
0veral1, the findings do not indicate a consistent tendency for Zgre/Zfya
correlations to be "moderated"by "level of ESLproficiency" as defined by the
variables under consideration.
0
Levels of zgre/zfya relationships did not tend to vary directly with
levels of proficiency defined in terms of relative verbal performance,
r
relative analytical performance, or self-reported Ehglish camumcatmn
status. In analyses involving T0EFLas the classificatory variable, only
fore
subgroupwith ?DEFL scores of 585 plus (at about the eightp
seventh percentile for all graduate-level 'IDEm,examinees)were Zgrefifya
coefficients (-37, .36, ad -18 for Zq, Za, arxi Zv, respectively)
noticeably higher than those for the total quantitative sample(-31, -27, and
JO, respectively); correspcmdingmultiple correlations were -44 as
cmpared to .35.
Interpretation of correlational results for classifications based on
loEE%total score is ccqlicated by the fact that differences in Zgrenfya
correlation associated with "availability
versus nonavailability of XEFL
total score" were mre pronounced than those associatedwith differences in
score level amng studentswith TOEFLscores.
0 The miltiple
correlation for Zgre c-sites
in the "No TOEFL"subgroup
as comparedto multiple correlations of -36, .34, and
-44 for "Yes TWFL" students in lower, medium,and higher TDEFS-score
classifications, respectively.
wasorikyR
= 22,
There is no ready explanation for this unanticipated pattern of fir&q
regardingTDEFL. It is difficult to attribute the observeddifferences in
level of Zgre@ya correlations bebeen the "YeslDEJ?L"and "NoT0EFL" subgroupsto Engliskproficiency related factors. For example, the man relative
withinkiepartmntstandingof the "No IDEFL" subgrouponGE%Everbalwas
about
the saxwas that for studentsin themiddle TEFL-score range-man ZQa =
-0.09 as ccmpared
to -0.11). This suggeststhat the "NoVXFL" students (presumbly screenedfor ESLproficiency by other mans) were roughly camparable
to "Yes?DEFL"students in terms of the type of "ESLproficiency" measuredby
GEEverbal items. The "No TOEFL"and "Yes TWFL" subgroupswere also roughly
comparablewith respect to relative standing on analytical ability, but the
Table
Variable
#
(N)
Actual
Relative
Verbal
PerEormance
RVPI high
289
RVPI med 352
RVPI low 572
Relative
Analytical
Self-reported
BCE
English
Other
Other
TOEFL
better
better
better
Total
Less
Total
Score
Zgre-a
Zgre-a
Zgre-v
Zq,Za
(0.03)
(-0.07)
(0.01)
-0.15
-0.20
0.20
0.36
-0.15
-0.08
1.07
-0.02
-0.53
. 28
. 29
.35
23
132
.28
.15
. 16
. 11
.30
. 36
. 37
30*
136
. 37s
36
130
.32
.30*
Index
0.14
0.02
-0.18
(0.17)
(-0.01)
(-0.15)
-0.02
0.00
0.02
0.98
-0.08
-0.90
0.46
-0.04
-0.30
34
130
. 30
30
:25
. 24
.11
.02
. 10
198
883
132
-0.09
0.02
0.00
(-0.01)
(0.00)
(0.00)
-0.06
0.02
0.00
0.04
-0.01
0.00
0.51
-0.11
0.00
.35
.30
. 27
. 10
12
:04
.34
. 34
.34
27
:33
.40
.41*
252
265
252
198
0.09
0.15
-0.07
-0.22
(0.11)
(0.02)
(-0.07)
(-0.08)
0.17
0.11
-0.07
-0.27
0.40
-0.02
-0.31
-0.07
0.75
-0.11
-0.55
-0.09
. 37
.25
36
:21
.36
. 31
23
.: 12
18(.15)
112C.11)
.01(.12)
.06
. 43
.34
36
122
44(.44)
:34(.34)
36c.36)
:22*
0.00
0.00
0.00
0.00
.31
.27
.lO
. 35
.35+
391
499
353
(a)
(a)
(b)
1213
is
the
equation
0
bserved
mean.
based on data
The entry
in
for
the
total
parentheses
quantitative
is
z-scaled
sample
(N =
mean
1,213)
estimated
Zfpa(eet
:
Eor
.32
.38
.38*
Zgre-verbal
is
negative
(suppression)
in
this
composite.
from
) =
Zgre-q
and Zgre-a
+ .176
.238
Zgre-q
,
using
Zgre-a.
a
score
and CRE verbal
score
predicted
from
Relative
Analytical
Performance
Index:
A
score
predicted
from quantitative
score
communicative
either
better
in
ability,
departments
in which
at least
one student
better
communication
in a language
other
1111The TOEFL analysis
was based
on pooled
data
for
967 students
from 75 departments
with
five
least
one of whom also
had a TOEFL total
score.
A total
of 769 students
from
data,
at
TOEFL scores,
and 198 had GRE and FYA only.
Where TOEFL N was less
than
5 for
a department,
a variable
defined
as either
TOEFL total
score
(when available)
parameters
for
TOEFL/VERBAL,
for
students
without
TOEFL.
Coefficients
in parentheses
indicate
either
simple
Ztoefl/Zfya
tion
obtalned
when Ztoefl
was substttuted
for
Zgre-verbal
in the analysis.
FYA
Weight
.36
#I
the
observed
GRE verbal
# Relative
Verbal
Performance
Index:
The discrepancy
between
quantitative
score
using
a regression
equation
based
on data
for
U.S.
GRE examinees,
comparable
discrepancy
index
involving
observed
analytical
ability
score
and analytical
using
a U.S.
regression
equation.
Self-Reported
BCB etatue:
Self-reported
relative
English
or better
in some other
language;
(a)
categories
include
only
students
from
reported
BCE status;
(b)
Includes
data
for
departments
in which
all
students
reported
than
English.
*
Zq ,Za,Zv
Status
585+
550-584
than
550
No TOEPL
++ Initial
entry
general
reg ress ion
Zgre-q
Index
0.01
-0.04
0.02
Performance
RANPI high
RANPl medium
RANPI low
Zfya++
(Est)
11
or more students
with
complete
GRE/
these
departments
had GRE, FYA,
and
TOEFL was z-scaled
with
reference
to
or estimated
TOEFL (from
GRE-Verbal)
correlation
or the multiple
correla-
-4o“No TOEFL” subgroupwas lower in terms of quantitative ability
(by -0.27
withind-deparmnt standard deviations, on the average, and correspondingly
lower in n~an FYA(by -0.22 standardunits).
For the respective subgroups,as in the total quantitative sample,adding
the analytical score (Za) to the QREquantitative score (Zq) yielded sorts incrementin validity, but including the GREverbal score (Zv) or the ?OEFL
did not yield any incrementsin validity.
tctal score (Ztoefl)
cxnparativePerformance of subgrclups
By virtue of the processof z-scaling, mans of each department-level
Sample,and for all aggregationsof data involving intact deparent-level
samplesof foreign ESLstudents, were 0.00. For subgroupswithin a department,
or for aggregations of data involving selected members
of departxwntal samples, the meansof z-scaled variables indicate, in standard units, the average
deviation of subgroupmembers
from the mans of their respective departments.
For example, the Zfya meansin the KvpI analysis were 0.01, -0.04, ad 0.02
for nwbers of the high, medium,and low EWPI subgroups,respectively. The
"m3dium
FWPI" subgroupIlysan (-0.04) indicates an average FYA that was .04
standardunits belaw deparmntal FYAIIEW; other z-scaledmeans may be
interpreted in the safe way.
Predicted or expectedaverage standingon the z-scaled Fw (Zfya) criterionwas computedfor each subgroup based on z-scaled quantitative and
z-scaled analytical scores only, using the total quantitative sa@e regression equation: Predicted Zfya = Z’fya = ,238 q + ,176 Za = 0.00. These estimated means are shuwnin parenthesesfollowing the obsenred man Z@a for
eachsubgroup.
Overall, the average relative within4eparWnt academicstanding of the
respective subgroups
was generally consistent with expectation basedon their
relative standing on GREquantitative and analytical ability.
The lixnited predictive role for GREverbal is pointed up by the fact
higher and lower '%SLproficiency" groupsin every analysis were sharply
ferentiated in terms of meanverbal score (man Zv), but the direction of
Zfya differences MS not necessarily consistentwith the direction of
verbal score differences. This pattern is epittized by results of the
analysis,
that
difmean
the
RVPI
o Students in the high KWI groupand studentsin the low RvpI group had
z-scaled FYAs averaging 0.01 and 0.02, respectively; their FYAs were
typical for their respective departments. However,they differed by about
1.6 standard deviations, on the average, with respect to z-scaled GRE
verbal scores: high RVPI students averaged 1.07 standardunits above departmental verbal nwns while low KVPI students, typically, were 0.53
standard units below departmentalm-verbal means.
-41Section VI: Correlation of a Standard Zqre Predictive C-site
with
Zfya for Foreign ESL Students Classified by National-Origin
and Graduate Major Area-Data for Quantitative Departments
Results reported in preceding
idity (across types of quantitative
quantitative and analytical
scores
and ~a in the combinedquantitative
sections indicate substantial general valdepartments) for the standard composite of
specified by the regression of Zfya on Zq
-238 zq + -176 za.
sample--=ly,
This section presents data on the correlation between this general predictive composite and Zfya (a) in samples of students, (N > 9 only) from all
quantitative deparmnts combined, classified by country of citizenship and by
regions defined for the study, and (b) in samples classified by both region
and graduate major area-that
is, engineering, math-science, and economics.
These classifications
introduced some control for linguistic-cultural+ducational background variables
associated with national origin,
as well as for
type of quantitative major.
The analyses were concerned in part with assessing the consistency of the
relationship between a standard predictive c-site,
weighted to maximize the
multiple correlation
of Zq and Za with Zfya in the general quantitative
Sample, across the subgroupsdefined wholly or in part by national
origin.
Also of interest was the question of whether the correlation between Z’fya
(the predicted z-scaled FYA) and Zfya (the observed z-scaled value) would tend
to be stronger in subgroups that were harnogeneouswith respect to national-linguistic-cultural
backgroundthan in the heterogeneous general sample.
Table 12 shows simple correlations between Z'fya and Zfya for larger
COntingentS
within
designated regions and for all
StudentS
fra
countries in the respective regions.
These analyses are based on pooled data
for all quantitative departrrrents. Table 13 showscoqarable coefficients
for
students classified by region and by type of quantitative major.
MtiOMl
.. .
’
.:
.
As noted earlier,
the subgroupingwithin regions (for example, Europe I
versus Europe II,
Asia I versus Asia II) was designed to take into account
differences in characteristic
patterns of English language acquisition
and
usage and/or in the typical IDEFL scores of contingents of U.S.-bound students
from the respective countries. No country from Africa I or America I was
represented by as lMny as 10 students.
Students from Category I countries as camparedto those from countries in
Category II
are assurtled typically to have "richer English language backgrounds." The findings in Table 12 and Table 13 indicate that the introduction
of control for national-linguistic
origin did not have a systematic influence
on the level of relationship between Z'fya and Zfya,
0 Coefficients for Category I national and regional contingents in Table
appear to be camparable to those for Category II contingents.
12
l%le
Table 12
Correlation of a Ccnposite of Z-Scaled GREQuantitative (zq) amI
mlytical
(za) S&es with z-Scaled Fw (Zfya) for SubgAps
Defined by Region and Major Graduate Area
Correlation of a Camosite of Z-Scaled GREQuantitative (Za) and
Analytical (Za) S&es with z-scaled FYA (Zfya) for Sh@ups
Defined by Country and Region: All Qmntitative Fields
Country/Region
France
WestGelnlMy
spain
m-1
r
N
i:
.41
.ll
62
.36
.54
-.24
78
-42
.15
.28
98
rwh-
N
Ekm-
nanics
N
r
iii
Korea
Malaysia
Pakistan
Peoples’ Rep
Taiwan
Thailand
Asia II
146
::
41
294
20
.40
.32
*so
.23
.09
.49
.19
.30
643
.34
1213
.35
26
All StlKhts
17
.65
13
.60
Africa II
33
.40
America I
6
.29
AlKXic% II
N
.39
13
L&anm
Iran
Chile
Cola&da
Mexico
N
Science
Greece
mrkey
mrcqm II
13
.39
.43
.35
Note. lbe predictive -site
was
specified by .238 (Q) + .176 @a),
with weights based cm the regression
ofZfyaonzqardZainthepooled
matrix of departmntally z-scaled
data for students fran all quimtitative departmnts. Coefficients tabled
indicate the sinple correlation
betweenthe predictive rmposite and
Zfya in the s&q-cmps designated.
.32
Coefficients are shown by country only
for countries represented by 10 or mre students in the total quantitative
sanplei regional coefficients include data for all students fran a regicm.
Thus, for exanple, the total of 62 students fran Europe I included 17 fran
West Getmmy, 13 fran France, and 13 fran Spain, plus 19 fran other muntries
in the regicn. (See Table 4 for carplete ernmmation of mmtries in regiw.)
m-1
29
.35
18
.72
15
-09
62
.36
.07
Iamlpe II
49
.47
20
.47
9
78
.42
nicbaf3t
76
.22
20
-45
2
-
98
.26
Airica
I
10
.69
5
.79
2
-
17
.65
.39
9
Africa II
16
Am3rica I
0
lumricaII
35
.29
27
.35
24
ASiaI
112
.35
68
-29
Asia II
374
.31
203
All
702
.33
373
Ragiars
3
-64
33
.48
6
.29
.30
86
.32
8
.60
188
.34
.34
66
.51
643
.34
.37
138
.37
1213
.35
-
8
3
.49
-
bite. t‘ he predictive -site
was specified by .238 (Zq) + .176 (Za) = 0.0, a
regtessim equation based m the pooled matrix of departmntally z-scaled data
for 1,213 stuhts
fran all quantitative fields. The coefficients tabled represent the sinple correlation betwen this carposite and Zfya for fhe desiqated subgroups.
I
-430
For 22 subgroups (by region and type of quantitative
major) with coefficients in Table 13, the median coefficient
was I: = -37.
0
shmn separately in Table 12, the mdian
For the 20 country contingents
the
validity
coefficient
for the standard composite was II = .36, and for
nine regional contingents the median was r = -34, compared to the general
sample coefficient
of r = -35.
The results in Tables 12 and 13 indicate that
the correlation
between the
z-scaled criterion
and the standard c-site
of z-scaled predictors
tended to
be relatively
consistent for subgroups defined
in terms of national
origin
Correlations did not tend to be higher for
subgroups
and/or academic area.
were heterogeneous with respect
to
that were homqeneous than for those that
national origin.
.‘
. :..
‘..
..*..
-_
.
:
.;
_
;A
Section VII : An Exploratory Analysis of the Ccnnparative Performance
of Regional Subgroups on FYA and GRE VariablesStudents from All Quantitative
Departments
Consistent with the primary objectives
of this
study,
the analyses
reported in the preceding sections
were designed to assess the patterns
and
levels of correlation
between the FYA criterion
and the GRE predictors and the
possibility
that c;RE/Fw relationships
might be affected
by introducing
controls for
selected English-proficiency-related
test and background variables or for national origin.
The average levels of within-department
GREF’yA correlations
tended to be
similar for samples of foreign
ESL students frm different
types of quantitative departments-engineering,
math/science,
economics. Based on analysis
of
departmentally z-scaled data pooled across various cmbinations
of quantitative departmmts, the level of predictor/criterion
correlation
was not influenced by intrcducing
controls for
level of “English proficiency,”
variously
indexed, or for national origin.
This section describes a supplementary,
exploratory
ferences in the average academic performance of students
groups defined for the study and (b) the extent to tiich
in average academic performance tended to correspond with
in average GRE performance.
The analysis was based on
sample of students from quantitative
departments.
analysis of (a) difin the regional
subsubgroup differences
observed differences
data for the cmbined
An analysis based on the departmentally
z-scaled
MA and GRE variables
(Zfya and Zgre) provided information regarding the average standing
of members
of regional subgroups on these variables
within their respective
departments
of enrollment.
A parallel
analysis based on FYA (as computed and reported
by
departments) and GRE scores (on the 200-800
scale used for reporting)
provided information regarding
the average standing of members of the regional
subgroup on these variables without regard to their departments of enrollmnt.
In deciding
account:
upon this
approach,
the
following
factors
were taken
into
The regional subgroups differed markedly in size (from N = 17 for Africa
= 643 for Asia II).
The regional mix in many of the individual
I, toN
of the
departmental Samples, mdian
N = 12, could not be representative
regional mix in the total sample.
Moreover, n-embersof the respective
regional subgroups were not necessarily enrolled in cmparably
“representative”
arrays of departments with
respect to (a) field and/or major area
(engineering versus econmics
or
engineering versus math/ science, for
example), (b) degree of selectivity
(level of GRE scores), (c) grading standards
(level of grades awarded
relative
to level of GRE scores), and so on.
-46In such circumstances, observed regional-group differences in average
within-departmnt
standing will reflect to so~llt?
extent factors associated with
the nonrandm distribution of regional membersamng the various departmnts.
Accordingly, it was considered important to analyze not only the relative
within-department standing of subgroupsbut also their relative standing without regard to their departments of enrollment. It was reasoned that parallel
findings would permit stronger conclusions about regional differences in
academic performance than muld be possible if comparisonswere based solely
an z-scaled data.
Tfie usefulness of this approach is contingent, in part, on the degree of
comparability between the regression of FYA on GREquantitative and analytical
ability scaled scores for all ESL students without regard to their departmmts
of enrollment,
and the pooled within-department regression results for the
sam co&h&
Sample ( that is, the regression of Z&a on Zgre variables) .
TO make a determination regarding this question, a mltiple
regression
analysis based on the FYA as reported by departmnts and GREvariables on the
familiar 200-800 scale was conducted for the total sample (N = 1,213) of ESL
students. %e results obtained closely paralleled results of the regression
of Zfya on Zgre scores in this sang Sample.* For example:
In the total sample analysis, simple correlations of GREQ, A, and V with
Fw were -34, -28, aryj -11, respectively, as compared to -31, -28, a& JO
for the pooled within-department analysis.
cklly quantitative
and analytical scores had significant weights when m
was regressed on Q, A, and V in the total sampleanalysis; standard partial regression coefficients for Q and A were ,268 and ,144, respectively,
in the analysis without regard to department of enrollmnt,
as comparedto
,238 and ,176 for Zq and Za in the pooled within-department analysis.
For the total sample analysis, R = -36 for the Q,A c-site,
to R = -35 for the Zq,Za composite.
For the z-scaled withitiepartment
region, for Zfya, Zq, Za, Zv, and Z'fya,
zq + ,176 za.
as conpared
data set, mans were corquted, by
where Zrm = predicted Zfya = ,238
Regional mans were also computedfor Fw, GRE-Q,GRSA, GRE-V, and FYA’,
where EYA’ = predicted Fw = -0013 (Q) + -0006 (A) + 2.2831, a regressim
equatim based m the total-saqle
analysis alluded to above. These mxns were
expressed as deviations from the grand mzan for all quantitative
students
without regard to departtwnt of enrollmnt.
* Trends in findings of regression analyses conductedseparately for engineering, math/science, and econmics samples were similar to those reported here
for the ccmbined sample of students from quantitative departments.
-47-
Means of groups on the original FYA and GREvariables are shownin Table
14. Table 15 shows (a) these means expressed as deviations from the grand
means for all
students without regard to departmmt and (b) corresponding
regional means on the departmentally z-scaled variables.
In both tables,
regions are listed in desceding
order with respect to nx?anFYL Findings for
the sample of mglish-native-language
(ENL) students are included for perspective.
with regard to differences amng regional
performance," several trends are noteworthy:
groups in
"average academic
The ranks of regimal
groups in tenors of mean Fw correspmkd
closely
to
their ranks in terms of nbzan Zfya.
Ekhding
America II, the ranks for
IEL groups m the tw0 idices
of academic stiiding corresponded perfectly.
At one extreme, students from the two European contingents earned grades
averaging about 3.6, and they were in department-level samples in which
they tended to outperform other ESL students.
At the other extrem?,
students from the two African contingents and the Mideast earned grades
averaging about 3.3, and they wzre in departmnt-level samples in which
they tendti perform at a lowar level than other ESL students.
The two large contingents from Asia (accounting for soma68 percent of all
foreign students in the study sample) earned grades averaging approximately 3.5, corresponding to the grand mean for all students, and their
z-scaled mans were approximately 0.0. By inference, Asian students tended to provide a substantial "ccmmn element" in the regional mix of the
respective department-level samples; their academicperformance tended to
influence both the department-level sample FYAmeansand the total
sample
EYAnEan.
Effects associated with department of enrollment are illustrated in the
data for students fromZ%nerica II. Their FYAmaan of 3.44 placed them 0.12
standard deviations below the grand mean for all students without regard
(mean FyA= -0.12),
todepartnmt
but their man Zfya was 0.11, indicating that their F'Y% tended to be higher than the general ESL mans in
their respective departments. Thus, by inference, these students tended to
be from department-level ESL samples with comparatively lm man Fw.
It is also noteworthy that on both indices of "relative academic standing," the average standing of students from Category I countries (with ESLconducive backgrounds) was similar to that of students from Category II
countries (with ES-resistant
backgrounds).
These backgrounddifferences
are
reflected in regional means on the verbal masure and on the analytical
measure: the verbal and analytical. nxzans of Region I ESL stdents were higbr
than those for Regim
II
ESL &dents,
Data for the contingent of ENL (English-native-language students) suggest
that their academic performance tended to be roughly comparable to that of the
typical ESL student in their respective departments. Iheir man FYA, 3.54,
placed them0.12 standardunits abovethe grand11y3an
for all ESLstudents and
theirmean Zfya= 0.08 (z-scaled relative to the man for ESLstudents in
their respectik departnrents) indicated similar relative withikkpartzxznt
standing. EM; studentshad the highest standing of any group on m
(man =
726, 0.35 standard deviations above the grandman). However,their Zq mean
was -0.09,
indicating slightly lcwer than average relative standing -within
their respective departmznts.myinference, the ENLstudents tended to be fram
selective-departnx& that enrolled ESL (and other) students with very high
quantitative scores.
Cherved versusPredicted CriterimStadingofRegicnalGrwp6
Table16 showsthe observedmeansofregional groups onF!Aandon Zfya
correspondingpredicted mans. FYA' (predicted FYA) isbasedan the
(predicted
regressionof FYA on GRE< and GE&A in the total sample; Z’fya
Zfya) is basedon the regressionof Zfya on Zq and Za in the samesanple. ITA
and FYA' means are expressed as deviations froanthe grandmeanin standard
deviation units (grandnwn = 3.49, S.D. = O-41),
and the
Figure 1 points up the generally parallel nature of findings fra
the
across-departrwntsand the within-deparmts analyses. In each of the three
frames, subgroups
are ordered frczn left to right on man FYA. MeanFYA and
man FY7V, frcxnthe across-departmentsanalysis, are shownin the top left
frame,andmeanZfyaandxbzanZfya', frcunthe within-departmentsanalysis, are
&CM in the bottaanleft fraxb Meanresiduals (meanof "observedminus predieted performance"values) frcxn the across-deparmnts analysis and the
within-deparbwnt analysis are plotted in the top right franut.
The profiles of nwn residual values are quite similar. Eb-opean students, with generally higher criterion standing, tended to have higher
observedthan predicted standing, and students from the Mideast and Africa
II, with generally lower criterion standing, tended to have lower observed
than predicted standing. Therewasnxxe consistencybetweenobsewed and
predicted standingfor the remaininggroups (EM;, Asia I, and Asia II with
higher observed criterion standing, and Africa II with lcxr
criterion
standing).
my for studentsfrcxn mrica II xere across-departmentresults appreciably different from withitiepartrtkznt results. In the across-deparbt-kznts
analysis, obsewedFw and predicted FYAvalues for these students were
similar-bothwere belcw the grand mean(FBI = 3.49) for all students,
without regard to deparwnt of enrollnu3nt.Hwwer, this group enjoyed
ccqaratively higher average within-deparbent standing (man Zfya was
O.ll), while their averagepredicted relative withi&eparMent
standing
was lower (meanZ’fya
was -0.09).
Sucha pattern of findings reflects
effects associated with the particular "regional mix" and/or grading
standardsof the particular departmentsin which these students were
enrolled, as indicated earlier.
-5oTable 16
Grand Mean FYA and Mean FYA' as Comparedto Within-Department Mean
Zfya and Mean Z'fya for Regional Groups: Pooled Data for
All Quantitative Departments*
Region
N
Europe I
Europe II
Asia II
Asia I
America II
Africa II
Mideast
Africa I
All
All
"I"
"II"
ESL Total
Note.
As deviations
from grand mean #
FYA
FYA'
Mean
Zfya
Mean
Mean
FYA'
Mean
Zrfya
62
78
3.63
3.57
3.53
3.47
0.34
0.20
0.10
-0.05
0.27
0.22
0.04
-0.08
643
188
3.52
3.49
3.51
3.52
0.07
0.00
0.05
0.07
0.02
-0.02
0.05
0.05
86
33
98
17
3.44
3.34
3.32
3.31
3.39
3.32
3.42
3.42
-0.12
-0.37
-0.41
-0.44
-0.24
-0.41
-0.17
-0.17
0.11
-0.18
-0.34
-0.47
-0.09
-0.27
-0.16
-0.12
273
840
3.50
3.51
3.51
3.49
0.02
0.05
0.05
0.00
-0.00
0.04
0.03
0.01
1213"f
3.49
3.49
0.00
0.00
0.00
0.00
31
3.54
3.59
0.12-
0.24
0.08
0.13
Regional groups are in descending order with respect to meanFYA.
*FyAFyAregularly scaled FYA; FYA' = predicted FYA, using -0013 (Q) + -0006
(A) + 2.2381, an equation based on the regression of FYA on GRE scaled
scores for the combinedsample of ESL students from quantitative departments.
Zfya = departmentally z-scaled FYA; Z'fya = predicted Zfya, using ,238
,176 Za=
O-00, an equation based on the regression of Zfya on the
departmentally z-scaled GREpredictors for the same combinedsample of ESL
students from quantitative departments.
Zq+
# The observed and predicted FYAmeansfor regional groups are expressed
here as deviations from the grand FYAmean (ESL total mean= 3.49) in total
sample standard deviation units (S.D. = 0.41).
E
.
TRENDS
IN “OBSERVED
MINUS
PREDICTED
FERFORMANCE”:
RESULTS
BASED ON THE ACROSS-DEPARTMENTS
ANALYSIS
(TOP LEFT),
COMPARED
TO RESULTS
BASED ON THE
WITHIN-DEPARTMENTS
ANALYSIS
(BOTTOM
LEFT)
ACROSS-L)EPAH’FMENTS
ANAL%-SIS, BASED ON DATA
FOOLED
WITHOUT
DEPARTMENT-LEVEL
STANDARDIZATION
Mean
FYA end mean
predicted
FYA (FYA’)
l , computed
without
regard
to department
of enrollment
1.0
Legend
:
3.0’
:
E-2
Regionol
(higher
:
:
ENL
E-l
:
:
As-2
subgroup6
[3,6]
to
:
Am-2
FYA’ -
WA
o
FYA
.8
(31
.c
0
.6
1
.0013
I
Legend
,,
FYA
0
Zfyo
t
-1.0s
:
Af-1
ENL
E-l
M-E
ordered
from
lower
13.33)
on
+
:
Af-2
As-l
x
.c
E-2
left
to
meon
right
FYA
GRE-Q
+
Regional
(higher
.0006
subgroups
[3.6]
As-2
As-l
to
Af-2
Am-2
ordered
lower
[3.3])
Af-1
M-E
from
left
on
mcon
to
right
ITA
GRE-A
WITHIN-DEPARTMENTS
ANALYSIS,
BASED ON DEPARTMENTALLY
STANDARDIZED
(Z-SCALED)
DATA
Average
relative
standing
(Zfya)
vemua
rverage
predicted
relative
etanding
(Zfya’)
l
Legend
.6
.4
Y
Zfya
0
Zfyo’
t
Flguro
-.6
-.8
yz
u_
ENL
E-l
E-2
Regionol
(higher
As-l
As-2
subgroups
[3.6]
to
Af-2
ordered
lower
[3.3])
from
on
+
-
Zfya’
Af-2
M-E
Am-2
left
to
meon
,238
right
MA
Z(q)
+
-176
Z(o)
1.
Flndlngs
brmod
dopartmont8
l naly818
ol
on l nrlyala
of data
acroa8
vor8u8
flndlng8
brsod
on
data
wlthln
doprrtmont8
-
FYA’
-
Zfyo
-52-
Generally speaking, the findings reviewed in this sectian indicate
that
there were differences amongthe regional subgroupsin level of acadexnic performance. There was a general tendency for subgroupswith higher average criterion performance to have higher scores on GEEpredictive ccqosites.
How
ever, the findings also suggestedthe possibility
of predictive
bias for
regional subgroupswithin the foreign J3SLstudent pomation.
For exaxnple,
European students tended to have mt
higher relative standing on 21y3an
FYA
and mzan Zfya than expected fram the corresponding sets of GREmasures while
the opposite was true for students from the Mideast and Africa.
These particular findings should be viewed primarily as working hypotheses for future
research concerned specifically with the assesgnent of predictive bias.
In the mantime, departnvsnts may assess the relevance of the trends
revealed in this exploratory analysis by observing the typical patterns of
academic performance of students from various regional subgroupssuch as those
defined for this study.
-53References
Braun, H. I., & Jones, D. H. (1985). Use of empirical Bayesx&hods in the
study of the validity of academicpredictors of graduateschool
performance(GREBoardProfessional Report GREB
No. 79-13P& EIS
RR-84-34). Princeton, NJ: EducationalTesting Service.
Burton, N., & Turner, N. J. (1983). Effectiveness of the GraduateRecord
Examinationsfor predicting first-year grades: 1981-82sum~ry report of
the GraduateRecordExaminationsValidity StudyService. Princeton, NJ:
Educational Testing Service.
Fducational Testing Service (1983). TOEFLTest and scoremanual. Princeton,
NJ: Author.
Educational Testing Service (1985) . Guideto the use of the GraduateRecord
ExaminationsProuram.1986-87. Princeton,.NJ: Author.
Kingston, N. M. (1985). The incremntal validity of the GREAnalytical
masure for predicting first-year grade-pint average(unpublishedpaper
presented at the annual meting of the American Educational Research
Association).
Linn, R. L., Hamisch, D. L., & Dunbar,S. B. (1981). Validity generalization
and situational specificity: An analysis of the prediction of first-year
grades in law school. Applied PsychologicalMeasurexwnt,
-5, 281-289.
Miller, R., & wild, C. L., I&. (1979). Restructuringthe GraduateRecord
ExaminationsAptitude Test (GRE Board'Rxhnical Report). Princeton, NJ:
mcational Testing Service.
Mosteller, F. M., & Bush, R. R. (1954). Selected quantitative techniques. In
G. Lindzey (Ed.), Handbook
of social psychology(Vol. 1, pp. 289-334).
Cambridge,MA: Addison+esley.
K., Schmidt, F. L., & Hunter, J. E. (1980). Validity generalization
results for tests used to predict job proficiency andtraining successin
clerical occupations. Journal of Applied Psychology,-65, 373-406.
Pearhan,
Powers,D. E. (1980). The relationship belweenscoreson the Graduate
ManagerwrtAdmissionTest and the Test of mglish as a Foreign Language
(TOEFLResearchReport No. 5). Princeton, NJ: ticational Testing
Service.
Sharon,A. T. (1972). English proficiency, verbal aptitude, and foreign
student successin Americangraduate schools. Educationaland
PsychologicalMeasureIzlent,
-32, 425-431.
Swinton, S. S. (1985). The predictive validity of the restructuredGREwith
particular attention to older students (draft project report). Princeton, NJ: Educational Testing Service.
-.
-54-
Willingham, W. W. (1974).
Predicting successin graduateeducation, Science,
183, 273-278.
Wilson, K. M. (1979). Thevalidation of GREscoresas predictors of firstyear performancein graduatestudy: Report of the GRECooperative
Validity Studies Project (GREBoardResearchReport No. 7-R).
Princeton, NJ: EducationalTesting Service.
Wilson, K. M. (1982a). A comparativeanalysis of XEEL examineecharacteristics (TOEFLResearchReportNo. 11). Princeton, NJ: Ehcational
Testing Service.
Wilson, K. M, (1982b). @XC and GFE Aptitude Test performancein relation to
primary languageand scoreson TQEZL(TDEFLResearchReport No. 12 & EIS
R&82-28). Princeton, N?: EducationalTesting Service.
Wilson, K. M. (1982c). A study of the validity of the restructured GRE
Aptitude Test for predictinq first-year performancein graduate study
(GREBoardResearchReportNo. 78-6R & EIS RR-82-34). Princeton, E;IJ:
Educational Testing Se-mice.
Wilson, K. M. (1984a). Foreignnationals taking the GREGeneral Test during
1981-82: Highlights of a study (c;RE BoardResearchReport GREBNo.
81-23aR & EIS RR-84-23). Princeton, NJ: EducationalTesting Senrice.
Wilson, K. M. (1984b). Foreignnationals taking the GREGeneral Test during
1981-82: Selected characteristics and test performance(GREBoard
Professional ReportNo. 81-23bP& ETSRR-84-39). Princeton, 1w:
Educational Testing Senrice.
Wilson, K. M. (1984c). The relationship of GREGeneral Test itemtyPe part
scores to underqraduategrades (GREBoardProfessional Report GREBNO.
81-22P & 615 R&84-38). Princeton, NJ: EducationalTesting Senrice.
predictive validity for foreiqn
Wilson, K. M. (1985). Factors affecting GM&XT
MBAstudents: An exploratory study (F1s RR-8517). Princeton, I%?:
Educational Testing Service.
Wilson, K, M. (1986a). The relationship of scoresbasedon GEEGeneral Test
item types to undergraduategrades: &I exploratory study for selected
zips
(GEIlBreport, in press). Princeton, NJ: Educational Testing
l
Wilson, K. M. (1986b). The GRESubject Test performanceof U.S. and Nan-u.S.
Examinees,1982-84: A coqarative analysis (GEBBreport, in press).
Princeton, NJ: EducationalTesting Service.
-55-
Appndix A: Sumnaryof Departrwnt-Level Data
A-l
GREand FyA Meansand Standard Deviations, and G6IE/Fw
Correlations for Departmnt-Level Samples of Foreign
ESL Students, by Field
A-2
Simple Correlation of GRE Predictors with FYA for
Foreign ESL Samples, by School and Department
A-3
Distributions
Area
A-4
Distributions of Standard Deviations of GRJZ General
Test Scores for Department-Level Samples in 86 Primarily Quantitative Departments, Six Bioscience, and Five
Social Science Departments
A-5
Distribtions
of Standard Deviations of GRE General
Test Scores for Departnmt-Level Samples in 86 Primarily Quantitative Departmznts, By Size of Sample and
without Regard to Sample Size, Respectively
A-6
Distributions of Correlations between GEZ Predictors
and FYZ4 for 86 Department-Level Samples of JZSL Students in Primarily Quantitative Fields
A-7
Distributions
of Department-Level GRE General Test
Validity Coefficients, by Size of Sample: Data of 86
Departmnts in Primarily Quantitative Fields
of GREMeans for Departments, by Graduate
Page 1 of 4 pages
Appendix A-l
GRExind FYA beans and Standard Deviations,
and GRE/FYACorrelations,
Department-Level !%mples of Foreign ESL Students, by Field
DEPAH lHC!dI/
SCHI I’1 hl’;.
N
FYA
Correlation
WithEm
Standard
deviation
Mean
CHE-V
GRF-0
ANALY
FYA
710.5
r40.0
697.1
751.7
6Y?.O
771.8
733.0
740.0
722.2
503.5
510.0
497.9
5Yl. 7
430.0
530.9
441.0
505.5
507.3
0.40
I). 28
0.32
0.15
0.31
0.19
il. 3h
0.3Y
0.31
125.6
60.0
136.2
H4.2
Ill.5
50.4
75.7
104.6
97.9
64.b
25.5
86.3
23.2
82.3
52.5
25.9
41.7
54.5
9A.5
53.4
115. I
90.4
74.6
HI.7
83.0
71.9
87.0
0.250
-.\36
0.474
0.226
0.546
-. 035
0.7Y2
-.242
0.113
-.023
0.555
0.4R2
0.73Y
-.I37
0.643
0.236
0.056
0.315
0.36fl
O.LR9
0.467
0.134
0.781
0.600
0.061
-0 260
0.362
45.3
64.1
b0.9
66.8
48.7
67.6
74. I
50.2
60.6
84.9
62.9
88.3
106.7
85.7
94. R
91.4
48.7
87.?
-0
26b
-.Obl
0.729
-.07D
0.342
-.614
O.ltitl
0.02R
-0.047
0. I36
0.116
0.053
0.10b
0.7RH
0.504
-.094
-.OYh
0.163
0.2D7
0.132
0.836
0.060
0.410
-.49R
-. 397
0.089
0.08R
0.109
0.315
O.j48
0.049
0.082
-* 054
0.293
0.158
0.15‘)
0.430
0. I85
0.451
0.209
0.466
0.327
D. 521
0.75Y
0.215
0.024
0.016
0.471
0.296
0.365
0.7R5
0.387
0.307
0.234
0. C?Y
0.436
0.309
0.470
0.219
0.340
0.405
-. 182
0.504
0.193
-0 026
0.770
0.165
0.419
O.bR5
0.671
0.451
0.15R
0.362
0.400
0.519
0.617
0.482
0.3H2
0.047
0.355
0.342
WF-V
ME-0
ANAL Y
CHCY ICAL E’JGIN
17
64
2
9
64
7
I4
64
10
64
14
b
5
64
I4
64
30
13
64
(15
5
73
64
II
64 TUT AL
80
3.66
3.77
i‘.CH
3.41
3.51
4OH. d
3t14.4
430.0
43A. 3
406.0
345.4
3?4.3
4A0.r)
400.7
CIVIL
65
6s
65
65
65
65
65
65
65
32
30
10
42
13
11
19
c,
lh2
3.5h
3.45
3.71
3.44
3.59
3.64
3.65
3.00
3.52
415.6
333.3
415.0
74b. 7
365.4
356.4
365.3
jHY.0
367.5
7?b.‘#
h73.3
732.0
691.0
723.1
690.9
674.7
632.0
6Y6.2
533.7
426.3
537.n
471.7
530.n
490.9
457.4
47?. 0
4A3.9
0.40
0.29
0.37
0.46
0.34
0.30
0.30
0.63
0.37
106.0
64.9
55.4
A8.8
72.1
Al.4
6h. 5
14q.9
83.1
FNGIN
3J
31
I4
31
19
12
+
‘h
52
15
13
256
3.49
3.4Y
3.54
3.41
3.76
3.54
3.47
3.55
3.57
3.21
3.50
336.7
403.5
349.3
403.7
380.3
339.7
471.1
J90.4
339.1
343.1
781.2
692.7
742.9
649.7
717.6
727.4
134.2
711.4
74R.3
715.3
708.5
719.4
467.3
522.3
482.9
517.6
486. a
536.7
537.5
544.0
466.0
4PI.5
511. I
0.36
0.76
0.37
0.33
0.31
0.29
0.46
0.45
0.33
0.67
0.40
79.9
105.2
103.7
97.7
107.9
4d.9
129.9
llh.
1
Pl.6
116.1
103.3
74.4
34.3
119.9
72.5
42.9
7R.?
65.9
55.0
60.B
66.5
91.4
95.2
13R.0
100.9
lo?. 6
IOH.?
107.9
103. I
11 A.0
111.1
104.4
INDIJSIRI
AL FNG~N
67
2
17
hf
7
13
hT
13
9
h?
24
13
67
38
75
GI
13
1H
h7 TIJTAL
RY
3.2d
3.63
3.47
i.44
3.57
3.76
7.45
443.3
410.9
35 1. A
365.4
37R.H
404.4
302 .v
717.5
735.0
691.R
c.tln. 0
hRl.2
b44.4
697.4
567.5
476.
47R.9
437.7
4Ht.b
433.9
477.9
0.31
0.32
0.41
0.35
0.33
0.70
0.33
91.7
85.3
129.5
163.6
73.5
122.2
406.2
64.15
4O.R
71.6
100.7
6A. 7
Rb.2
73.9
111). 3
111.9
100.6
R4.5
71.3
‘36. 7
92.3
ENGIN
2
7
I’,
24
30
33
38
49
TJTAL
ELFCTklCAL
66
7
66
10
66
13
60
14
66
24
bb
30
f~6
33
66
34
66
41
LO
49
hC TOTAL
4.31
3.59
3.43
3.62
for
t
66.9
GRE-V
.
c
Appendix A-l,
Page 2 of 4 pages
contimed
QIE and WA jeans and Rarx!lard Deviations, and QIE/FyA Correlations,
Deprtmnt-Level
Samples of Foreign ESL Students, by Field
Correlation
WithFYA
Standard
deviation
Mean
for
OF PA~TYENI/
GRE-V
WE-0
ANAI Y
3.77
3.40
3.67
3.64
3.67
3.40
3.47
3.66
3.57
432.9
3H7.1
34d.O
337.5
391.0
35R. 8
360.0
438.0
768.9
751.4
736.4
724.5
7OA.2
751.0
hR5.9
697.8
711.0
714.5
521.4
535.0
475.5
455.0
534.0
457.5
476.9
536.0
408.5
0.31
0.49
0.30
0.41
0.31
0.39
0.67
0.36
0.41
702
3.51
379.7
709.5
496.5
5
5
3.53
3.53
?9R.O
298.0
690.0
690.0
408.9
204.5
370.0
337.5
374.3
352.4
SCti~Jt~L NCI.
MECHANICAL
bd
r
68
13
68
14
bR
24
68
30
bR
33
68
30
6R
b5
68 TOTAL
N
ENGIN
7
14
?O
24
10
1’
13
10
115
ALL ENCIN
APPLIEO MATH
54
28
54 TlllAL
FY4
GHF-0
ANAL Y
107.7
90.9
70.0
R2.4
dO. B
lOB.7
55.2
131.2
87.7
29.7
75.6
55.4
95.7
38.4
79.1
5?.0
bb.?
67.3
88.6
108.4
tl1.3
97.9
91.3
111.9
67.h
10% 3
94.7
0.37
95.8
64.7
95.3
470.0
470.0
0.54
0.54
48.2
48.2
43.6
43.6
76.8
76. a
716.7
702.7
709.3
67O.A
657.9
693.3
527. B
462. 7
496.0
458.3
477.1
477.2
0.31
0.25
0.41
0.43
0.35
0.36
8B.8
49.1
74.4
113.5
107.4
84.6
99.7
66.0
60.5
59.5
105.3
73.6
503. b
402.0
455.8
42fl. 5
514.0
439.0
472.0
410.0
500.0
453.1
455.0
0.28
0.61
0.30
0.47
0.20
0.29
0.52
0.26
0.26
0.47
0.37
52.0
124.6
70.4
76.7
130.8
71.2
bJ.8
76.7
115.4
77.0
83.5
59.2
75.0
67.4
64.4
49.2
70.5
124.7
111.4
57.5
104.9
76.0
463. I
39fI.n
528.0
437.7
437.9
0.25
0.47
0.29
0.3H
0.37
89.0
77.1
131.7
63.1
83.4
STATISTICS
59
3
59
7
59
14
59
24
59
28
59 TUTAL
7
54
3.72
3.71
3.40
3.42
3.39
3.54
CHEMISTRY
62
7
62
34
b?
?B
62
33
62
34
62
38
62
Cl
62
65
67
67
62
92
62 TOTAL
11
IO
26
I3
5
IO
5
6
9
16
111
3.53
3.14
3.64
3.17
3.51
3.49
3.67
3.42
3.47
3.49
3.4H
350.0
352.0
355.8
133. I3
37kJ.o
314.0
3Btl. 0
330.0
427.8
3Rfl. 1
360.1
710.9
664.0
707.7
660. fl
697.0
658.0
630.0
628.3
7015.6
610.0
671.4
HATlJEMAlICS
72
13
72
24
72
34
72
38
72 IfJIAL
R
I5
5
11
39
3.87
3.32
3.50
3.!bs
7.52
356.2
334.7
372.0
334.5
343.B
703.7
634.0
752.0
650.0
667.9
15
12
FYA
GRf-W
61.6
85. A
47.7
R3.1
74.5
CRF-V
CRE-0
ANALY
0.533
0.3Hl
0.766
-.I35
-.bltl
0.?02
-.I
38
0.535
0.104
-. 597
O.OY5
O.C~l2
0.563
0. 700
0.004
0.587
0.725
0.347
0.457
-.I45
0.5RB
0.755
-. 704
0.375
0.347
0.763
0.300
0.114
0.2H9
0.278
-.272
-0.272
-. 176
-0.178
-.458
-0.458
147.4
86.6
109.3
60.9
166.6
107.7
0.131
-.119
0.507
0.206
0.036
0.189
0.458
0.500
0,?75
-0 104
0.761
0.730
0.474
-.I?7
0.630
0.550
0.563
0.425
86.6
130.1
RB.9
113.3
71.3
73.7
166.6
Ab.3
74.2
Mb, 7
94.9
-.243
0.424
0. II34
0.071
-.723
-0 III
0.013
0.708
-e 124
-. Ofl4
0.012
0.506
0.666
0.401
0.365
-. 375
0.654
-.019
0.430
0.007
0.2AA
0.353
0.447
O.bhA
-.177
0.541
-. 646
0.773
-0 026
0.492
0.104
0.705
0.190
-. Oh5
-. 098
0.71 r
-. 190
-0.013
0. hh0
0.550
0.574
0.644
0.602
0.220
0.234
0.480
0.246
0.266
168.9
109.0
99.1
103.9
118.6
Appendix A-l,
Page 3 of 4 pages
contimed
CiM and FYA mans and Standard Deviations, and c;RE/Fw Correlations,
Department-Level Sanples of Foreign ESL Students, by Field
Gut-v
GP E-0
ANAL V
595.0
413.3
39H. r
410.0
437.5
403.6
440.4
71R. 3
723. A
131.2
720.0
710.0
719.1
771.6
611. I
496.9
4’Jh. 7
h20.O
542.5
53?. 7
537.3
503.3
47h. 7
137.3
410.9
45R.l
416.7
75H. 3
6BY. 2
710.6
77Q.3
ho 4. 2
524.4
4b9.0
5Ob. 4
553.7
522.5
52b.0
7cs.o
Mb.
33tJ. 6
4hR.5
437.9
427.4
657.1
1:
7
11 t
3.70
3.79
3.ots
3.53
3.67
3.77
3.45
3.27
3.39
3.70
3.43
494.3
57L. 2
373
3.50
387.8
ECCINW ICS
tic
7
84
7
1)4
I3
R4
I4
84
?4
R4
30
84
32
a4
34
64
41
94
A7
84 TOTAL
10
15
12
6
II
11
48
5
1:
A
138
7.40
3.5R
3.65
3.73
3.5)
3.24
3.19
3.76
3.70
3.77
3.39
549.0
3RO.O
400.0
453.3
400. Y
371.9
374.0
364.0
477.5
ECONfWICS
138
OUA’rll~AllV
1213
PHVS ICS
7b
2
7b
7h
76
76
76
76
GRF-V
CRF-O
ANAL V
0.79
0.33
0.?7
0.58
0.32
0.42
0.36
Y9.1
112.0
AC.9
192.0
83.3
75.0
101.6
53.4
6H. 7
55.1
107.2
67.9
04.4
71.1
14v. 3
177.4
R5.2
107.0
97.8
129.6
117.7
148.1
186.6
75.6
87.3
104.2
149.1
135.0
2Y.7
155.2
104.9
115.9
83.5
30.9
82.6
67.6
66.2
7h. b
27.0
114.8
50.6
54.7
63.0
R1.5
71.6
74.5
7Y. 5
14.h
106.4
AO.0
65.5
103.0
79.7
0.137
0.155
576.0
0.37
0.23
0.54
0.60
0.77
0.45
0.45
0.54
0. i‘n
0.16
0.43
-. 188
0.540
-0.013
695.3
492.4
0.39
95.5
70.5
97.4
398.8
746.0
736.7
6H8.3
756.7
67Y. 1
63ll.O
570. d
702.0
hfl4.2
607.5
449.9
597.0
498.0
CRO. 8
50 3.3
CR 7.3
514.5
4118. 7
416.0
525.R
441.2
469.2
0.53
0.#?8
0.37
0.24
0.3h
0.18
0.35
0.29
0.15
0.34
0.32
125.6
52.1
55.1
54.5
YR.6
55.3
109.0
119.1
61.8
136.0
RY.R
60.4
71.7
97.1
4L.R
110.3
121.9
103.3
40.2
68.7
59.9
R7.7
57.7
95.6
05.0
A5.5
73.6
R9.6
104.R
117.2
103.9
100. tl
94.7
3.39
348.8
649.9
469.2
0.32
89.1)
87.7
3.49
3H4.4
698.4
49?.
0.77
95.0
69.1
3.75
7
3.5,)
n
I’)
30
33
41
ICIlAL
5
1:
51
Correlation
withm
Standard
deviation
Mean
for
3.Btl
3.45
S.b7
3.55
3.62
TVA
GWf-0
0.539
0.485
-.I02
O.Alb
0.450
0.605
0.452
0.341
0.431
0.337
0.578
0.63L
0.433
0.452
0.416
0.540
0.367
0.706
0.144
-.004
0.4h7
0.356
0.249
0.740
0.055
0.628
-.767
0.463
-.2d7
0.520
0.445
-.015
0.234
0.2?0
0.023
0.351
0.748
-.097
0.669
0.352
-.2?5
-.752
0.351
0.338
0.495
-. 176
-.OHO
0.210
0.080
0.359
0.079
0.041
0.631
-. 7?9
0.465
0.27R
0.315
0.632
0.713
0.170
0.376
0.497
0.600
0.692
0.157
0.164
0.392
0.470
-.190
O.ZRt
94.3
0.210
0.317
0.282
95.9
0.097
0.311
0.175
-.33Y
0.319
-.667
0.042
0.025
0.305
0.011
cfJ*PuTER
78
7
IA
14
78
?4
78
30
7H
33
78
38
78
41
IR
49
78
60
7R
82
76 IOf
WATH/PHV
l?
2:
11
lh
12
5
SCI
7b’i.O
775.b
t,qo. 0
770. d
717. I
717.6
0
538.6
1
86.2
-.I23
052
0.377
-. 392
0.040
-. 225
-. b5iH
-0
-.936
*
.
Appendix A-l,
Page 4 of 4 pages
conchxbd
8i~ ard FYA !SUIS and &axxlard Deviations, and QIEF”w Correl?tions,
Department-Level Saqles of Foreign F5L Students, by Field
DFPAWT’+FNl
SCtlflJILNrJ.
Correlation
withm
standard
deviation
Mean
for
/
N
FVA
GRf-V
(;RC-0
ANALV
FVA
C.RF-V
GRF-O
ANALV
97.4
97.4
HICH~.HIIJLU~V
7
7
6
6
3.43
3.43
41O.c)
4lU.O
691.7
691.7
540.0
540.0
0.46
0.46
129.5
83.8
120.5
83.8
33?.CJ
340.1
343.5
514.3
503.1
507.0
417.1
361.9
370.7
0.44
0.48
0.47
39.0
174.7
76.0
64.8
119.0
136.0
3.07
2.99
453.7
369.1
404.7
653.7
616.4
666.8
517.5
469.1
4A9.5
0.47
0.67
0.59
7
3.35
371.4
7
3.35
371.4
612.9
612.9
504.3
504.3
55
3.19
375.5
595.8
74
IlrTAL
AGRICULTURAL
31
7
7
T.?b
3R
Lb
3.lH
31
TflTAL
23
3.21
BIlICHEnISlHv
34
7
a
41
II
fOrAL
19
BInLOGY
35
49
35
810
0.369
0.369
-.569
--I). 569
0.?45
-.09s
-0 155
-0.129
0.075
-. 260
-0.119
E
31
34
34
-.75,1
-0.751
lOTAL
SC1
?.9fl
110.3
112.1
7R.R
89.0
104.4
95.1
9Y.O
-.5R8
-0.23ti
96.4
55.3
0.664
96.6
55.3
0.664
03.0
94.5
72.1
55.9
62.7
0.59
41.5
0.59
61.5
450.5
0.52
80.7
100.0
H9.1
541.1
517.9
562.4
48R. 3
533.0
431.3
373.5
409.0
405.R
408.6
0.64
0.17
0.1’5
0.36
0.36
1?7.?
134.9
53.1
175.3
48.7
141.7
113.4
165.2
101.3
R9.0
87.3
Il.8
90.0
-0.023
-.3R4
-0.3AC
-* 641
-0.641
0.061
0.081
0.374
-.018
0.251
0.004
0.000
0.004
O.O?Q
0.122
0.130
0.430
0.133
0.6Rl
0.197
0.142
0.172
EOuC AT ION
85
85
85
d3
2
7
24
33
26
17
21
3.22
3.56
3.45
I2
3.66
85
TUTAL
76
3.43
421.9
323.5
296.2
334.2
351.3
9
3.57
353.3
611.1
464.4
9
3.57
353.3
611.1
464.4
3.44
351.5
541.3
414.5
POLITICAL
97
97
TfIrAL
101 AL
15O.h
0.17
85.4
122.1
85.4
122.1
11 8.6
118.6
0.727
0.17
0.34
06.6
147.6
93.1
0.253
0.37
93.9
75.3
95.4
0.102
SCI
7
SOCIAL
Mb.8
SC1
85
1353
3.411
0.727
-.097
-0.097
0.116
0.282
0.2R2
0.184
0.262
Appendix
A-2
Page 2 of
Page 1 of 4 pages
Simple Correlation
of GRE Predictors
with FYA for
ESL Samples, by School and Department
Foreign
GRE/FYA correlation,
School
School
Deeartmeot
N
Correlation
Department
by school
N
of GRE predtctor
with PYA
and department,
continued
Correlation
vith
GEE-V
02
Chem Engineering
Civil
Engineering
Induet Engineering
Statistics
Physics
Economics
Education
07
Chem
Civil
Elect
Indust
Mech
Engineering
Engineering
Engineering
Engineering
Engineering
Statistics
Chemistry
Physics
Computer Sci
Economic6
Agriculture
Biochemistry
GIli5-A
GRB-V
Gas-Q
17
32
12
9
6
10
.25
-. 27
.40
.13
-. 34
-. 10
-.02
.14
.42
.46
.54
.08
.3?
.21
.52
.47
.34
.17
26
.37
.25
9
30
33
12
11
11
13
12
15
-. 18
-.07
.11
-. 18
.53
-. 12
-. 24
.32
-. 12
.67
7
8
17
9
14
Chem Engineering
Elect Engineering
Mech Engineering
Statistics
Computer Science
Economics
.08
19
Elect
.56
.12
.45
.68
-. 60
.50
.51
.48
.42
.36
.69
.13
.36
.61
.46
-. 13
.45
.43
.24
.33
24
Chem
Civil
Elect
Indust
Hech
-.73
,24
.71
-. 10
.71
.08
-.02
.73
.oo
-, 10
.Ol
.28
Engineering
Engineering
Engineering
Engineering
Engineering
Engineering
Statistics
Chemistry
Mathematics
Computer Science
Economics
Microbiology
Education
Political
Education
Science
28 Applied
Chem Engineering
Elect Engineering
14
31
.47
.32
.48
.29
Elect Engineering
Induet Engineering
Mech Engineering
Mathematics
Physics
Economics
14
9
14
8
8
12
.45
.58
.38
-.06
-.67
.35
.47
.67
.lO
.66
-.lO
.08
.47
.28
30
13
.39
.48
-. 14
.22
.34
.50
32
6
31
20
15
9
6
Mathematics
Statistics
Chemistry
42
19
13
24
12
10
15
21
11
6
21
l
.33
.61
.2S
.54
.04
l 23
.05
.55
-.02
.08
.19
-. 14
.21
.42
-.lO
.37
-.25
-.
.ll
.52
.45
.56
-.lO
.67
.55
.37
.63
.37
-.75
.02
.12
-. 27
.04
.18
-.
Economfcs
30
13
12
10
5
12
11
-.06
.34
-.05
-.62
.04
-.33
.35
.64
.79
.76
.20
.82
.31
-.32
Economics
48
.34
.46
Chem
Civil
Elect
Hech
Engineering
Engineering
Engineering
Engineering
Physics
Computer Science
5
.23
.05
.27
.51
-.05
-.22
.76
.40
Y
.
Page
GRE/FYA
correlations,
Lkpartvmot
School
by
school
N
and
department,
Civil
Elect
Mech
.36
.45
.14
12
.43
.13
.68
52
.16
-.72
.J2
.50
.02
-.38
.57
.28
.31
-.65
.48
.39
-.09
12
.19
-.03
-. 14
-.ll
-. 19
-.22
-.40
.05
.35
.32
.25
-.29
Agriculture
16
.51
Engineering
Chemistry
15
5
. 16
Science
Education
38
Engineering
60
13
8
16
5
Mathemat
its
Economics
5
5
Civil
Engineering
19
Indust
Engineering
25
13
10
11
Elect
-.61
.29
.20
.OJ
.50
.22
.oo
.16
.59
.65
.64
-.oo
.08
.60
.08
-.02
.60
-. 94
.32
.4J
-.03
.43
.52
.43
.09
.22
.44
Physics
Science
11
Economics
12
.Ol
.30
-. 66
-. 18
Civil
Engineering
Elect Engineering
Computer Science
5
13
7
.03
.43
.13
-.lO
.47
.16
Biology
7
.66
-. 38
-.64
.47
-.02
Computer
49
17
Chemistry
Mech Engineering
Chemistry
Mathematics
Computer Science
41
11
36
Computer
Science
5
13
-,
19
correlations
School
Department
by
school
N
and
department,
67
73
Chemistry
Chem
Indust
Engineering
Engineering
Computer
87
91
9
11
18
Science
Economics
8
4 of 4 pages
concluded
Correlation
of CUE predictor
with FYPA
CUE-V
GBg-A
.02
.04
Computer
Elect
Cm-Q
Page
GRE/FYA
-. 50
.44
.38
.54
.63
.46
Engineering
Engineering
Engineering
Chemistry
Physics
34
4 pages
Correlation
of CUE predictor
with FYA
GRE-V
33
3 of
continued
CU-Q
-.12
.Ol
-.24
.27
.36
.O6
.54
.36
-,08
.63
Biology
11
-.
Chemistry
16
-.08
59
CBg-A
. 10
-. 26
.36
.23
-.
19
-. 16
-.26
.29
.21
-62A-3
Distributions
Score
category
760+
740-759
720-739
700-719
680-699
660-679
640-659
620-639
600-619
580-599
560-579
540-559
520-539
500-519
480-499
460-479
440-459
420-439
400-419
380-399
360-379
340-359
320-339
300-319
280-299
Engineer/
Math/Sci
Q A
V
of GRE Means for Departments, by Graduate Area
Economics
Q
AV
Bioscience
QAV
Sot Science
QAV
1
10
15
20
14
5
6
4
12
1
1
1
1
1
1
2
3
18
7
10
13
7
8
2
1
1
1
1
2
2
2
7
14
9
9
12
13
1
2
1
1
-1
l-
1
1
-1
l2
3
11
-1
2
-
2
1
4
-1
2
1
-1
11
12
1
1
1
1
1
1
11
2
l-
1
12
16
21
17
7
1
7
5
1
4
2
2
2
2
14
- 19
3 11
1 13
15
8
9
7
1
2
2
1
1
1
2
2
4
9
17
10
15
14
16
1
3
1
1
2
All Fields
QAV
1
2
No. depts.
76
Median
708 488 382
10
690 493 400
6
630 490 370
5
550 414 334
97
701 490 379
No. students
1075
Mean
705 495 382
138
650 469 399
55
596 450 376
85
541 414 352
1353
684 486 382
General samples*
Non-U.S. 652 458 364
U.S. 645 582 520
585 443 387
592 569 498
543 428 382
533 530 498
517 428 391
483 496 485
Note. Department means are based on data for a minimum of five students
identified
as nonnative English-speakers.
Q = GRE quantitative,
A = GRE
analytical,
and V = GRE verbal. Only students with GRE scores earned after
9/81 were included in department samples.
* Means for non-U.S. examinees and U.S. examinees tested between October
1981 and September 1982, classified
by intended field of study.
4
Distributions
of Standard
86 Primarily
Deviations
of
Quantitative
CRE General
Depa rtmente,
Teet
Scores
for
Department-Lev
el
Six Bioscienc
e, and Five
Sot ial
6
2
4
5
989
01568
5606
25924
57845
626
45689
91297
34599
67127
04235
11557
04467
6
60255
899
0
552
16
9
0229
78
to)*
(0)
(-)
(7)**
(3)
(-)
:3;
(6)
(2)
;I;
(‘1;
(9)
General
samplet
19
18
17
16
15
14
13
12
11
10
9
8
6
(3)
(9)
4
3
2
2
Median
of P oreig n ESL
Dep artme nts
CRE-Quantitative
GRE-Verbal
19
18
17
16
15
14
13
12
11
10
9
8
Sample8
Science
502
150
05713
76
23446
20561
02400
03558
02349
0184
36677
68888
557
05667
in
GRE-Analytical
5
366
24489
11446
12294
12395
Students
(5)
(5)
(-)
(-)
(5)
(-)
:I;
::;
:9’,
t-)
(7)
(4)
(2)
(2)
79
19
18
17
16
15
14
13
12
11
10
9
8
6
I;;
4
3
2
89
80
85
90
142
119
116
123
131
142
70123
11367
08901
05566
12457
1682
38
9
528
44888
51256
90224
24452
13357
7889
56778
4
89
15679
87
89
118
117
Read sample
standard
deviations
by combining
the
initial
digit(s)
with
sucessive
subsequent
Note.
example,
from the last
row in the distribution
of sample
standard
deviations
for
GRE quantitative,
without
regard
to size,
it
may be determined
that
there
were
five
samples
with
standard
deviations
(that
is,
standard
deviations
of 23,
26,
26,
27,
and 27).
digit(s)
within
each row.
For
for
samples
from departments
between
23 and 27,
inclusive
* First
example
six
,
ical
co lumn of parenthet
with
ve rbal
0 ne sample
entries
standard
represents
deviation
** Second
column
of parenthetical
entries
represente
from education
and one from political
science):
one
# Th ese
ar e eta ndard
devia tions
of scores
inte nded g radua te major
as math /science,
the dist
of 120,
ribut
ion of standard
and so
anoth er 110,
the dtstribution
science
sample
for
a genera 1 sample
bioscience,
a nd social
of
with
of foreign
science,
standa
a stand
deviations
on.
for
rd de viations
ard d eviation
GRE
examinees
tested
respectively
(Wilson,
the
for
of
five
127,
during
1984,
bioscience
samples.
social
science
and so on.
1981-82,
Table
9).
For
samples
classified
by
(four
Distributions
of
86
CRE
19
Standard
Primarily
Depar
13
11
179
10
9
8
Mdn
Depts
GRE-A
11
246
0
10
9
45
129
3446
11557
127
056
23
025555
001144668888
1229
99
1239
8
2
3
6
5
02349
9
4
0
01
3667
008
7
13
12
06
01235
44888
11
10
26
456889
16
8
024
03558
8
7
0229
0446778
366
124489
168
6
5
56
2
056679
4557
4
2
457
g
15+
GRE-A
12567889
15679
6
24
2
4
2
Hdn
-
GRE-Quantitative
89
Hdn
19
*
GRE-Analytical’
66
19
17
17
16
16
15
15
14
14
13
5
989
01568
16
13
12
5606
11
10
25924
626
57845
45689
9
91297
9
34599
11557
0229
67127
04467
78
Hdn
-
95
18
17
16
179
79
008
12
502
12
11
150
11
70123
528
10
05713
10
11367
44888
13357
9
76
23446
08901
51256
7889
366
9
8
05566
90224
56778
20561
24489
12451
24452
4
8
5
6
04235
6
6
02400
11446
68888
5
4
60255
899
552
5
03558
02349
12294
12395
557
3
2
0
Note.
in
28
1335789
022456778
2445
18
example,
without
(that
Students
35
9
18
8
-
N
GRE-Q
015
2
GRE-Verbal
13
[ZSL
Respectively
(Z)
19
15
14
Foreign
Size,
15
14
15
67
of
Sample
GRE-V
19
18
02
12457
Samples
t o
Regard
17
16
56
055669
Without
19
18
13
12
2
Department-Level
and
= 10-14
GRE-Q
5
11367
089
for
Sample
17
16
15
04
6
8
105
(27)
Scores
of
5
89
057
9
34599
Size
14
9
01568
259
578
Test
by
GRE-V
2
12
General
N
GRE-A
18
17
16
15
14
CRI?
tments,
5-9
N =
GRE-Q
V
of
ns
Deviatio
Quantitative
05667
79
89
15679
1682
38
9
0184
2
Read
sample
from
regard
is,
the
to
standard
standard
last
size,
row
it
deviations
deviations
by
combining
in
the
may
be
distribution
determined
of
26,
23,
36677
26,
the
of
that
27,
and
sample
there
27).
initial
standard
were
five
digit(s)
deviations
samples
with
successive
for
with
subsequent
CRE
quantitative,
standard
deviations
digit(s)
within
for
samples
between
23
each
row.
from
departments
and
27,
inclusive
For
Distributions
GRE Verbal
of Correlation8
between
GRE Quantitative
vs FYA
.a
.a
.J
.6
.5
.4
.3
.2
.l
lO
-. 0
-. 1
-. 2
-. 3
-. 4
-.
-. 5
-. 7
-. 8
-. 9
CRE Predictors
and FYA for 86 Department-Level
in Primarily
Quantitative
Fields
.J
29
7
0134450
0237
0224455518
011335779
13366899
123444587
235566708
.6
.5
.4
.3
.2
.1
.O
-. 0
-. 1
-. 2
-. 3
-. 4
-. 5
-. 6
-. 7
-* 8
-. 9
0012224488999
2244577
49
1267
2
2
23669
vs FYA
GRE Analytical
.8
01334456118
00124456619
02235556611188
12366661
0248899
0124466
012456888
0229
00038
38
0
Samples of BSL Students
.J
.6
.5
.4
.3
.2
.l
.O
-. 0
-. 1
-. 2
-. 3
-. 4
-. 5
-. 6
-. 7
001334 799
0245689
1333344566111889
11234456697889
01223334568
033667
56669
23
3489
669
0
06
0
5
-. 8
4
-. 9
Mdn*
.06
.36
.35
Wtd mean**
.10
.31
$27
Note.
vs FYA
4
68
Department-level
coefficients
are specified
by combining the initial
digit
(with decimal)
in each row with successive digits
in the same row. In the distributions
of correlations
between CRB verbal scores and first-year
average,
for example, coefficients
were
.72, .79, .67, .50, and so on.
Data are for engineering,
math and physics1 science,
and economics samples.
* Median of distribution
of 86 department-level
coefficients.
** Size-adjusted
averages of department-level
coefficients.
standardized
(z-scaled)
predictor
and criterion
variables
: :.,
:.: ._..
These coefficients
indicate
the relationship
for 1,216 ESL students in the 86 quantitative
between departmentally
departments.
Distributions
N < 10
.8
.J
.6
.5
.4
.3
.2
.1
.O
-. 0
-. 1
-. 2
-. 3
-. 4
-. 5
-. 6
-. 7
-. 8
-. 9
Mdn
Depts.
Note.
29
03458
13
33
12344
568
29
27
4
67
2
.03
(27)
CRE General Test Va lidity
Coefficients
for Foreign ESL Students,
of Department-Level
Data for 86 Departmenta in Primarily
Quanti tative
Fields
CRE Verbal vs PYA
N- lo-14
N = 15 plus
4
0237
0245558
13
9
7
56
012248899
2445
9
-.05
(34)
1
247
05779
16689
458
2378
04
7
.16
(25)
GRE Quantitative
N - lo-14
N < 10
.8
.J
.6
.!!I
94
.3
.2
.l
.o
-.o
-.l
-. 2
-.3
-. 4
-. 5
-.6
-. 7
-. 8
-.9
2
36
367
4467
356
6
48
6
14
2
0038
8
0
4
.36
(27)
269
0344578
0019
22577788
126
0
0
5688
0
0
.4J
(34)
vs FYA
N - 15 plus
1
256
056
3667
2899
12446
028
29
.28
(25)
by Size
of Sample:
GRE Analytical
vs PYA
N < 10
N - lo-14
N - 15+
.8 4
.J 8
.6 039
.5 68
.4 467889
.3 449
.2 23
.l 03
.O 669
-.o 3
-.l
9
-. 2-. 3-.4 6
-. 5-.6 5
-. 7
-. 8
-. 9
.39
(27)
Department-level
coefficients
are specified
by combining the initial
digit
(with decimal) in
sive digits
in the same row. In the distributions
of correlations
between GRE verbal scores
for example, coefficients
of .72, .79, .50, .53, .54, -55, .58, and so on, were obtained for
fewer than 10 (between 5 and 9) ESL students;
GRB quantitative
coefficients
for samples of
dents were .82, .J3, .76, .63, .66, .6J, 54, .54, .56, .57, and so on.
6
0179
0245
1333357
2589
245
67
2
34
669
0
34
9
467
1136678
013368
36
56
8
0
0
.40
(34)
.31
(25)
each row with succesand first-year
average,
departments with
fewer than 10 stu-