Brief Assessment Instruments William P. Wattles, Ph.D. Francis Marion University 1

Brief Assessment Instruments
William P. Wattles, Ph.D.
Francis Marion University
1
Brief therapy and assessment
• Managed Care emphasizes:
– Cost containment
– Documented treatment efficacy
2
Assessment
• A full test battery is not longer an option for
most practitioners.
• Psychologists’ biggest challenge is
demonstrating the financial efficacy of their
services.
3
3 Brief Instruments
• Treatment planning
• Outcome assessment
• Monitoring
4
Brief Instruments
• Brief Symptom Inventory (BSI)
• Beck Depression Inventory (BDI)
• State Trait Anxiety Inventory (STAI)
5
Brief Instruments
• Good reliability
• Adequate validity
• Good relevancy
6
Beck Depression Inventory
William P. Wattles, Ph.D.
Francis Marion University
7
• Measures self-reported depression
• Response bias may lead some to exaggerate
or minimize (or deny) symptoms.
• Diagnosis of depression requires
examination by a clinician.
8
Beck Depression Inventory -II
• 21-item self-report instrument
• Measures severity of depression in adults
and adolescents 13 and older
• Corresponds to criteria in DSM-IV
9
History
• Created in 1961
• Revised in 1996 after 35 years
• Based on descriptive statements reported by
psychiatric patients who were depressed but
not by other psychiatric patients
• Resulted in 21 items
• Original version read by trained observer
10
21 Items
•
•
•
•
•
•
•
•
Mood
Pessimism
Sense of Failure
Self-dissatisfaction
Guilt
Punishment
Self-Dislike
Self-Accusations
•
•
•
•
•
•
•
•
Suicidal Ideas
Crying
Irritability
Social Withdrawal
Indecisiveness
Body Image change
Work Difficulty
Insomnia
11
21 Items (cont)
•
•
•
•
Fatigability
Loss of Appetite
Weight Loss
Somatic
Preoccupation
• Loss of Libido
12
Revision
• Items dropped
–
–
–
–
Body Image change
Work difficulty
Weight loss
Somatic preoccupation
• Items added
–
–
–
–
Agitation
Worthlessness
Loss of energy
Concentration
difficulty
13
Items changed
• Old item
– insomnia
– loss of appetite
– Loss of libido
– Fatigability
• New Item
– changes in sleeping
pattern
– changes in appetite
– loss of interest in sex
– tiredness or fatigue
14
Critical items
• “The clinician should pay special attention
to the responses to item 2 (pessimism) and
Item 9 (suicidal thoughts or wishes) as
indicators of possible suicide risk
15
Administration
• Time: 5-10 minutes to complete
• Answers should be for the past two weeks
– extended from one week for DSM-IV
• Check to be sure that all items have been
completed
16
Scoring
• Each item is rated 0 - 3
• If more than one is selected choose the
higher figure
• Scores can range from 0 to 63
17
Interpreting Scores
• Choice of cut scores depends on purpose for
using the test.
18
• Sensitivity-the probability of correctly
classifying a respondent as depressed.
• Specificity-the probability of correctly
classifying a respondent as not depressed
19
Simple thinking
Not depressed
depressed
Number of symptoms
20
Simple thinking
Not depressed
depressed
Criterion
Number of symptoms
21
Reality
Not depressed
depressed
Number of symptoms
22
Not depressed
depressed
Criterion
Number of symptoms
23
Correct reject
False alarm
Number of symptoms
24
Miss
Hit
Criterion
Number of symptoms
25
Hits
97.5% 84%
False Alarms 84%
50%
50%
16%
Number of symptoms
26
Not depressed
depressed
Criterion
Number of symptoms
27
Truth about the Person
Decision
depressed
based on
Test
not depressed
Depressed
Not depressed
Sensitivity
Type II error
Type I error
Specificity
28
ROC Curves
• Receiver-operating characteristic curves
capture in a single graph the various
alternatives as you move the criterion to
higher or lower levels.
• False alarm rate on the X (horizontal) axis
• Hit rate on the Y (vertical) axis
29
ROC Curves
30
ROC Curves
• the area under the ROC (AUR) curves for the
BDI–II was .78 (95% CI, .68 to .86)
• The AUR is an overall index of the accuracy of
discrimination provided by a scale, and an AUR of
.50 represents chance discrimination..
• A BDI–II total cutoff score of 24 and above had
the highest clinical efficiency (72%) with a
sensitivity rate of 74% and a specificity rate of
70%
31
Hits
ROC Curves
False Alarms
32
Cut score threshold
• If purpose is to detect the maximum number
of persons with depression the cut score
threshold should be lowered.
• For research where it is important to obtain
a “pure” group, cut score should be raised to
minimize false positives.
33
Cut off scores
Total Scores
0-13
14-19
20-28
29-63
Range
minimal
mild
moderate
severe
34
Sample Results
Penn Sample
Mean
std dev
Nondepressed
7.65
539
Mildly depressed
19.14
5.7
moderately depressed
27.44
10
severly depressed
32.96
12
35
Psychometric characteristics
• Outpatient samples from four outpatient
clinics
–
–
–
–
277( 55%) Cherry Hill, New Jersey
50 (10%) Bala Cynwyd, Pennsylvania
127 (25%) Philadelphia, Pennsylvania
46 (9%) Louisville, Kentucky
36
Sample characteristics
• Gender
– Women 63%
– Men 37%
• Race
–
–
–
–
White 91%
Black 4%
Asian America 4%
Hispanic 1%
• Disorder
–
–
–
–
mood dis. 53%
anxiety dis 18%
adjustment dis 16%
other dis 14%
37
Depression Ratings
• All patients were diagnosed by experienced
psychologists or psychiatrists according to
DSM criteria
• Patient participation was voluntary with
informed consent.
38
Student Sample
• 120 intro psych students
– women 67%
– men 44%
• Predominately white
• mean age 19.6 (SD=1.84)
39
Reliability
• Internal Consistency
– Coefficient Alpha
• outpatients .92
• students .93
– All items correlated significantly with the total score.
– outpatients ranging from .39 (Loss of interest in sex) to
.70 (loss of pleasure)
– students ranging from .27 (Loss of interest in sex) to
.74 (self-dislike) .
40
Reliability
• Test-retest stability
– 26 Philadelphia outpatients administered BDIII a week apart.
– Test-retest r= .93
41
Validity
• Content validity-the construction process
consisted of 21 areas that differentiated
depressed from non-depressed patients.
• Content validity- BDI-II reworded and
added to assess DSM-IV criteria for
depression.
42
Validity
• Construct validity
– Correlation with original BDI r=.93
– Mean score 2.96 points greater than original
BDI
43
Validity
• Convergent validity
– correlates with Beck Hopelessness Scale
– r=.68
– correlates with Scale of suicide ideation
– r=.37
– Hamilton Psychiatric Rating scale for
depression
– r=.71
• Discriminate Validity-not measured
44
Validity
• The Beck Depression Inventory-II and the
Reynolds Adolescent Depression Scale (RADS;
Reynolds, 1987) were administered to 56 female
and 44 male psychiatric inpatients whose ages
ranged from 12 to 17 years old.
• The Cronbach coefficient alpha(s) for the BDI-II
and RADS were, respectively, .92 and .91 and
indicated comparably high levels of internal
consistency.
• The correlation between the BDI-II and RADS
total scores was .84,p <.001.
45
Item-option characteristic curves
• Each item demonstrates increasing
monotonic relationship with self-reported
depression
• Items 6, 9, 11, 21 show variation in rank of
items selected
– patients with sever depression unlikely to
acknowledge suicidal intent
46
Factorial Validity
• Identified two factors
– Somatic Affective
• loss of pleasure, crying, agitation, loss of interest,
indecisiveness, loss of energy, changes in sleep,
irritability, changes in appetite, concentration
difficulty, tiredness.
– Cognitive
• sadness, pessimism, past failure, guilty feelings,
punishment, self-dislike, self-criticalness, suicidal
thoughts, worthlessness
47
Race and Gender
• No differences found between white and
non-white scores
• Women scored higher than men:
– Men Mean= 20.44 (SD=13.28)
– Women Mean=23.61 (SD=12.1)
48
Sleeping and eating
• Changes in sleeping pattern
– 30% sleeping more
– 50% sleeping less
• Changes in eating
– 39% decreased appetite
– 20% increased appetite
49
BDI as outcome measure
50
State Trait Anxiety Inventory
STAI
William P. Wattles, Ph.D.
Francis Marion University
51
State-Trait Anxiety Inventory for
Adults
• The STAI is the definitive instrument for
measuring anxiety in adults. It clearly
differentiates between the temporary
condition of “state anxiety” and the more
general and long-standing quality of “trait
anxiety”.
52
STAI
• http://www.mindgarden.com/Assessments/I
nfo/staiinfo.htm
• The STAI has forty questions with a range
of four possible responses to each.
53
STAI
• Determines anxiety in a specific situation and as a
general trait
• Two twenty-item scales
• For individual or group administration
• Provides norms for clinical patients, high school
and college students, and working adults
• Efficiently scored
• Can be completed in about ten minutes
• Sixth grade reading level
54
The S-Anxiety scale
• The S-Anxiety scale consists of twenty
statements that evaluate how respondents
feel "right now, at this moment."
• 1 = Not At All 2 = Somewhat 3 =
Moderately So 4 = Very Much So
• A. I feel at ease
1 2 3 4
• B. I feel upset
1 2 3 4
55
The T-Anxiety scale
• The T-Anxiety scale consists of twenty
statements that assess how respondents feel
"generally."
• 1 = Almost Never 2 = Sometimes 3 =
Often 4 = Almost Always
• A. I am a steady person 1 2 3 4
• B. I lack self-confidence 1 2 3 4
56
STAI State Anxiety
• Evaluates how respondents felt at a particular time
in the recent past and how they anticipate they will
feel either in a specific situation that is likely to be
encountered in the future or in a variety of
hypothetical situations.
• Is found to be a sensitive indicator of changes in
transitory anxiety experienced by clients and
patients in counseling, psychotherapy, and
behavior-modification programs.
57
STAI State Anxiety
• Assesses the level induced by stressful
experimental procedures and by unavoidable reallife stressors such as imminent surgery, dental
treatment, job interviews, or important school
tests.
• For screening high school and college students
and military recruits for anxiety problems, and for
evaluating the immediate and long-term outcome
of psychotherapy, counseling, behavior
modification, and drug-treatment programs.
58
STAI State Anxiety
• Proven useful for identifying persons with
high levels of neurotic anxiety and for
selecting subjects for psychological
experiments who differ in motivation or
drive level.
59
STAI
• STAI-Y is a 40-item Likert scale
• Measuring State anxiety items 1-20
• And Trait anxiety items 21-40
60
Scoring
• Each item weighted 1-4 with approximately
half the items reverse scored.
• State and Trait scores can range from a
– Minimum of 20
– Maximum of 80
61
STAI Reading level
• 4th or 5th grade reading level
62
Interitem consistency
• Coefficient Alpha
– State
– Trait
.92
.90
63
Test-retest Reliability
• Test-retest reliability
– State
– Trait
.62
.84
64
Cut-off scores
• Not provided
• Trade off between
– Specificity-correctly identified anxietydisordered clients
– Sensitivity-correctly identified non-anxietydisordered clients
65
Norms
66
Clinical Norms
67
Cut-off scores
• The recommended cutoff for the STAI is 2
SD greater than the mean raw score adjusted
for age and gender. Grontkovsky et al. 2004
68
Cut-off scores
69
Cut-off scores
• A cut-off of 39-40 is normally used for
clinically significant symptoms of a state of
anxiety.
• In pre-operative patients the cut-off was set
at 44/45
70
Cut-off scores
• Geriatric in-patients. Cut-off 54/55 optimal
– 7 false positives, 2 false negatives
71
Cut-off
• I don't think any cutoffs exist. It seems that
the measure is only scored continuously,
which would make sense since scores vary
widely according to normative group, and
the measure doesn't assess a diagnosable
disorder.
72
• Cutoffs for an anxiety scale can be of two kinds.
The first is just to take the top and bottom thirds to
form contrasting groups to be participants in an
experimental study. The second is the question of
whether they are high or low anxiety. I divide the
responses by the number of items (unless already
done so in the scoring) to put them back on the
original scale, and then those whose scores are
above the midpoint are anxious and those below
the midpoint are not anxious.
• Rich Gorsuch
73
Symptom Check List
William P. Wattles, Ph.D.
Francis Marion University
74
Symptom Check List
• The Symptom Check List (SCL-90-R)
–
–
–
–
–
6th grade reading level
List of symptoms versus personality measure
90 symptoms
12-15 minutes
Over 1,000 studies published
75
Symptom Check List
• Descriptive rather than interpretive data
make for a straightforward interpretation.
• T score great than 63 suggest clinically
significant levels of psychological distress.
76
Global Indexes
• Global Severity Index
– Combined rating considers intensity and number of
symptoms
• Positive Symptom Distress Index
– A measure of intensity. An average rating for all
symptoms
• Positive Symptom Total
– A measure of the number or breadth of symptoms.
77
SCL-90-R Symptom Dimensions
• Somatization- Distress is primarily
experienced through physical problems.
Somatization is an ineffectual coping
mechanism.
• Obsessive-Compulsive- Focuses on
irresistible, repetive, unwanted impulses
thoughts and actions.
78
SCL-90-R Symptom Dimensions
• Interpersonal Sensitivity- Low self-esteem,
negative expectations in social setttings.
• Depression- include full range of depressive
symptoms.
• Anxiety- Focuses on apprehension,
nervousness, and dread. Includes physical
components such as rapid heart rate and
tension.
79
SCL-90-R Symptom Dimensions
• Phobic Anxiety-Excessive or irrational fear
of person, place or thing.
• Paranoid Ideation-Hostility projection,
grandiosity, suspiciousness, need for
control. Delusions may be present
• Psychoticism-Person may be withdrawn,
isolated and experiencing hallucinations.
80
81
82
• The SCL-90-R test is normed on 4 groups:
–
–
–
–
Adult psychiatric outpatients
Adult nonpatients
Adult psychiatric inpatients
Adolescent nonpatients
83
(Brief Symptom Inventory)
• Shorter version of SCL-90-R
84
The End
85