Three year outcomes in tertiary care

Transcription

Differentiating Three Year Outcomes Following Tertiary Child and
Youth Inpatient Psychiatric Treatment
Investigators:
Jeff St. Pierre, PhD., C. Psych
Shannon Stewart, PhD., C. Psych
Christine M. Cullion, MA
Child and Parent Resource Institute
Alan W. Leschied, PhD., C. Psych
The Faculty of Education
The University of Western Ontario
London, Ontario CANADA
Research Coordinator: Steve Cook, MA
Research Assistants: Kendra Aaronson, Lisa Clarke, John Noftle, Donna Skrodzki, Carol
Stocker and Aryan Wahab
Statistical Consultant: Andrew Johnson, PhD, UWO
September 2008
2
Three Year Outcomes in Tertiary Care
Acknowledgements
We would like to thank all the children and their families who face these struggles on a daily
basis, and were willing to share their lives with us. Many clinicians, support staff and managers
at CPRI, and teachers throughout southern Ontario have helped answer questionnaires, collect
data, or supported our work on this study over the past six years. Our research team worked long
hours, and demonstrated a dedication to both understanding the needs of children and youth, and
to double checking our numbers for accuracy. Several colleagues assisted by brainstorming or
reading drafts of this work; we thank them for improving our research. If readers see an error,
would like to discuss the findings, or have a question about the results of this study, please
contact the first author at CPRI.
[email protected]
September, 2008
CPRI-UWO 2008
3
Table of Contents
List of Tables ……………………………………………………………….
List of Figures ……………………………………………............................
6
7
Executive Summary ………………………………………………………..
8
Introduction ………………………………………………………………... 12
Literature Review ………………………………………………………...... 13
Natural History of Child Psychopathology.............................................. 13
Developmental pathways of serious mental health disorder ……..... 13
Continuity of externalizing and internalizing behaviour …………... 14
Situating Out-of-home Treatment within the System of Care................. 15
The status of the residential/inpatient treatment outcome literature for
children and youth …………………………………......................... 16
Methodological Challenges for Follow-up Studies in Tertiary Care …. 19
Research Design Challenges ………………………………………. 19
Measurement ……………………………………………………….. 20
Statistical Analysis …………………………………………………. 20
The Present Study …………………………………………………... 20
Method ……………………………………………………………………....
Description of Sample ………………………………………………….
Referral Route to CPRI ………………………………………………...
Mental Health Treatment at CPRI ……………………………………..
Clinicians at CPRI ………………………………………………….
Description of Measures and Timelines ………………………………..
Measures ……………………………………………………………….
The Brief Child and Family Phone Interview ………………….........
Child and Adolescent Functional Assessment Scale …………….......
The Conners’ Rating Scales – Revised .................................................
The Social Skills Rating System ………………………………...........
The Parenting Stress Index ……………………………………..........
Data Analysis …………………………………………………………...
21
21
22
22
23
23
24
24
25
25
26
26
26
Results ……………………………………………………………………....
Appreciating sample characteristics in psychiatric milieu therapy …….
Services received ……………………………………………………
Summary of Symptoms from the BCFPI admission scores …………
School functioning pre- admission ………………………………….
Mental health needs at pre-admission ……………………………...
Family coping pre-admission ……………………………………....
Summary of needs at intake ………………………………………...
Treatment Outcomes …………………………………………………...
Placement …………………………………………………………..
Child and Youth Mental Health Symptoms ………………………….
28
29
29
33
33
35
36
37
37
38
39
CPRI-UWO 2008
4
Child and Family Wellness …………………………………………. 42
School Outcomes ……………………………………………………. 45
Positive Indicators …………………………………………………... 46
Differential Outcomes: Heterogeneity of change over three years in a
system of care …………………………………….............................. 47
Discussion …………………………………………………………………...
Strengths and Limitations ..................................................................
Future Directions ................................................................................
Deliverables ........................................................................................
54
61
64
68
Appendices …………………………………………………………………... 71
Appendix A BCFPI Data Set ……………………………………………... 72
Reliability Statistics as a measure of Internal Consistency
on symptomatology subscales in the Brief Child and Family
Phone Interview (BCFPI) …………………………………… 73
Appendix B Clinical admissions by county of referral …………………...
74
Appendix C CPRI data collection ………………………………………...
76
Appendix D Knowledge Transfer ………………………………………... 78
Publications, Presentations, Poster, Student Theses, CPRI Newsletters
Abstracts of student theses based on The Center of Excellence Support for Research at CPRI
Julia Oosterveen: Client satisfaction with mental health services:
A qualitative approach to understanding the family’s experience
of residential care …………………………………………… 83
John Noftle: The trajectory of change for children and youth within
residential treatment ................................................................ 85
Julie Anne Gerrits: Examining the impact of residential treatment:
Relevance of family and the out-of-treatment environment ..... 86
Sarah Yaremko: Gender informed analysis of residential treatment
outcomes ................................................................................... 87
Donna Skrodzki: Wards of the state: Examining residential treatment
outcomes for children involved in the child welfare system ..... 88
Jennifer Cordick: A longitudinal assessment of the trajectory of change
for children and youth in mental treatment …………………. 89
Nadia Tanel: Longitudinal assessment of global functioning among
high risk high need children: The impact of stimulant
medication ………………………………………………….. 91
Aaryann King: And investigation of the relationship between academic
achievement and emotional and behavioural disorders in a
residential population………………………………............. 93
CPRI-UWO 2008
Appendix E
5
Capacity Building: Further Funded Research Based on the
Centre of Excellence Grant ………………………………….. 94
1. Investigating the Impact of Intensive Residential Treatment on
Children Referred from the Child Welfare System: The Nature of Maltreatment.
Dr. Alan Leschied, Dr. Shannon Stewart, Canadian
Institutes for Health Research …………………………………… 95
2. Short-Term Residential Treatment: Effectiveness of Intensive
Intervention for Children with Complex Mental Health Needs.
Dr. Shannon Stewart (P.I.), Centre of Excellence Program
Evaluation Grant, 2007. ………………………… ……………… 98
3. Diversion from inpatient child psychiatry. Dr. Alan Leschied,
Dr. Jeff St. Pierre, University of Western Ontario SSHRC Internal
Research Grant Application, fall 2008. ………………………….. 99
Appendix F Advisory board ………………………………………………… 100
References …………………………………………………………………… 102
CPRI-UWO 2008
6
List of Tables
Table 1
Percentage of Children Receiving Mental Health Services Other than the
Target Admission ........................................................................…………… 29
Table 2
Psychotropic Medications Prescribed at Admission in Descending order of
Sample Usage within Each Drug Class …………………………………….. 33
Table 3
CAFAS Rating of School Functioning ……………………………………... 34
Table 4
Discharge Living Placement ………………………………………………... 38
Table 5
Discharge Guardian ………………………………………………………… 38
Table 6
BCFPI Repeated Measures T-scores over a three year period in the system
of care with Effect Sizes ……………………………………………………. 41
Table 7
CAFAS Repeated Measures subscale scores over a three year period…….... 44
Table 8
Social Skills Rating System - Parent ………………………………………. 47
Table 9
Social Skills Rating System – Teacher ……………………………………... 47
Table 10
Model Fit Information for Latent Class Analysis ………..…………………. 48
Table 11
Latent Class Cluster Service Differences …………………………………... 53
Table 12
Latent Class Cluster differences at end of study …………………………… 53
CPRI-UWO 2008
7
List of Figures
Figure 1
Percentage of Children on Psychotropic Medication ………………………. 32
Figure 2
Inpatient and Outpatient BCFPI Broad Band T Scores …..........................… 33
Figure 3
Diagnostic Count at Discharge ..............................………………………..... 36
Figure 4
BCFPI Broad Band Scales Over Time ……………………………………... 40
Figure 5
Four Time Points …………………………………………………………… 42
Figure 6
Adaptive Functioning Over Time …………………………………………... 43
Figure 7a
Two Latent Class Cluster Trajectories Over 3 Years ………………………. 49
Figure 7b
Two Latent Class Cluster Trajectories Over 3 Years ………………………. 50
CPRI-UWO 2008
8
Executive Summary
This evaluation reflects scientific progress in understanding the long term impact
of early onset comorbid childhood mental health disorders, while describing outcomes
from multiple perspectives. Tertiary mental health support for children, youth, and their
families requires an appreciation of the phases of child development, family coping, the
natural course of psychopathology, along with an understanding of systems of care.
Coordinating classroom, family, and individual supports over time requires ongoing
communication between multiple helpers. Applied research in tertiary mental health
therefore entails a longitudinal monitoring of the fit between disabling symptoms and
environmental supports, collecting reports across home, community and school. Further,
for those children and youth involved in out-of-home treatment due to significant
psychiatric symptoms or for safety to themselves and others, there is an increased duty to
monitor their well being and any positive or negative impacts attributed to the out-ofhome care.
In this study, all admissions between spring 2002 and fall 2006 to a regional
mental health psychiatric milieu treatment program at CPRI in southern Ontario were
tracked for symptoms, adaptive functioning, and services for three years following their
intake interview. These clients, aged 6-17 at the time of referral, represent a complex
sample of youth with chronic and severe acting out behaviour problems often
accompanied by diagnosed mood concerns and significant family disruption. We
compiled at intake two or more psychiatric diagnoses for most clients. Neurological
deficits in impulse control, attention and learning problems in the classroom, social
aggression and social rejection, observable handicaps in their interpersonal behaviour
according to parents and teachers, along with significant levels of anxiety and mood
dysregulation, were accompanied by exposure to trauma, as many had either witnessed
family violence or were themselves victims of violence or abuse. Over 20% of the 225
children and youth admitted were wards of a Children‘s Aid Society in Ontario. The
targeted admission was unique in that it was on average four months in length, offering
multidisciplinary treatment, skill development and school programming not typically
available in one setting, while allowing for intensive family involvement.
Multiple agencies from 17 counties across Ontario feed into this family focussed
psychiatric inpatient service when community treatment gains are stagnant or a family or
child is in crisis of home or school breakdown. Communities are to reserve these beds for
those with the most intensive needs by utilizing a multiple gating intake procedure. This
was found to be true as standardized intake reports from parents on the Brief Child and
Family Phone Interview (BCFPI), clinicians on the Child and Adolescent Functional
Assessment Scale (CAFAS), and teachers on the Conners‘ Rating Scales and Social
Skills Rating Scales indicated extremely high scores, often one standard deviation above
those of outpatient mental health clinic referrals. While as tertiary care clients it was
CPRI-UWO 2008
9
anticipated that these children and youth would have already been exposed to treatment
efforts, the extent of service provision measured over time was staggering. In this group,
all had received mental health services prior to admission, typically beginning at age six,
and 71% had already been out-of-home for treatment or child welfare protection, 94%
were on psychotropic medication, and 44% had police contact. Parent and teacher rating
scales indicated extreme concerns with behaviour, impulse control, mood, social skills,
accompanied by exclusion from normal community and school activities. The majority
were receiving special education supports for behaviour, social-emotional and learning
difficulties. Parents reported extreme levels of stress and family conflict, with evidence of
trauma and depression in children and youth and their families.
Given the severity of need, understanding the percentage of referents that improve
over time in the classroom, in family relationships, and in general well being, is essential
research. A review of the natural history of psychopathology indicates there is continuity
in behaviour problems and mood concerns seen in a substantial proportion of children
who experience mental illness. Unfortunately, a literature review indicated less than
adequate research attention has been paid to tertiary system of care attempts to alter the
negative trajectory of outcomes seen in those struggling to cope with more than one
presenting problem or risk factor. There are now reviews of residential treatment
programs (group care of typically more than one year) indicating that a majority of clients
demonstrate gains in terms of reduced behavioural difficulties or better adjustment.
Attempts to briefly summarize this tertiary care literature are difficult due to differing
samples, the differences in treatment offered, and the lack of experimental controls,
however generally it is clear that some but not all children and youth benefit from these
treatments, the gains may dissipate over time, and not all out-of-home care is equally
beneficial. The present investigation attempted to add to the existing literature by:
following up clients for a longer time following discharge from four months of intensive
child and family multidisciplinary treatments, by measuring outcomes in multiple ways,
and by describing the nature of services received.
To distinguish different trajectories of outcome, latent class cluster analysis was
used to predict changes in both BCFPI and CAFAS scores over time. Of those admitted,
170 individuals agreed to be interviewed two years post-discharge from the targeted
admission. Distinct developmental trajectories over time were identified, confirmed by
independent informants and cross-setting reports. Two thirds of subjects demonstrated
significant symptom reduction and functional improvement over the course of three
years, extending the large to moderate effect sizes seen in related research. High ongoing
service usage was highlighted, even in the improved group, an area of the literature often
neglected. For example, half of the improved group were out of home again for
treatment, child protection, or incarceration in the two years following discharge, 70% of
the non-improved group. Police contact for violent behaviour was at 48% for the group
we distinguished as not improved, only 15% for the improved group. Strikingly, roughly
CPRI-UWO 2008
10
80% of both groups were still in school, a testament to the education system that
education is offered to all. It was noted that at intake the non-improved group had the
highest levels of symptoms, more classroom difficulties and reported greater caregiver
distress. These children and youth were truly handicapped over time by their disabilities
in interpersonal functioning.
Initiated as an attempt to study the longitudinal outcome of an intensive familyfocussed psychiatric inpatient treatment, the results indicate that describing the ‗outcome‘
of one treatment while others remain underway requires modern multivariate statistical
modelling with more frequent data tracking, as there was some evidence of a doseresponse relationship to treatment for certain subjects. A developmental psychopathology
chronic care model seems best suited to understand the needs and service demands over
time in tertiary populations experiencing severe early onset psychopathology, life
disruption, and family dysfunction or breakdown leading to out-of-home treatment.
This report reviews the literature on child and youth developmental
psychopathology and intensive inpatient treatments, and maps the life stories of 170
children and youth over three years. We offer the following recommendations for tertiary
care, with a check-mark offered where we feel the southern Ontario service system is
particularly well-positioned:
 Triage or screen low symptom profiles away from the intensive, expensive
tertiary care system 
 Measure symptoms, functioning, skill change and service usage over time
using structured, standardized measurement tools
 Include at least two pre-measures and two post-measures to allow for a true
wait-list ―control‖ comparison to periods of intensive treatment or placement
change
 Research in different regions across Ontario is needed to begin to track the
nature of treatments provided, including access to evidence based treatment
 Track service usage in tertiary care with a ‗quick‘ phone check-in every three
months
 For applied research, record multiple perspectives on valid standardized scales
to fully understand needs and outcomes. Presently, risk ratings on the CAFAS
identified different trends from the BCFPI symptom scales 
 Use prospective samples, with consent to track and contact involved agencies
obtained at the outset, using a common unique identifier across the Ministry
of Children and Youth, Ministry of Health, Ministry of Education, and
Ministry of Community and Social Services
 For school services (a key service provider), a report card and IEP analysis
would be beneficial to understand academic outcomes, days absent,
suspensions, and social functioning.
CPRI-UWO 2008







11
There is evidence that treatment classrooms are required that can both weather
periodic aggression and program for academic progress that will change from
one day to the next. This is due to ongoing changes in availability for learning,
which is caused by active psychiatric symptoms and executive functioning
deficits, and in-home conflicts
Planners must build community wide linkages in forming a system of care
A number of tertiary care clients require long term, ongoing care and do not
benefit from discrete, disjointed service episodes, which place families back
on wait lists for support
Children and youth referred to CPRI for inpatient services typically access
out-of-home care a number of times in their lives. At least half will again do
so after leaving CPRI. This indicates that community services must consider
the nature and costs of tertiary follow-up services they are prepared to offer at
the time of referral. For these clients, a short stay in treatment is clearly not
able to eliminate long term mental health and child welfare service access, but
is geared to stabilize and support crisis symptoms safely. Other models of
intensive services (e.g., multidimensional treatment foster care, multisystemic
therapy) must be considered, as Ontario currently has no clear system to
handle ongoing aggression in children and youth without relying on
fragmented community service systems or Children Aid Societies to fund
placements.
Staff training in tertiary inpatient mental health should focus on linkages and
continuity of care, with some attention paid to ongoing staff morale knowing
that presently one third of clients will show little response to treatment and
will return to treatment providers to utilize intensive services on more than
one occasion
Graduate level social science students should be welcomed into applied
clinical-research settings, to experience messy ―real world‖ data sets. Students
can thus enhance their awareness of the need for multivariate statistical
methods that can handle data in multiple forms, change over time, and more
than one related covariate with differential attrition
Scientist – practitioner relationships, such as the current one between CPRI
and the University of Western Ontario that was fostered by the Provincial
Centre of Excellence for Child and Youth Mental Health, should be assisted in
terms of time allocation and dollars 
CPRI-UWO 2008
12
Introduction
Since the advent of the 21st Century, a clarion call has been issued to advance an
evidence-based practice approach in children‘s mental health. Tertiary care for children
and youth who present with multiple, complex and ongoing mental health, educational
and social behaviour difficulties has yet to be a part of this revolution in accountability
for treatment outcomes. These clients, who are seen more often by clinicians, and receive
the most expensive clinical resources our communities have to offer, are too often
ignored when it comes to intensive research (Barth, 2005; Curry, 2004). This
juxtaposition of clinical demand and research neglect can be understood in part due to
research method challenges that overwhelm the most talented of applied mental health
researchers. Tertiary psychiatric care should by definition be offering treatment programs
to those clients who failed to benefit from earlier evidence based treatments, which are
just recently being implemented broadly in North America (APA, 2008). In child and
family tertiary care research, multiple, overlapping and longitudinal treatment initiatives
must be monitored across agencies with differing mandates and governance, with a
dynamic definition of who is in charge, who is the client, and when does treatment
become active.
The framework for such research does exist. Investigations into the social ecology
of developmental trajectories by colleagues in the field of adolescent delinquency (e.g.,
Broidy et al., 2003), can direct clinicians involved in tertiary mental health services.
Research into tertiary care treatment outcomes would seem to require an ‗arranged
marriage‘ of social science, statistical expertise, clinical acumen, and system
administration. Research grants are difficult to obtain when sampling characteristics are
seemingly impenetrable, tertiary clinical treatments cannot be summarized in a manner
readily coded by scientists, and statistical complexities require modeling understood by a
select few with access to what is all too often obscure software unknown outside of
academia.
The present investigation began as a clinical program evaluation of inpatient
services at the Child and Parent Resource Institute (CPRI), a tertiary, regional child and
youth mental health centre in London Ontario. In its beginnings, a $5000 seed grant was
used to launch the evaluation, which subsequently evolved into an applied research
project tracking post-discharge outcomes. The present longitudinal investigation is a
result of a CPRI-University of Western Ontario (UWO) partnership funded by a grant
from Ontario‘s Provincial Centre of Excellence for Children‘s Mental Health. This
funding offered yet a further step in understanding the tertiary care needs, services and
outcomes over three years in the lives of children and youth and their caregivers.
CPRI-UWO 2008
13
Literature Review
The following literature review places the needs of tertiary care children and youth who
utilize inpatient treatment in a context that appreciates the natural history of childhood
psychopathology. This is followed by a depiction of the current research into tertiary
services, focusing on commonly used out-of-home or group care intervention methods.
These literatures are reviewed relative to the impact of treatment in reducing the
symptoms and sequalae of serious childhood/youth mental health disorder, while also
acknowledging the research obstacles to predicting the true course of child and family
development in the face of multiple dynamic and static stressors.
Natural History of Child Psychopathology
Literature regarding the natural history of psychopathology with children indicates that
that neuro-behavioural problems of impulse control, mood regulation, and aggression
tend to persist over time for a substantial number of children referred for mental health
services. This has significant implications for the nature of service delivery.
Developmental pathways of serious mental health disorder. Child psychopathology is
considered heterogeneous with respect to both its etiology and outcome. Most forms of
psychopathology coexist with other disorders, which makes it difficult to determine
boundaries between the categories of disorders (Lilienfeld, 2003). Despite the instability
of diagnostic categories from childhood to adulthood, broad band forms of
psychopathology represented in internalizing and externalizing disorders demonstrate
considerable continuity (Bastiaansen, Koot, & Ferdinand, 2005; Stanger, MacDonald,
McConaughy, & Achenbach, 1996) and are highly persistent and stable over time
irrespective of the specific diagnosis (Birmaher & Axelson, 2006; Zeitlin, 1986). For
example, medium to high stability has been shown for a broad range of psychopathology
in referred (e.g., Heijmens Visser, Koot, & Verhulst, 1999; Heijmens Visser, Van der
Ende, Koot, & Verhulst, 2003) and non-referred treatment samples of children and youth
(Hofstra, Van der Ende, & Verhulst, 2000; Koot, 1995).
The growth in the field of developmental psychopathology reflects an increasing
realization that many childhood problems have life-long consequences and costs for
children, their families and society. Further, most mental health disorders experienced in
adulthood are rooted in early childhood experience. These determinants from early
childhood reflect intra-individual, inter-individual and macro-systemic forces reflecting
genetic factors, neurological insults, cognitive deficits, temperamental dispositions,
parental psychopathology, dysfunctional parenting practices, stress, poverty and lack of
social support (Mash & Dozois, 2003; K. H. Rubin, Stewart, & Chen, 1995; Rutter,
2005).
CPRI-UWO 2008
14
One of the primary developmental pathways leading to serious mental health
problems in adolescence is established by the pre-school period (Caspi, 2000; Moffitt &
Caspi, 2001). Behavioural problems reflected in aggression, non-compliance and
defiance are major mental health problems demonstrating stability from toddlerhood
through to adulthood (Kim-Cohen et al., 2003). Evidence from community-based
samples suggest that difficult temperament and behavioural problems exhibited as early
as two years of age are predictive of continued problematic behaviour that can last into
later childhood and adolescence (Burke, Loeber, & Birmaher, 2002; Dodge & Pettit,
2003; Hill, 2002). Children with severe mental health difficulties tend to struggle early,
often and throughout their school careers (Nelson, Stage, Duppong-Hurley, Synhorst, &
Epstein, 2007) and are more likely to have problems in interpersonal functioning along
with an increased rate of psychiatric problems (Moffitt, Caspi, Harrington, & Milne,
2002). Such pervasive effects on individual development have been shown to restrict lifelong opportunities (Fergusson, Horwood, & Ridder, 2005a, 2005b).
Continuity of externalizing and internalizing behaviour. Despite considerable variation in
the children‘s age and length of follow up, research is surprisingly consistent with respect
to persistence rates for both externalizing disorders, 40-56%, and internalizing disorders,
23%-57% (Briggs-Gowan et al., 2003; Lavigne et al., 1998). For example, patterns of
oppositional symptomatology, co-morbidity and persistence of disorders at a two year
follow-up in a clinic-referred group of preschool boys reflected 76% of the children were
diagnosed with Oppositional Defiant Disorder, Attention Deficit Hyperactivity Disorder
or both. Additionally, 25% had other diagnoses as well, primarily anxiety and/or mood
disorders (Speltz, McClellan, DeKlyen, & Jones, 1999).
There is additional evidence suggesting a continuity of behavioural problems,
specifically physical aggression, from childhood to adolescence (Broidy et al., 2003;
Tremblay et al., 1992). Loeber, Green, Lahey, Christ & Frick (1992) for example found
that when comparing younger boys to older boys, the younger boys demonstrated the
highest level of disruptive behaviour and progression of onset from less to more serious
mental health symptoms over time. Research has also suggested that, at least for boys,
childhood disruptiveness is one of the strongest predictors of adolescent and adult
criminality (e.g., Fergusson & Horwood, 1995). In one large scale longitudinal follow up
study designed to test the developmental course of a broad range of behavioural and
emotional problems in a sample of youth aged 4 to 16 years, considerable continuity and
persistence was identified with respect to mental health problems. Specifically, 41% of
children and youth identified as problematic with respect to their mental health, remained
at risk at a 14-year follow up suggesting that child and adolescent problems can persist
into adulthood (Hofstra et al., 2000).
Similar results have been reported in studies focussing on internalizing disorders.
Although studies vary across time spans, developmental periods, methodology and
CPRI-UWO 2008
15
measures (Rubin, Burgess, Kennedy, & Stewart, 2003; Rubin & Stewart, 1996), results
suggest that the most common internalizing disorders, anxiety (Albano, Chorpita, &
Barlow, 2003) and depression (Hammen & Rudolph, 2003), are stable over time. For
example, a group of prepubertal children and postpubertal youth assessed for
psychopathology at two year time intervals over a five year period had similar
internalizing symptomatology, severity of index episodes and recurrence (40%)
(Birmaher et al., 2004). Other researchers report a 50% persistence rate for major
depressive disorder (Goodyer et al., 1997). Furthermore, pervasive impairment of
seriously emotionally disturbed children and youth occurs in approximately 10% of
clinical referrals (Epstein, Kutash, & Duchnowski, 2005). As such, evidence suggests that
certain internalizing disorders experienced in early childhood place a child at risk for the
development of co-morbid conditions, and, if left untreated, lead to chronic mental illness
into adolescence and adulthood (Albano et al., 2003).
In summary, research suggests that up to 50% of untreated childhood disorders
will persist into adulthood (Goodyer et al., 1997; Hofstra et al., 2000), and 10 % of these
children will continue to display severe and pervasive impairment throughout their life
course (Epstein, Kutash, & Duchnowski, 2005). This extensive literature over the past 15
years suggests there is a continuity of behavioural and emotional problems in clinicreferred children and youth, and that these problems should be viewed as chronic in
nature for some individuals.
Situating Out-of-Home Treatment within the System of Care
Similar to other service delivery systems, services for children and adolescents who
experience mental health disorders are organized within a framework that attempts to
ensure that those most in need receive the most intensive services, when appropriate and
available. In this framework, children and youth who receive out-of-home treatment
should have received previous intervention supports. In order to appreciate the position of
intensive treatment providers such as CPRI within this hierarchical system of care, the
following section briefly summarizes the nature of the primary, secondary and tertiary
care systems that have been developed to provide support for children /youth who
experience serious mental health disorder.
Primary prevention programs are delivered to the general population without
consideration for the special needs of any segment within that population and target the
potential onset of a disorder(s). Epidemiological research at this level helps identify
broad predictors of risk, for example poverty. With respect to childhood aggression,
forms of primary prevention would include a school-based program on media literacy,
since evidence suggests that exposure to violent content increases the likelihood that
childhood aggression will be displayed.
Secondary prevention programs target the risks and needs of youth who have
developed the risk for a disorder but who have as yet not displayed the behaviour that
CPRI-UWO 2008
16
would normally correspond with it. Staying with the example of childhood aggression,
this group of interventions will include targeting children and youth who have
experienced risk factors such as a history of maltreatment or victimization through dating
violence and due to this past experience hold an increased potential to display aggression
as a result.
The present study focuses on a tertiary population. Tertiary care programs are
directed at those clients who have already evidenced behaviours that are significantly
impairing their social or academic functioning. Children and youth seeking services
within the tertiary care system may have experienced both primary and secondary
services. In addition, individuals who do not make gains in clinic based visits, or
experience aggression toward themselves and others, may require an altered environment
to support their needs safely. A description of the successful provision of tertiary care in
serious child and youth mental health is provided in the following section.
The status of the residential/inpatient treatment outcome literature for children and youth
A decade has passed since Alan Kazdin and John Weisz‘s seminal article regarding
treatment outcomes for emotionally disturbed children: ―…empirically supported
treatments, even in their current state, are preferable to practice procedures that lack
supporting evidence‖ (Kazdin & Weisz, 1998, p. 31). The vast majority of the published
evaluative literature since has focused on community-based services, often for unitary
presenting problems, while largely ignoring the ongoing common use of tertiary and outof-home services for complex mental health concerns. Within the system of care, acute
psychiatric hospitalizations and long term residential placements for children and youth
are often requested by caregivers and teachers who are frightened by emotionally
disturbed children and/or youth who exhibit violence (Connor, Miller, Cunningham, &
Melloni, 2002). Children referred to residential treatment centers are referred from
numerous referral sources including county welfare departments, mental health services
and corrections (Libby, Coen, Price, Silverman, & Orton, 2005). Within the present
system of care, intensive residential treatment is often the final treatment option for
seriously emotionally disturbed children and youth.
Residential ―group care‖ in its various forms has come under increased scrutiny
due to high costs, the overuse of unregulated agencies who fail to provide adequate and
validated treatments, and often administer such interventions to children without offering
a ―less restrictive‖ alternative. Too often, these treatment providers fail to describe
treatments, record fidelity and track outcomes, i.e., the standards of evidence based
practice (Hair, 2005; Hurley & Goldsmith, 2007; Lyons & McCulloch, 2006). However
the rallying cry to eliminate out-of-home treatment for mentally ill children and youth,
while focused on ‗dollars and sense‘, may not be led by the families of those at risk of
harming themselves or others (Ainsworth & Hansen, 2005). Nor apparently is it led by
those who have recently reviewed the literature. Current reviews continue to indicate that
CPRI-UWO 2008
17
a need is met by tertiary group care. Meta-analyses of the outcome literature in North
America, the United Kingdom and Europe report that while residential care is deemed as
a ―radical intervention‖ and ―last resort‖ the evidence also suggests:
―the main conclusion that can be made from our meta-analysis [1990-2005] is
that children and youth, after a period of residential care – on average – improve
in their psychosocial functioning‖ (Knorth, Harder, Zandberg, & Kendrick, 2008,
p. 133);
―residential treatment outcome studies from 1993 to 2003 … showed that children
and adolescents with severe emotional and behaviour disorders can benefit and
sustain positive outcomes from residential treatment‖ (Hair, 2005, p. 551);
―a review of available studies of the effectiveness of residential treatment
delivered in group home settings and residential treatment centres concludes that,
despite methodological shortcomings and variability in programming, residential
services do improve functioning for some, but not all, youth.‖ (Frensch &
Cameron, 2002, p. 307).
Significantly, a closer look at the evidence in the three literature reviews just
quoted finds several trends. First, the lack of regulated and evidence based practice in
group care can lead to vastly different treatment effect sizes. Lyons and McCulloch
(2006) effectively demonstrate significantly different outcome trajectories in a U.S. statewide analysis of a large number of agencies offering residential (long term) care. While
some sites demonstrated excellent outcomes, some showed no such gains or even
worsening symptoms over time. Second, long term outcomes are seldom measured, and
when they are there is evidence of returning symptoms and problems over time, as
demonstrated thoroughly in the developmental psychopathology literature reviewed
earlier. Third, it is difficult to find core predictors of risk across differing tertiary
samples, especially if the sample characteristics are ill-defined and outcomes are not
tracked by multiple means. This is in contrast to diverse population samples where the
basic risk markers such as poverty are more readily identified.
These reviews indicate there is a need to track developmental trajectories across
multiple informants, and that this needs to occur beyond the one year post-discharge
period. This follow-up also requires well defined sample characteristics and parameters
of treatment usage are included in the analysis. In longitudinal investigations of tertiary
samples, subjects experience changing levels of both community and group care over
time. Green, Jacobs, Beechman, Dunn, Kroll, Tobias & Brickman (2007) note that as the
‗agent of change‘ is complex and dynamic in this area of research, quasi-experimental
designs are still required to delineate basic measurement issues and differential outcomes,
two goals of the current study.
The present investigators had difficulty finding an adequate literature comparison
to our Southern Ontario tertiary system of care sample. Complex sample characteristics
CPRI-UWO 2008
18
and vastly divergent definitions of residential, inpatient, out-of-home or group care
hampered the search for a literature-based control that would serve as a comparison. It is
challenging to compare outcomes (Hussey & Guo, 2002; Phillips et al., 2000) with the
array of services that are offered within the residential network, and across the diversity
of clients served.
In terms of outcome studies, one investigation most closely resembled the
sampling and treatment characteristics in the present study. Green et al., (2007) reported
on 150 children and youth across 8 inpatient psychiatric units in the U.K. Subjects
received an average of 16 weeks of multidisciplinary assessment, milieu and psychiatric
therapy, and special education. They were followed for one year post discharge, with
significant improvements in symptom reduction reported. The present research study,
while initiated prior to an awareness of Green‘s investigation, parallels that study in terms
of treatment description, length of inpatient stay, and overlapping sample characteristics.
The current study differs on the measures used and in its use of a two year post-discharge
follow-up in contrast to Green‘s one year findings. For context it is important to note that
the present study diverges from the U.S. literature which tracks longer term ―residential
treatment‖ (Lyons & McCulloch, 2006), typically in group care for over one year with a
less psychiatrically involved sample. In addition, the ―psychiatric hospitalization‖
literature of typically less than one month inpatient duration, tends to reflect more acute
clients in regard to self-harm crises rather than chronic severe acting out behaviour. A
more accurate descriptor of the present targeted admission model would be ―psychiatric
milieu therapy‖, which distinguishes it from maturational placements in the residential
literature and brief stabilization in hospitals that do not attempt cognitive-behavioural
skill development as part of the treatment.
Duchnowski, Hall, Kutash and Friedman‘s (1998) review of the alternatives to
residential treatment for children with serious emotional disorder notes that the
residential-based treatment literature continues to be challenged in needing to report on
who the children and youth are who receive service, along with an understanding of their
treatment histories, and a more adequate reporting of longer term educational and life
situations. Clearly, residential treatment for chronically impaired individuals needs to be
situated within a broader system of care that encompasses a child and family‘s social
ecology. Hair (2005) suggested in her review of residential outcomes that ecological and
systemic variables have to-date offered more predictive power than individual psychiatric
prognostic indicators, an observation also noted by John Lyons and his associates (2001).
Residential treatment investigators have not routinely measured pre-admission and postdischarge service delivery efforts when tracking symptomatology, a methodological
shortcoming that was addressed in the present study.
Modern statistical methods now allow for an understanding of differential change
trajectories in developmental psychopathology. For example there has been success in
identifying subgroups with differential outcomes in the adolescent delinquency literature,
CPRI-UWO 2008
19
(e.g., Roeder, Lynch, & Nagin, 1999), and in the adult severe mental illness literature (W.
V. Rubin & Panzano, 2002). In tertiary children‘s mental health we are aware of only
three research teams that have attempted to track heterogeneity in symptom trajectories
over time in samples which included comorbid, tertiary symptoms, while mapping
outcomes and attempting to model theoretical predictors (Halliday-Boykins, Henggeler,
Rowland, & DeLucia, 2004; Hodges, Xue, & Wotring, 2004; Rosenblatt & Furlong,
1998). However, these three investigations were limited in two key ways. First, their
trajectory clusters were assigned based on change on a single outcome measure which
typically was either based on parent report of symptoms (two studies) or a clinician‘s
rating of functional impairment; second, outcomes were monitored over the medium term
- six months (two studies) to 16 months. These studies found that subgroup cluster
classification was primarily based on initial symptom or impairment levels with
differential treatment responsiveness. An attempt to replicate and extend these findings
was undertaken presently.
Methodological Challenges for Follow-up Studies in Tertiary Care
Providing robust treatment outcome studies in the area of tertiary care poses at least three
fundamental challenges. The first is in the area of the research design used to isolate the
effects of treatment on outcomes. The second relates to measurement issues. The third is
related to applying the most appropriate statistical analysis on which inferences with
respect to the impact of treatment can be made.
Research Design Challenges. Although inpatient treatment is amongst the largest and
most expensive components of the mental health system for children and youth (Leon et
al., 2000), there has been little research evaluating the effectiveness of long-term
outcomes (Knorth et al., 2008). The methodological obstacles inherent in working with
samples presenting exclusively with highly comorbid symptom presentations (Jensen,
2003) include the following:
1. Evaluating the impact of one part of the treatment process with clients who
receive a complex array of tertiary residential and outpatient treatments
2. Random assignment is not available for ethical and treatment concerns (Curry,
1991, 2004)
3. The challenge of examining longitudinal designs to understand complex
developmental psychopathology predictor models (Keiley, Lofthouse, Bates,
Dodge, & Pettit, 2003)
The existing literature reflects a need for quasi-experimental studies in this area for
purposes of measurement development and theoretical modeling of outcomes (Green et
al., 2007). The present study employed a one-group within-subjects multimodal
longitudinal design to track the trajectory of change over a three-year period with a
CPRI-UWO 2008
20
sample of high-risk, high-needs children and youth within the broader context of the
system of care.
Measurement. The Province of Ontario has adopted two intake measures for all
children‘s mental health agencies. The first is a well known clinician rated assessment of
youth functional status, the Child and Adolescent Functional Assessment Scale (CAFAS;
Hodges, 2000), and the second, the Brief Child and Family Phone Interview (BCFPI;
Cunningham, Pettingill, and Boyle, 2004), is a newly developed parent report of child
symptoms and family functioning. The CAFAS has demonstrated a low to moderate
relationship with the Achenbach Child Behavior Checklist (CBCL) (Rosenblatt &
Rosenblatt, 2002), a measure upon which the BCFPI is based. This distinction between
measures is to be anticipated and welcomed, in that multiple measures, methods, and
perspectives are the hallmark of child clinical practice. Parent, teacher, youth reports,
and clinician ratings, each provide a unique perspective. As such, the present
investigation utilized a multifaceted method of understanding child and family
functioning at multiple points over three years. Rosenblatt and Rosenblatt (2002)
emphasize that while both the CBCL and CAFAS tend to be helpful in detecting clinical
problem status and in an ability to detect change over time, there is a lack of agreement in
describing change at an individual level. For example, they note ―the answer to the
question of what proportion of the youth is showing improvement could range from 20%
(agreement on positive change on the CBCL and CAFAS) to 66% (positive change on
either the CAFAS or the CBCL)‖ (p. 270). They add that an administrator who chooses
to adopt one or the other of these outcome numbers will likely engage in very different
decisions regarding the future of that clinical service. They offer the consensus opinion
that some type of increased specificity of matching or clustering is required to determine
those clients who seem to do well in treatment on some outcomes in response to some
forms of treatment.
Statistical Analysis. The present investigation utilizes the conservative approach to
within-subject outcome analysis. Hence, successful change on both the BCFPI and
CAFAS were analyzed simultaneously to assess outcome agreement across guardian
symptom ratings and clinician functional adaptation/risk ratings. This same theoretically
robust and multi-modal method of outcome monitoring was used in attempting to identify
between-subject variability in change trajectories.
The Present Study. The present investigation tracks clients who present with early onset,
broadly elevated symptomatology as identified by cross-informant accounts based on
diagnostic (categorical) and questionnaire (dimensional) evaluations (Youngstrom,
Findling, & Calabrese, 2003). As such, these clients are at risk of developing a severe and
protracted course of mental health problems with comorbid symptoms emphasizing both
CPRI-UWO 2008
21
internalizing (Sterba, Prinstein, & Cox, 2007) and externalizing concerns (Moffitt et al.,
2002). The question then is whether an alteration in developmental course is possible?
Can we identify determinants of these alterations? In order to address this issue, children
and youth with a history of multiple, tertiary service utilization, including a targeted
inpatient stay, were monitored over three years to examine the developmental course of
symptoms, adaptive functioning, and service usage.
Specifically, the goals of this study sought to address:
1. A deeper understanding of the extent of challenges faced by a tertiary
population of children and youth and their caregivers.
2. The factors that predict the differential trajectory of change over time within a
tertiary sample of children and youth.
3. How communities support these children/youth and their families within a
system of care that included a psychiatric admission.
4. The long-term outcomes and effect sizes at two years following discharge from
an intensive psychiatric milieu program.
METHOD
Description of the Sample
The sample consisted of all consecutive admissions of children and youth aged 6-17
years accepted for inpatient treatment at the Child and Parent Resource Institute (CPRI).
CPRI is a large child/youth mental health centre operated directly by the Ontario Ministry
of Children and Youth Services. Clients diagnosed at referral with a developmental
handicap are directed to other units at CPRI, and were not a part of this study. Otherwise,
there were no diagnostic exclusionary criteria. Archival program evaluation data
available at this regional centre monitored consecutive inpatient referrals (n=360) from
October 1, 2002 to July 1, 2006, ensuring that a consistent battery of standardized intake
rating scales were completed with all children/ youth admitted. From these original
referrals, 230 individuals entered residential treatment during the time frame (M = 12.06
years, SD = 2.46, 171 boys). Five participants were not included in the analysis as they
voluntarily left residential care within the first two weeks, and did not have an
opportunity to benefit from the programming at CPRI. From the final pool of 225
participants, 170 families (75%) completed the long term follow-up telephone interview.
Of those who did not complete the follow-up interviews, 11 (5%) had moved and could
not be located, 29 (13%) were contacted but did not respond, and 16 (7%) declined to
participate. Anecdotally, 3 of the families that refused to participate reported that their
children were doing exceptionally well, and they did not wish to relive their past troubled
histories.
CPRI-UWO 2008
22
Referral Route to CPRI
All children and youth referred for residential treatment at CPRI first proceeded through
their local mental health single point of access agency following consultation with the
community case manager and family or guardian. There are ten single point access
centers existing in the 17 counties served by CPRI, extending from Windsor to Niagara to
Owen Sound (see map in Figure Appendix B). This multiple gating, single-point
integrated community intake process utilizes standardized intake tools and a ―least
intrusive intervention‖ mode of practice, ensuring adequate community treatment efforts
had preceded the inpatient referral. This referral process overcomes at least two concerns
noted in the literature, including: a) the tendency to use residential treatment as a
―dumping ground‖ due to lack of service (Barth, 2005) and b) choosing inappropriate
clients for residential service, such as those who would be better served by less intensive
services (Lyons et al., 2001). This referral process attempts to ensure that only those
children and youth with extreme levels of need and high risk of permanent school and
home break down are accepted for inpatient treatment.
Children/youth who were admitted received assessment, treatment and care plans
developed collaboratively with the community team prior to admission. The
children/youth and their families/guardians were active participants in the care planning
of their child. Linkages to community-based care and social support structures for the
children/youth were initiated and reinforced in keeping with the philosophy of a
continuum of care from the specialized stay to the community to make the transition as
seamless as possible. CPRI has outpatient and in-home support services that can be
accessed by clients prior to or following an admission. Collaboration with community
agencies and schools was fostered to enhance functioning within this continuum of care.
Mental Health Treatment at CPRI
While CPRI is a regional provider of highly specialized treatment services to a large 17county catchment area including urban and rural populations, it is also a research based
organization, committed to developing effective and efficient treatment services. Subjects
were referred to five cottage-like psychiatric inpatient units: three child units and two
adolescent units. All programs are licensed directly by the Ontario Ministry of Children
and Youth Services. Units are generic rather than disorder-specific services with each
unit accepting a range of disorders. Units differ slightly in admission policy and typical
length of stay but the treatment models are convergent on current best practice, drawing
on structured behavioural milieu and individualized intervention strategies. Treatment
efforts, guided by unit psychiatrists, psychologists and social workers, reflect
programming which emphasizes multimodal clinical assessment, adaptive skill
development, parent training and family counselling and coordinated discharge planning.
CPRI-UWO 2008
23
The living milieu is structured to promote interpersonal skill development, with
concomitant psychotropic medication and psychosocial, family-oriented and educational
interventions. An onsite school offers full time attendance in a personalized, special
education environment which is essential for students who have often been suspended
from school prior to admission.
All participants had an individualized plan of care, formally reviewed monthly by
the family/guardian, community case coordinator, and CPRI clinicians. Discharge dates
were flexible, based on the child/youth‘s progress and dictated by the needs of each
client. The average length of stay for the child/youth in the present study was 4 months,
and CPRI outpatient services were often provided during the immediate pre-admission
and post-discharge phases. Post-discharge follow-up could include outreach assistance in
the home or classroom as well as ongoing therapeutic contact and medication monitoring.
Active involvement and support of the parent/guardian was considered essential. A
majority of children and youth in residence at CPRI returned home every weekend, thus a
quarter of the child/youth‘s stay while at CPRI was spent at home, with child and family
community/home goals in place. Anecdotally, many guardians noted the environment
was an ideal one for safety, mental health stabilization, and promotion of interpersonal
skill development, an important consideration in discharge planning when transitioning
back to a less structured environment.
Clinicians at CPRI. CPRI is a multidisciplinary mental health centre, with many
qualified professionals participating in the daily care of children and youth. Several
disciplines interact daily with the children and youth in residence. Educational
requirements for all direct care counsellors on the units at CPRI included, at a minimum,
a college diploma in a child and youth worker program or equivalent specialization which
generally consisted of 2 years post secondary school in an accredited program. All
therapists received frequently updated crisis management training and other areas specific
to the program such as anger management and social skills. Direct doctoral level
supervision of front line staff was coupled with access to ongoing professional education
regarding evidence based treatments for children and youth with various mental health
needs. A staff psychologist directed behavioural programming within the residences, and
staff psychiatrists prescribed all psychotropic medications with daily monitoring
protocols. Social workers were available for family therapy. Staffing levels were
mandated, and allowed for careful supervision of clients and close monitoring of their
safety.
Description of Measures and Timelines
This follow-up study was partially based on an existing program evaluation database that
collected various parent/guardian, teacher, and clinician measures at four time periods:
pre-admission, admission, discharge, and 6-month post-discharge. The present research
CPRI-UWO 2008
24
attempted to expand this multimodal battery at 2-years post-discharge. The mean time
frame for the longitudinal BCFPI follow-up assessment was 154.85 weeks (SD = 25.0).
This equals on average three years between the first and last structured interviews with
the 170 parents and guardians. The mean time period between the first and last CAFAS
clinician rating was slightly less (M = 141.9, SD = 24.7). The long term follow-up
interview was on average completed just over 2 years after the target admission ended (M
= 122.3 weeks, SD = 22.0). A graphical depiction outlining the different measurements
tools and different measurement occasions available from the broader CPRI dataset is
found in Figure Appendix C.
Measures
Multiple measures providing complimentary information along with different sources of
data, for example parent and teacher perspectives, were used to evaluate child, youth and
family functioning across the multiple time points. The following section provides a more
detailed summary of the rationale and support for the inclusion of each measure.
The Brief Child and Family Phone Interview (BCFPI; Cunningham, et al., 2004),
is a structured phone interview conducted with the caregiver based on Ontario norms and
is similar to the most commonly utilized instrument in children‘s mental health, namely
The Achenbach Child Behaviour Checklist (CBCL). Considerable promise with this new
measure has been reported in both its reliability and validity in a clinical population
(Boyle et al., in press; C.E. Cunningham, Boyle, Hong, Pettingill, & Bohaychuk, in press)
as well as from a validation study based on the present sample of children and youth 1.
Standardized scale (T) scores provide normative data on subscale factors describing
several externalizing, internalizing, family and individual functioning factors. Like the
CBCL, the BCFPI provides a standardized screening for clinical triaging and profiling
based on parent report. Internal consistency scores in this study indicated adequate
reliability (see Appendix A); especially given the brief screening consists of a few items
per factor. The content validity of the measure is based on the mapping of items to the
DSM criteria.
Potential advantages of the BCFPI are noteworthy. The phone interview increases
its utility for outcome investigations, given the often reported low rates of respondent
replies when questionnaires are mailed. Our BCFPI rates were more than double the
mail-in questionnaire responses. The considerable promise of this new measure requires
the demonstration of criterion validity, in both concurrent and predictive validity
investigations. The present research represents a beginning in addressing this issue, as
data collected based on the parent and teacher Conners‘ data, a measure with established
psychometric properties and sensitivity in screening a clinical population and monitoring
1
A manuscript is currently in preparation which will report on the psychometric properties of the BCFPI
based on the current sample of children and youth.
CPRI-UWO 2008
25
treatment effects, is also included. To our knowledge this study also represents the first
published test of the BCFPI as a measure of outcome monitoring with a clinical
population.
Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 2000). The
CAFAS is a multidimensional rating of level of functioning, whereby clinicians‘ rate a
child/youth‘s impaired or restricted role functioning within home, school, and family
domains. The CAFAS has received considerable research attention, given formal
adoption across children‘s mental health systems in thirty American states. It has
demonstrated predictive utility (Hodges & Wotring, 2004) of outcomes and service usage
in populations similar to that in the present investigation. It consists of subscales
measuring functional impairment in eight domains: school, home, community (respect for
rights of others), behaviour toward others, moods (emotional regulation), self-harm,
substance abuse, and thinking (rational thought). Each domain is rated in ten-point
increments on a scale from 0 (no impairment), 10 (mild), 20 (moderate) to 30 (severe
impairment). The total score is an eight-subscale total, with a range of 0-240.
Interviewers in the present study were trained using the CAFAS manual to reach a
consistent level of interrater reliability (ICCS for individual scales of .80). The CAFAS
is completed by the primary clinician working with the family every three months at
CPRI. For this long term follow-up investigation, our interviewers asked a series of
questions regarding seminal life events (e.g., ―has your child been suspended from
school?‖) beyond those found in the BCFPI, allowing the CAFAS rating of functioning to
be completed.
The Conners’ Rating Scales – Revised (Conners, 1997). The Conners‘ Scales have
been widely used in clinical and research endeavours in children‘s mental health over the
past thirty years (Gianarris, Golden, & Greene, 2001). It was chosen as part of the core
clinical battery at CPRI to document home and school symptomatology and functioning.
Both parent and teacher reports of child behaviour were collected using the Connors.
A Canadian and U.S. based norming sample based on the Connors allows parent
and teacher ratings to reliably depict their view of a child‘s behaviour across settings.
The latest version of the Conners includes the short 10 item subscale, now called the
Conners‘ Global Index or CGI, that has been most heavily researched. The brief 10-item
Conners‘ Global Index scales were rated biweekly by the parent, teacher, and frontline
staff working closely with the child, as a monitoring tool during the residential stay.
Conners‘ CGI scores have been found to distinguish clinical versus non-clinical groups
and are sensitive to medication and psychosocial treatment effects. However, the ability
of the various subscales to predict specific disorders has yet to be established (Gianarris
et al., 2001).
CPRI-UWO 2008
26
The Social Skills Rating System (SSRQ; Gresham and Elliott, 1990) is a widely
used normative rating scale with supportive validity documenting both parent and teacher
views of the child‘s social skill repertoire. Subscales describing social assertion,
responsibility, empathy, and self-control, along with internalizing and externalizing
behaviour symptom scales are totalled for the teacher and parent report. Importantly, it
also allows the teacher to rate functioning and academic competence which are
theoretical constructs of interest.
The Parenting Stress Index (PSI: Abidin, 1995) and the Stress Index for Parents
of Adolescents (SIPA: Sheras, Abidin, & Konold, 1998) is often utilized by researchers
and clinicians interested in documenting family stress and potential problems in the
parent-child relationship. The PSI Short Form Total Stress score provides an indication of
the overall level of stress reported by the parent and reflects personal parental distress,
stresses derived from the parent‘s interactions with the child, and stresses that result from
the child‘s behavioural characteristics. Barnes & Oehler-Stinnett (1998) note that the
factor structure of the short form is better developed than that of the long form, with good
reliability, and supports the use of this measure as a means of assessing family
maladjustment.
Chart reviews were utilized to gather data on the following: Pre-admission use of
clinical services, coordination of services, formal estimates of intellectual and academic
performance, charted psychiatric diagnosis, and psychotropic medications administered
during the inpatient stay.
Client satisfaction with services, outcomes and relative well-being was assessed
through a questionnaire developed by clinical psychologists at CPRI.
Qualitative data. Qualitative data to assist us in placing the quantitative data in a
theoretical context (i.e., clinical significance rather than statistical significance), was
collected through intensive interviews with seven families by a UWO graduate student.
Child and youth self-report measures were also collected, but given the large
attrition, this data will not be reported presently. In line with the literature (Handwerk,
Larzelere, Soper, & Friman, 1999), youth who did respond, self-reported far fewer
problems, with correlations in the often reported 0.3-0.4 range with parent report.
Data Analysis
Attrition in the sample was modest given the retrospective nature of the longitudinal
design. There were 225 families considered for the study, with 170 agreeing to
participate. Of those who agreed to participate in the follow-up phone interviews, we had
a total data completion rate of over 90% at three time points which included pre, post and
follow-up for both the BCFPI and CAFAS, with the missing values being predominately
located at the 6 month post assessment period. For analyses of outcome trajectories on
CPRI-UWO 2008
27
the BCFPI and CAFAS, these mid-point missing values were imputed in the Latent Gold
software imputation module, which utilizes a log-likelihood approach for missing data
(Vermunt & Magidson, 2005, p. 44-46). Latent class modeling is a useful application in
follow-up studies with intermittent missing data (Roy, 2007). The Latent Gold program
allows for simultaneous covariates of varying data types including continuous and ordinal
data to be entered. Lower response rates existed for the mail-in questionnaires from
parents and teachers, so these outcomes are reported separately with the sample size (N)
noted and no missing data estimation undertaken.
While the outcome analyses will focus on the 170 subjects who agreed to
participate at follow-up, an examination of the full 225 consecutive clients who
completed CPRI inpatient treatment was also undertaken. In a logistic regression
analysis, we could find no statistically significant differences in our sample of 170 to
those 55 individuals who did not participate in the long term follow-up on variables such
as referral severity as measured by the BCFPI and CAFAS, sex, age at admission, length
of admission, and child welfare status.
Descriptive statistics were utilized to carefully depict mental health, adaptation
and living conditions in the present sample, essential if outcome effect sizes or
multivariate predictor models are to be understood in relation to other investigations of
developmental psychopathology and treatment responsiveness. Disabling symptoms,
social skills, functional adaptation within the home, school and community, levels of
functioning, and treatment service access were all formally coded from the standardized
questionnaires or through chart reviews, and entered into an SPSS database.
Repeated measures analyses of variance were examined for overall trends, with
no missing data estimations. This indicated significant mean within-subject improvement
on all major reports of symptoms and functioning at both the post-discharge and followup time frames, allowing for post-hoc investigations. For some variables the change was
linear, for some it was best depicted by a non-linear, quadratic change model. In
particular, there was some rise reflecting functional deterioration in mean CAFAS scores
between the post and follow-up assessments. Scatter-plot analyses indicated the betweensubject variability in change over time was also substantial, so a means of analyzing
differential trends was required. Given the absence of literature on differential trajectories
of change in developmental psychopathology in this comorbid population, and given
typical predictors of ―low risk‖ had been largely eliminated in these subjects at intake, it
was difficult to envision forced, non-arbitrary theoretical grouping variables that would
account for the observed variance between subjects. Attempts to group subjects by
treatment program (as the five inpatient units differed by age, gender, treatment team), or
county of subject origin (i.e., different pre and post supports), did not reveal significant
differences on overall mean outcomes on the BCFPI and CAFAS. The good news, that no
one inpatient unit or community follow-up program demonstrated overall negative
CPRI-UWO 2008
28
outcomes, (cf. Lyons et al., 2001) was shared with the appropriate audiences, an
important moral boost to tertiary care workers.
It was determined that an exploratory analysis of between-subject variability
would be undertaken. The multivariate repeated measures were entered into the Latent
Gold software package to perform a latent class cluster analysis to generate groups of
people with similar change patterns over time. This new finite mixture modelling
technique, also known as model-based cluster analysis, has several advantages over other
clustering methods in that it ―discovers‖ naturally occurring subpopulations, assigns
cluster membership using the highest posterior probability score for that case, and also
offers a statistical fit measure (Bayesian Information Criterion or BIC) that compares
different models and offers the most parsimonious solution to dividing up the data into
subgroups (Mun, von Eye, Bates, & Vaschillo, 2008).
Latent class modeling is useful in finding groups or subpopulations of subjects
who have similar tendencies or response patterns. For example Roeder et al. (1999), used
this method to determine that life course persistent offenders tended to hold neurological
deficits reflected in scores on a measure of executive functioning and be reared by
caregivers with poor parenting skills. Presently, the BCFPI and CAFAS were jointly used
as the dependent measures to predict latent class membership, allowing for a robust
theoretical model of outcomes. Similar to Halliday-Boykins et.al. (2004), these differing
classes or clusters of subjects were then analyzed in order to describe children and youth
who make gains over time versus those who experience ongoing serious difficulties.
Given that latent class models identify similar change patterns but not rigid class
membership, the use of this classify-then-analyze paradigm is restricted by the
probabilistic uncertainty of the cluster membership (Roeder et al., 1999). As such, the
present analysis should be considered preliminary, and in need of replication in an
alternate sample of tertiary, comorbid subjects.
RESULTS
With the primary goal of this study to address the developmental outcomes of tertiary
clients in the system of care over the course of three years, it is essential to develop an
appreciation of the characteristics of these children and youth. Many investigations of
inpatient and residential treatment offer inadequate sample characteristics. Hence, the
Results section is organized in two sections. The first section provides a summary of the
characteristics of the sample of children and youth who participated in psychiatric
inpatient treatment. The second section summarizes the relevant developmental outcomes
in regard to psychopathology and adjustment.
CPRI-UWO 2008
29
Appreciating sample characteristics in psychiatric milieu therapy
Unique to this study was the opportunity through both standardized client screening and
file-based review to characterize the sample in numerous ways. This allows the reader to
appreciate the magnitude of the problems facing young children and children‘s mental
health care planners, while enabling researchers to match and compare these findings to
other empirical work.
Services received. A chart review documented the services already received prior to the
target admission to the psychiatric milieu. As a tertiary centre, CPRI has a standard intake
practice of asking for consent and then requesting directly by mail all previous reports
from mental health, school, child welfare and corrections agencies. At follow-up parents
or guardians were asked specific questions about service involvement in the time since
inpatient discharge, approximately two years earlier. As some reports may not have been
received by CPRI initially, and as parents may have failed to recall all instances of
support they and their child had received post-discharge, the following summary of the
history of service delivery should be considered a minimum reflection of service levels
across all time periods. Table 1 summarizes mental health, child welfare and law
enforcement service delivery contacts recorded from the chart reviews prior to admission
and based on parent report at follow-up.
Table 1
Percentage of Children Receiving Mental Health Services Other than the Target
Admission
Factor
Ever receive services?
Ever out of home?
If so, inpatient/residential tx?
If so, group/foster care?
If so, youth justice facility?
Psychotropic medication?
Police contact?
Pre-Treatment
% Yes
100
71
48
45
11
94
44
Two-Years Post-Treatment
% Yes
88
57
19
34
12
79
52
Note. The values represent the percentage of residents with available data (valid percentages). Out of home
percentages do not add to 100 as majority of children experienced more than one out of home placement.
a
10% of clients were reportedly refusing their prescribed medications at follow-up (adolescents primarily).
Examination of Table 1 reflects that overall, the residents at CPRI are significant
consumers of service prior to the target admission. One hundred percent of children had
already received some form of child or family intervention, either mental health or child
CPRI-UWO 2008
30
welfare, prior to the psychiatric inpatient stay; for almost half of the group, this was a
second mental health inpatient stay, and for 45% a child welfare intervention had already
occurred. One parent indicated:
―Before we actually went into the residential program we were involved with
counselling, we were involved with the school board, with psychologists, social
workers. At one point we even had the police involved because of the issues that
he had been through. And we had private counselling for him where we had him
tested. We did some IQ testing... central auditory processing, we went to a doctor
who specialized in diet... we went to have his hearing tested‖ (family interviews Oosterveen, 2008).
We can also report that 18% of the sample was already in the care of the local
children‘s aid society (CAS) at the time of referral to inpatient treatment; 32% of children
were supervised by the CAS without being made wards of the state. Maltreatment was a
significant factor as reflected by parents or guardians on the intake BCFPI: 56% had
witnessed verbal or physical abuse 30% of the referents had been physically abused, 25%
neglected and 19% sexually abused,
To summarize, pre-existing mental health treatments were the norm for this
sample prior to the present inpatient referral, with 71% already having received out-ofhome care for purposes of safety or treatment. A substantial number of these children
have been traumatized. The reported levels of family violence and high levels of CAS
care indicate the severity of safety needs. The follow-up use of services reported in Table
1 will be discussed in the treatment outcomes section below.
Amongst the distinguishing characteristics of children/youth who enter residential
psychiatric treatment is the number and nature of medications that have been prescribed
prior to their admission. Table 2 and Figure 1 summarize their experiences with
pharmacotherapy. Highlights from this data indicate that 94% of children were under
medical care receiving psychotropic medication prior to the target admission, with many
children receiving multiple medications. On average, 57% of the children were
prescribed more than 1 medication at admission, 48% at discharge, and 48% at follow-up.
Indeed one of the frequent goals identified by parents upon admission to residential
treatment is a review of the medications by a child psychiatrist to ensure the nature and
dosages are appropriate and necessary to the identified need and diagnosis.
CPRI-UWO 2008
31
Table 2
Psychotropic Medications Prescribed at Admission to CPRI, in Descending Order of
Sample Usage within Each Drug Class (for class percentages see Figure 2)
Treatment of:
Medication
Classification
ADHD
Methylphenidate (Concerta)
Methylphenidate (Ritalin)
Dextroamphetamine (Dexedrine)
d-and l-amphetamine (Adderall)
Psycho-Stimulants
ADHD
Atomoxepine (Strattera)
Other (Norepinephrine)
Aggression, agitation
Risperidone (Risperidal)
Quetiapin (Seroquel)
Olanzapine (Zyprexa)
Haloperido (Haldol)
Loxapine (Loxitane)
Chlorapromazine (Thorazine)
Pimozide (Orap)
Anti-psychotics
Citalopram (Celexa)
Sertaline (Zoloft)
Fluoxetine (Prozac)
Fluvoxamine (Luvox)
Paroxetine (Paxil)
Clomipramine (Anafranil)
Anti-anxiety (antiobsessionals)
Anxiety and depression
Venlafaxine (Effexor)
Bupropion (Wellbutrin)
Trazodone (Desyrel)
Miratazapine (Remeron)
Imipramine (Tofranil)
Amitriptyline (Elavil)
Despiramine (Norpramin)
Anti-depressants
Clonidine (Catapress)
Buspirone (Buspar)
Other
Bipolar mood disorder
Lithium Carbonate (Eskalith,
Lithonate)
Lithium
Bipolar mood disorder
Valproic Acid (Divalproex,
Epival, Epakote)
Carbamazepine (Tegretol)
Excarbazepine (Trileptal)
Topiramate (Topamax)
Lamotrigine (Lamictal)
Gabapentin (Neurontin)
CPRI-UWO 2008
Mood Stabilizers (anticonvulsants)
32
Psychotropic medications
Other
Follow-Up
Medication
Lithium
Discharge
Anti-convulsant mood
stabilizer
Admission
Anti-depressants
Anti-anxiety
Stimulants
Anti-psychotics
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Percentage
Figure 1. Percentages of children on psychotropic medications at three time points.*
* Based on chart review for admission and discharge, parent report for follow-up two years later. Percentages do
not add to 100 as majority of children on more than one medication. ―Other‖ is primarily Clonidine at admission;
Strattera added 10% to this category at follow-up. Medications received by less than 1% of sample not included.
Figure 1 depicts the drug classes used by community and CPRI psychiatrists to
treat this population over time. Both upon admission to CPRI and at the follow-up period,
Risperidone was the single most commonly prescribed medication At the study‘s end,
anti-psychotics (such as Risperidone) were used in low doses with children and youth to
control agitation and aggression, and were prescribed to 74% of the participants.
Stimulant medication for ADHD was the second most commonly prescribed drug class.
By study‘s end Concerta was the drug of choice in this class to help manage ADHD, used
by 34% of the participants.
Note that the medications at admission and discharge were substantiated by CPRI
clinical records of medications administered, while follow-up medication rates were
based solely on parent/guardian reports. Over the course of the inpatient stay, there was a
trend towards lowered use of anti-psychotics, anti-convulsants, mood stabilizers and
CPRI-UWO 2008
33
stimulants. Two years following discharge the use of low doses of anti-psychotics to
control agitation and aggression had clearly increased.
Summary of Symptoms from the BCFPI admission scores. Elevations on scores across all
indices for CPRI admissions are high; their extreme nature is more fully appreciated
when situated relative to BCFPI scores from the population of children and youth who
seek mental health treatment on an outpatient basis around Ontario. Figure 2 reflects the
elevations across all areas, with symptom and functioning scores above the 99th percentile
being common in this sample. Clearly, the typical child mental health clinic referral T
scores of 70 are eclipsed in the present study by a standard deviation or more on the key
subscales of externalizing behaviour problems, nine items tapping mood and self-harm
concerns, total mental health, and global child and family functioning.
Inpatient and Outpatient BCFPI Broad Band T Scores
105
100
T Scores
95
90
CPRI Inpt
85
Ontario Outpt
80
75
70
65
60
External
Internal 6 Mood + Total MH
3 S-H
Global
Child
Gobal
Family
Figure 2 CPRI inpatient BCFPI intake T scores (N = 225) relative to Ontario outpatient
clinic referrals (N = 22,695). Higher scores represent pathology with 65-70 typical
of clinical cut off in the literature (normal T score mean is 50, standard deviation
is 10).
School functioning pre-admission. Prior to admission, schools were asked to have the
teacher that best knew the pupil rate their behaviour and academics. Of the teachers who
responded to this survey, only 57% indicated they taught in the regular classroom, 18%
reported they knew the pupil from a self-contained or segregated classroom placement,
and 11% were special educators (resource teachers). Community teacher academic
competency ratings (N=150, SSRQ scale) of students at pre-admission indicated that
CPRI-UWO 2008
34
students fell more than one standard deviation below the mean of classmates‘ academic
reading, writing and math skills (standard score M = 82, SD = 14). All children of
elementary school-age and some of the high school aged youth received an academic
assessment while at CPRI. Based on the Academic Skills subscale of the Woodcock
Johnson Tests of Achievement, the mean standard score was 90.8 (N = 131). Grade
equivalency scores were, on average, more than one full grade behind (M = -1.56), and
37% of the pre-adolescent sample scored more than one standard deviation below the
mean population score on academic skills. A chart review found full scale Wechsler IQ
scores reported for 145 subjects; it is anticipated that these would all be children with
some level of difficulty at school that prompted a referral to a psychologist. The average
full scale IQ score was 88.6 (SD =15.4). Equally compelling, 47% of those assessed
scored one standard deviation below the mean of 100 (IQ 85 or lower).
In addition, community classroom teachers were asked at intake to rate the
child/youth on the Conners‘ Teacher Rating Scale. These scores indicated that this
referral sample had high levels (more than 3 standard deviations beyond the mean) of
acting out behaviour difficulties in the classroom. For example the Conners‘ ratings (N =
153) on teacher‘s estimate of Oppositional Behaviour, were extreme with a mean T score
= 87 (SD = 18).
Table 3 depicts CAFAS severity ratings at the beginning and end of the study
specifically for the school subscale. Clinician ratings pre-admission indicated 92% of
subjects scored 20 or more on the intake CAFAS school subscale, indicating the child
had received support from school personnel due to behaviour, failing grades, or absence;
72% were suspended from school or their needs were not met even in a special education
environment reflected in a CAFAS score of at least 30.
With regard to adaptive behaviours, on the Social Skills Rating Scale, the
preadmission classroom teachers (N = 155) reported a significant deficit in socially
adaptive behaviours (standard score M = 72), almost two standard deviations below the
mean of 100. This score is typically considered to be in the context of the handicapped
range of skill deficit.
Table 3
CAFAS Rating of School Functioning
Impairment
Pre-Treatment
%
Minimal/no
1
Mild
7
Moderate
20
Severe
72
Two-Years Post-Treatment
%
19
5
23
54
Note. The values represent the percentage of residents with available data (valid percentages).
CPRI-UWO 2008
35
To summarize, it is the norm for students in this sample to be receiving extensive
attention from the school office or their local school board, due to a combination of
behavioural and academic difficulties and social skill deficits. Both teacher rating scales
and formal academic testing place students‘ more than one grade below their same aged
peers in achievement, with severe acting out behaviour difficulties observed by the
community teacher. Anecdotally, a number of children/youth had been removed from
school and were asked not to return until the CPRI period of inpatient assessment and
treatment was complete.
Mental Health needs at pre-admission. In contrast to the cited literature with respect to
children who experience social-emotional-behavioural disorders, comorbidity of
symptoms in the present sample was extreme. Figure 2 depicted the very high
externalizing and mood plus self-harm symptoms reported by parents on the BCFPI
dimensions. Concurrent symptoms at admission were reported by psychiatrists as
follows:
o Two psychiatric diagnoses - 86%
o Three psychiatric diagnoses - 68%
o Four psychiatric diagnoses – 38%
o Five psychiatric diagnoses – 18%
o Six psychiatric diagnoses – 10%
o Seven psychiatric diagnoses – 4%
These diagnoses reflected that 85% of children had an admission diagnosis of
disruptive behaviour disorders (e.g., Attention Deficit Hyperactivity Disorder (ADHD),
Oppositional Defiant Disorder (ODD), and Conduct Disorder). While behavioural
problems were the primary reason for referral, Figure 3 indicates that Anxiety
(Generalized Anxiety (GAD), Obsessive Compulsive Disorder (OCD)), trauma (e.g.,
Post-traumatic Stress Disorder (PTSD), Reactive Attachment Disorder of Infancy), mood
dysregulation and other problems may be seen as driving some of the behavioural
difficulties. Figure 3 lists the diagnoses by major category, as determined at discharge
from inpatient treatment. The discharge diagnoses are depicted as they are based on
intensive multidisciplinary assessments over four months, and ideally represent the most
accurate symptom profile of this sample. The three inpatient psychiatrists at CPRI
considered 86% of these diagnoses firm, and 14% provisional. Formal learning (e.g.,
borderline IQ, reading disorder) and language disorders were only cited a total of 61times
in the discharge psychiatric reports, a surprising result given the teacher reports noted
above, and the IQ and academic test results found on file.
CPRI-UWO 2008
36
Diagnostic Count at Discharge
PDD
8
Relational
30
Organic
9
Psychosis
4
Substance
4
Bipolar
16
Depression
26
OCD
12
Anxiety
10
GAD
57
Attachment
22
PTSD
25
Tourette
20
LD
61
Conduct
37
ODD
113
ADHD
131
0
Figure 3
50
100
150
Count of all charted psychiatric diagnoses at discharge
To summarize, this sample is significantly involved in child psychiatry with
comorbid externalizing and internalizing psychiatric symptoms substantiated by
caregiver‘s dimensional ratings, psychiatric evaluation, and clinician‘s categorical
reports.
Family coping pre-admission. The Parenting Stress Index (PSI) and BCFPI Global
Family Functioning items completed at referral indicate a severe level of distress by
caregivers. Subscale analysis indicated this was reflected in a lack of extended family
supports due to the burden anticipated on others, and extremely high levels of within
family discord, especially in the parent-child relationship. The PSI total scores at preadmission, which indicate how difficult the child is to parent along with an estimate of
subjective parental distress, reflected a mean of 118 (SD = 22; N =185) which falls above
the 99th percentile for parenting stress. When asked how often they felt depressed, more
CPRI-UWO 2008
37
than a third of the parents (35%) completing the BCFPI at intake indicated that they
experienced some form of depression 3 or more days per week.
Overall, families report distress in the parent-child relationship, fear of
unpredictable behaviour, violence from their child, and an overall concern with their own
capacity to cope with the challenges involved in supporting their child in the home,
school and community. The burden of receiving calls from the school to remove the child
or youth was high according to the parental narrative reports. For example, parents or
caregivers questioned their own competence, reported their child to be out of control and
impossible to live with.
―He would come home and as soon as he would open that door all hell would
break loose. The door would come open with his feet and he would just lose it
because he‘s held it in all day and he‘s ready to explode‖.
―His behaviour impacted the family so enormously...when you get to the point
where the younger children are scared of the behaviour of the older one you have
to realize something has to give‖.
―Nobody would visit, nobody would come over. I couldn‘t go to anybody‘s
house with him. Like I was totally isolated at home with my child and I couldn‘t
get rid of him to a babysitter to get groceries. He was always with me‖ (family
interviews - Oosterveen, 2008).
Summary of needs at intake. Overall, mean scores on standardized parent, teacher and
clinician reports at intake for inpatient psychiatric services place the present sample of
children and youth at very high risk of community adjustment failures, subject to
violence, experiencing school and home breakdown, accompanied by high levels of
service provision across agencies, with significant financial (medicine, clinic visits, time
off work during school suspensions) and emotional (depression, family discord) costs to
the parents. Extreme externalizing and significant internalizing psychiatric
symptomatology and social-behaviour difficulties are confirmed by parents, teachers, and
clinical report, and accompanied by indicators of significant cognitive and learning
deficits substantiated by teacher observations and actual test results.
There is substantial evidence that multiple gating single point of access screening
facilitated within various service agencies across multiple counties did reserve these
intensive inpatient beds for those clients with much greater needs and high risk of
ongoing difficulties (Figure 2).
Treatment Outcomes
This investigation utilized a multi-method, multimodal approach to monitoring outcomes
over time. As noted above, attrition was most striking for mailed questionnaires, thus
eliminating the youth self-report analysis and limiting teacher report data. Outcome data
will be reported with the sample size noted. From the 170 post-discharge and follow-up
phone interviews, on the BCFPI we obtained standardized symptom ratings of the
child/youth, social-adaptive behaviour ratings, and parent ratings of self and family
CPRI-UWO 2008
38
functioning. The interviewer rated functional risk on the CAFAS, and various seminal
outcome events were queried.
In examining the outcomes below, please note the high ongoing utilization of
services. Recall that the CPRI inpatient treatment program was situated within a system
of care model across southern Ontario. A brief test of this continuity of care philosophy
was undertaken. CPRI chart reviews indicated that all discharge treatment plan
recommendation reports were sent to guardians; 88% to a community mental health
provider, with 85% of all discharge plans containing recommendations for mental health
follow-up; 41% of plans were mailed to an involved child welfare agency; while 72%
mentioned specific recommendations for schools and educators.
Placement. The living arrangements specified in discharge plan of care reports found on
the clinical charts for all 225 admitted children and youth can be found in Table 4. Two
thirds of children were to live with family, and 28% in a group or foster care home under
the guidance of the CAS. Identified guardianship at the time of discharge can be found in
Table 5. It is noteworthy that 21% were discharged into the direct care of the Province of
Ontario as a ward of the CAS, with a further 22% of families receiving support and/or
monitoring from a child welfare agency.
Table 4 Discharge Living Placement
Placement
Percentage
Family
Group home
Foster care
Residential care
Youth Justice
68.9%
20.7%
7.7%
2.3%
0.5%
Table 5 Discharge Guardian
Custody/care
Percentage
Family
Parent with CAS supervision
CAS temporary ward
CAS ward
57.4%
21.8%
5.3%
15.6%
CPRI-UWO 2008
39
Child and Youth Mental Health Symptoms. Figure 4 details the results from the repeated
parent and guardian reports on three BCFPI broad band symptom scales that occurred
over a span of more than 3 years (please refer to Appendix A for the complete BCFPI
longitudinal dataset). The average pre-admission scores are extreme: 3 standard
deviations above the mean for externalizing symptoms and for mood plus self-harm
symptoms. As anticipated, anxiety scores were clinically elevated but not as extreme.
Although some authors have reported concern that anxiety symptoms in youth may
exacerbate with inpatient treatment (e.g., Wilmshurst, 2002), there was no evidence of
this occurring presently, as internalizing symptoms continued to improve over the three
years (Figure 4). This may be due to the emphasis on family involvement (weekends at
home, family goal setting) in the CPRI inpatient program.
Changes at preadmission through to 6 months post-discharge summarized in
Figure 4 indicate noteworthy gains following inpatient psychiatric milieu treatment. The
effect size for reduced report of symptoms of psychopathology can be found in Table 6.
Calculation of effect size in one sample pre-post investigations is known to be at some
risk of inflation. Carlson and Schmidt (1999) in their large meta-analysis of the training
literature, demonstrated an overall effect size of .908 in single group pre-post designs, in
contrast with an overall effect size of .791 in pre-post control group designs. As noted
above, differential treatment effects lead to an increase in the standard deviation after
treatment, so the basic effect size calculation2 is greatly impacted by the choice of the
standard deviation (SD) term. Although Carlson and Schmidt contend the use of pretreatment SD scores is acceptable, for data summarized in Table 6 we utilized the pooled
standard deviation, thereby reporting a more conservative estimate of effect size. The
large and statistically significant effect size reported for externalizing symptoms, a T
score drop of 10 points, was anticipated as behaviour problems are the primary target of
many of the treatment plans reviewed. Follow-up parental ratings indicate that on
average, children and youth continue to have significantly fewer behaviour problems
three years following their referral to treatment (ES = .78). The effect size for serious
mood instability concerns was moderate and was maintained at follow-up, with scores
below 70 on average after three years (Table 6).
In analyzing the clinical meaning of these T score changes, note that the post and
follow-up scores resemble the outpatient referral levels that can be seen in Figure 2. This
data offers parental rating support for the conceptualization of inpatient treatment as a
means to reduce crisis level symptomatology (i.e., home and school placement
threatened) to a level that equals that seen in a sample of outpatient children and youth.
2
Effect size calculations reflects the change in mean treatment scores divided by the pooled standard
deviation.
CPRI-UWO 2008
40
BCFPI Broad Band Scales Over Time
90
85
T scores
80
Externalizing
Mood + Self-Harm
Internalizing
75
70
65
60
Pre
Post
Follow-Up
Figure 4
Average reduction in mental health symptoms over 3 years as
reported on the BCFPI. Scores above 70 are typically considered to be in the
clinical range of psychopathology. (N = 123)
CPRI-UWO 2008
41
Table 6
BCFPI Repeated Measures T scores over a three year period in the system of care with
Effect Sizes
Subscale
N
BCFPI Externalizing 124
BCFPI Internalizing 123
BCFPI Mood + S-H 123
BCFPI Total Mental 123
Health
Intake
Post
Follow-up
Effect Sizes
Mean
(S.D.)
Mean
(S.D.)
Mean
(S.D.)
d1
d2
82.8
(9.4)
71.0
(14.1)
78.9
(19.2)
80.6
(9.4)
72.9
(12.0)
67.2
(14.8)
66.5
(19.4)
72.8
(12.6)
74.3
(12.3)
65.9
(13.7)
68.7
(18.7)
73.0
(12.0)
.92
.78
.26
.37
.64
.54
.70
.71
1
Calculated for difference between admission and 6-month follow-up BCFPI using Cohen‘s pooled
standard deviation formula. Note: calculations were computed using 1 decimal point.
2
Calculated for difference between admission and two-year follow-up BCFPI using Cohen‘s pooled
standard deviation formula. Note: calculations were computed using 1 decimal point.
It is worth noting that we do have evidence that the tertiary inpatient
multidisciplinary program at CPRI did have a unique impact on symptom reduction
within the broader system of care. Figure 5 depicts repeated BCFPI scores for clients
who waited more than six months to enter residence. When this occurred, which reflected
18% of the cases, the BCFPI baseline score was repeated pre-admission, offering a
unique opportunity to examine the impact of treatment more closely. Clearly, on average
across this subset of clients, the two pre-admission scores demonstrate little change, and
this holds true with a slight deterioration during the post to follow-up period. In contrast
there is a substantial reduction in behaviour symptoms, the primary target of the majority
of admissions, which coincides with the inpatient treatment stay and the immediate
months following that stay.
CPRI-UWO 2008
42
T-Score
Four Time Points
100
95
90
85
80
75
70
65
60
Externalizing
Total Health
1
2
3
4
Wave
Figure 5 BCFPI Mean T scores across time: Wave 1 and 2 pre-admission, Wave 3
post-discharge, Wave 4 follow-up (N = 22)
Child and Family Wellness. Figure 6 reflects the repeated measures BCFPI child and
family functioning questions. These questions focus on success in friendships for all
family members, life restrictions due to the child‘s behaviour or illness, and quality of
family relations. Extreme deficits in the child‘s school and social functioning are
accompanied by the most striking feature of the pre-admission scores: the level of
parental distress at the time of referral to inpatient treatment. Again, significant
improvements are seen over time (lower scores), coinciding with the time of the CPRI
admission. While these substantial score reductions depict statistical and arguably clinical
significance, the level of family maladjustment remains highly impaired at follow-up.
This is further supported by the follow-up response to the question: Does the child
require further services? 69% said yes. When asked at the earlier post discharge time
point, 86% said yes.
CPRI-UWO 2008
43
T scores
Adaptive Functioning Over Time
110
105
100
95
90
85
80
75
70
65
60
Family Functioning
Child Functioning
N = 123
Pre
Post
Follow-Up
Figure 6 Average reduction in child and family distress over 3 years as
reported on the BCFPI. Scores above 70 are typically considered to be in the clinical
range of psychopathology. (N = 123)
The CAFAS risk ratings were calculated by clinicians throughout treatment, and
by our trained telephone interviewers at follow-up. As seen in Table 7, the child‘s
adaptive functioning and risk to self and others improves significantly at discharge and
over the short term. There is evidence that at follow-up, adaptive functioning has
deteriorated. Note that the final research rater, although a reliable CAFAS rater, may
have held a different bias than the previous raters who were clinicians working with the
client, which may have elevated the scores.
CPRI-UWO 2008
44
Table 7
CAFAS Repeated Measures subscale scores over a three year period
Subscale
CAFAS School
CAFAS Community
CAFAS Behaviour
CAFAS Mood
1
N
Intake
Mean
Post
Mean
Follow-up
Mean
134
134
134
134
26.3
11.3
22.5
18.7
15.5
5.4
14.8
11.6
21.0
10.6
17.8
14.4
Model
df  2 Sign1
2
2
2
2
79.8
30.8
49.0
35.4
.001
.001
.001
.001
Friedman Non-Parametric Test indicates statistically significant changes over time
Note the return to baseline risk levels within the community scores at follow-up.
Evidence that children and youth in the study were at increased risk to experience
numerous poor outcomes at follow-up can be seen in the list of seminal events reported
by the parents in Table 1. It is also clear that many chronic service users who experience
punitive contacts within the school and community, access inpatient mental health
treatments.
Caregivers reported their own levels of personal distress and difficulties in the
parent-child relationship reflected on either the Parenting Stress Index (PSI for children)
or the Stress Index for Parents of Adolescents (SIPA). The PSI/SIPA were not collected
from group home staff or temporary caregivers. These questionnaires were collected at
four data points3. Average levels of personal distress in caregivers were extreme on all
subscales at intake, reflected in means above the 99th percentile.
Visual inspection of the data indicated a large decrease in reported caregiver
distress following the targeted treatment, however there was little change between the
discharge and post-discharge time frames. Paired t-tests indicated the total PSI raw score
did drop significantly with a mean raw score change of 17 points from the high preadmission scores when compared to the PSI at the time of discharge, t(88) = 6.58,
p<.001, and compared to the PSI completed 6 months post-discharge, t(53) = 5.85,
p<.001. An analysis of the PSI subscales indicates that the reduced stress was seen
across all subscales, indicating parents saw an improvement in parent-child interactions, a
less difficult child, and less personal distress. The discharge and post-discharge PSI total
did not undergo any significant change with a raw score change of only 1. 5, t(40) = -.45,
p = .6, on those individuals who completed both the discharge and post-discharge PSI. A
3
As with all follow-up studies of this nature, attrition was a concern and the researchers took this concern
into consideration in considering the meaning of these results.
CPRI-UWO 2008
45
direct test of the long term follow-up data is hampered by attrition in questionnaire
completion, and alteration in the measure of stress to the SIPA, as most of the subjects
were older at the follow-up period. On the SIPA at long term follow-up, the median raw
score was 238 (N=65), which falls at the 84th percentile when compared to the
manualized norms. Far fewer caregivers reported extreme levels of stress over time in the
system of care. In contrast to the 96% of subjects who fell in the clinical range before the
target treatment, at discharge 68%, at post-discharge 71%, and at long term follow-up
49% fell in the clinical range of self-reported parenting stress.
These results corroborate caregiver reports found on the BCFPI. For example,
parents reported improvement in response to the question ―I felt depressed‖ on the
BCFPI. At referral, 24% of parents reported feeling depressed 5 or more days per week,
post-discharge this fell to 11.2%, and at long term follow-up 13.2%.
School outcomes The fact that these clients had been identified as special needs students
by the different school boards serving these subjects has already been reviewed in terms
of learning and social-behaviour needs. The CAFAS school subscale (see Tables 3 & 7)
indicated a significant improvement in school over the short term, with increased
difficulties again in the long term. The teacher ratings over time were considered as
repeated measures, however it should be noted that in longitudinal research such as
represented by the current study, the actual teacher rater does change over time, and there
was considerable attrition in that not all clients consented to our contacting their teacher.
On the Conners Teacher Oppositional Behaviour subscale, there was a steady, linear
decrease in classroom behaviour difficulties over time, F(1, 23) = 25.41, p<.001. This
same linear decrease, showing significant drops in acting out behaviours from preadmission to post-discharge, and another significant decrease at follow-up, was
corroborated by the SSRQ Teacher Externalizing subscale, F (1, 21) = 30.02, p<.001. As
in the analysis of parent report of symptoms however, the post and long term follow-up
scores, while much improved, on average continued to fall within the clinical range,
indicating a need for ongoing behavioural resource support in the school environment for
many students. The between subjects analyses were significant, with noteworthy
differences in outcome, pointing to the importance of considering differential school
functioning trajectories across subjects. Finally, it is important to note that at the end of
the data gathering stage, 81.5% of subjects reported to be in school. This seems to be a
remarkable number, especially given typical high school dropout rates for adolescents.
This matches the clinical observations of the authors that many teachers and principals
work diligently to ensure youth have opportunities to remain in school, despite periodic
aggressive outbursts. The qualitative interviews found that treatment classrooms that
could cope with emotional dysregulation and periodic aggression in school were essential
on the service wish list.
CPRI-UWO 2008
46
―I counted seven different school placements including residential placement. He
wasn‘t functioning in school. And if you can‘t function in school… if he doesn‘t
go to school, what does he do all day?‖ (family interview - Oosterveen, 2008).
Positive indicators Overall, in the presence of the high degree of ongoing support needs,
there are numerous positive indicators. At follow-up, 44% of children and youth had
BCFPI total mental health scores in the normal range, reflected in a T score below 70, in
contrast with only 14% at referral three years earlier. At intake 77% of children and youth
met Hodges‘ requirement for being pervasively impaired according to the CAFAS4 while
only 43% did so at follow-up. In addition to the fact that the majority of children and
youth no longer meet this risk threshold, a substantial proportion now exhibit either mild
or no risk on these individual subscales: 25% at School, 45% at Home, and 39% for
Behaviour towards others. On the Community subscale 61% of children and youth had a
CAFAS score of mild/no risk, which indicates that they are not in trouble with the law
currently.
Another positive indicator was skill development. Statistically significant social
skill gains were reported by both parents and community teachers on the SSRQ. The
SSRQ guardian report of social gains was similar when contrasted over the short term
(pre-admission SSRQ standard score = 66.7, SD = 10.6, post = 81.3, SD = 15.09; t(89) =
-8.9, p<.001) and long term follow-up (Pre SSRQ = 69.3, SD = 13.2, Post = 79.3, SD =
16.9; t(76) = -.4.86, p<.001). Note that these are standard scores, so guardians‘ initial
ratings of overall social skills for their child fell in the handicapped range, two standard
deviations below the mean. This is a primary reason that the social milieu therapy at
CPRI focuses on basic interpersonal skills such as greeting skills, conflict resolution,
conversation initiation, and assertiveness, while offering assessment and therapeutic
support from speech and language pathology and psychology. Note also that the
significant change during the intensive treatment period at CPRI is clearly distinguishable
in slope from the lack of change during the longer term follow-up. Teacher results, which
included different community teachers in repeated measurement (new grades over the
course of the 3 years studied), evidenced this same impact from intensive treatment.
Teacher pre-admission SSRQ standard scores were significantly impaired (M = 72.1, SD
= 12.1) compared to post discharge (M = 79.3, SD = 10.3; t(95) = -4.9, p<.001) and long
term follow-up scores (M = 83.1, SD = 13.2; t(40) = -5.3, p<.001).
―I wanted him to gain the skills to be able to deal with his emotions in an
appropriate way. He was able to come out of there with a lot of strategies to
bring home... that he still uses some of those strategies within his life so it made
a big impact on him as far as for the two years since he‘s been there (family
interviews - Oosterveen, 2008).
4
Hodges suggests that scores of moderate to severe risk on each of the following subscales: School, Home,
and Behaviour towards others reflects pervasive impairment.
CPRI-UWO 2008
47
The complete descriptive data, listing all obtained SSRS values in contrast to the
paired t-tests above, are found in Table 8 for parent ratings, and Table 9 for teacher
ratings.
Table 8
Social Skills Rating System - Parent
Pre-treatment Six-months post- Two-years post
Social skills (standard score)
(n=224)
(n=90)
(n=77)
Mean
68.17
81.32
79.32
Std. deviation
11.95
15.01
16.92
Table 9
Social Skills Rating System - Teacher
Pre-treatment Six-months post- Two-years post
Social skills (standard score)
(n=155)
(n=121)
(n=42)
Mean
71.74
79.50
83.14
Std. deviation
12.92
11.12
13.07
To summarize the average trends, there is positive news that many caregivers and
teachers report significant symptom improvement and social gains in the child, and fewer
parents report high stress and depressive feelings at follow-up. This must be tempered by
the ongoing high levels of support required by this tertiary sample. Their needs are
remarkable. In the roughly 2 years since the inpatient discharge, at least half of the
children and youth:
 Lived away from home again for treatment, safety/welfare, or
incarceration
 Were suspended from school
 Had been involved with the police
In the follow-up period, 16 children had to be out of home three or more times, and 22
were out of home on two occasions. In the follow-up period, 51 children were suspended
three or more times from school and 19 were suspended twice.
Differential Outcomes: Heterogeneity of change over three years in a system of care
In a sample of children and youth with multiple ongoing concerns, the next step is to
document any differential outcome trends. Did some children and youth not benefit from
treatment? Did any members of the sample evidence ongoing gains in home, school and
community adjustment, reduced suffering, and less interpersonal and intrapersonal
CPRI-UWO 2008
48
symptomatology? Having provided a detailed overview of the basic descriptive statistics,
an analysis of the substantial heterogeneity reflected in large outcome variability found
over time was required. A latent class cluster analysis was performed on the 170 subjects
who agreed to participate in the longitudinal follow-up, in order to uncover different
trajectories of change. Repeated measures BCFPI and CAFAS outcomes across three
points in time were simultaneously classified in order to ensure a comprehensive, multimodal (i.e., reported symptoms and real world adaptive functioning) approach to outcome
prediction. Both 2 and 3 cluster models initially appeared promising in terms of
theoretical and statistical modelling, with the 2 cluster model consistently having the
lowest BIC scores and lowest classification error (see Table 10) when predicting
longitudinal outcome trajectories. Individuals were assigned to the trajectory group for
which their posterior probability of group membership was largest (Haviland, Nagin,
Rosenbaum, & Tremblay, 2008). The BCFPI and CAFAS profile mean changes over
time can be seen in Figures 7a and 7b, the two cluster model results. Note that the model
generated two groups with non-overlapping scores at any wave of data collection.
Two thirds of the subjects (cluster 1 – ―improved‖, 68% of 170 subjects assigned)
demonstrated statistically and clinically meaningful reductions in reported externalizing
and internalizing symptoms and negative life events; while one third of the children and
youth (cluster 2 – ―not improved‖, 32 % of subjects assigned) began with the most severe
scores and did not appear to benefit significantly from the intensive inpatient experience,
nor did they demonstrate any long term improvements (Figures 7a and 7b). In particular
in cluster 2, there is a pattern identified that those without symptom improvements in
intensive treatment are at risk of ongoing serious life difficulties as reflected in very high
CAFAS scores two years later. Further analysis of these high CAFAS scores on cluster 2
subjects at follow-up indicates that 96% of these children and youth scored 20 to 30
(moderate to severe impairment) on the school subscale and 63% scored 20 or 30 on
community subscale, which reflects a high probability of experiencing negative
outcomes, such as school suspensions and expulsions as well as trouble with the law.
Table 10
Model Fit Information for Latent Class Analysis (n=170)
1-cluster model
2-cluster model
3-cluster model
4-cluster model
CPRI-UWO 2008
Log Likelihood
BIC Value
Parameters
-4141.25
-4189.78
-4163.56
-4146.73
8544.13
8507.96
8522.28
8555.47
12
25
38
51
49
BCFPI Mean (T Score)
Two Latent Class Cluster Trajectories Over 3
Years
100
95
90
85
80
75
70
65
60
Not-Improved
Improved
1
2
3
Wave
Figure 7a. BCFPI Total Mental Health mean profile T scores for the improved and nonimproved latent class probability clusters. Wave 1 is preadmission to CPRI, Wave 2 is
average six months post-discharge, and Wave 3 is two years post-discharge. Two thirds
of subjects fall into the improved group, showing significantly reduced symptoms that
were sustained over time. One third of subjects showed no significant change in
externalizing and internalizing mental health symptoms over inpatient and outpatient
treatment efforts in the system of care in southern Ontario.
CPRI-UWO 2008
50
CAFAS Mean Total
Two Latent Class Cluster Trajectories Over 3
Years
150
130
110
Not-Improved
Improved
90
70
50
1
2
3
Wave
Figure 7b. CAFAS Total mean profile scores for the improved and non-improved latent
class probability clusters. Wave 1 is preadmission to CPRI, Wave 2 is average six
months post-discharge, and Wave 3 is two years post-discharge. Two thirds of subjects
demonstrated reduced punitive school and community contacts, while one third are rated
as having very poor home, school and community adaptation (high scores), which
includes violence towards others and reduced safety and personal coping.
CPRI-UWO 2008
51
Using known predictors of static and dynamic risk in developmental
psychopathology (Leschied, Chiodo, Nowicki, & Rodger, 2007), the resulting latent class
clusters were analyzed for distinguishing characteristics. As well, in order to guard
against arbitrary metrics (Kazdin, 2006) and understand whether the two groups of
subjects differed on real world outcomes and concomitant reports, distinguishing seminal
events and independent teacher reports were analyzed to confirm theoretical group
distinctions.
Recall that latent group outcomes were determined by different change
trajectories on both parent symptom reports and CAFAS clinician ratings of adaptive
functioning. There was also independent confirmation of these latent group assignments
from other informants. All teacher reports of functioning at the end of study were in the
expected direction, i.e., more pathology and fewer skills found in the not improved group
(cluster 2). Statistical significance testing was hindered by the small sample size for
teachers reporting on the not improved group at the end of study. Significantly more
problem behaviours were seen in cluster 2 on the SSRQ teacher total problem behaviours,
t(60) = -4.52, p<.001, while not reaching statistical significance for the Conners‘ teacher
Oppositional subscale, t(41) = -1.65, p=.106.
Other seminal events distinguished these two clusters, as summarized in Tables
11 and 12. A repeat out of home placement in the two years following discharge was
more common in the not improved group (70%) than the improved group (51%), X² (1, N
= 170) = 5.32, p=.021. Mental health inpatient treatment and youth incarceration was
twice as common in the not improved group over the two year follow-up (Table 11). The
extent of medication prescribed as the child left the inpatient unit predicted long term
outcomes. The greater the number of different medications prescribed at discharge (a
proxy for pathology), the more likely the child belonged to the non-improved group,
t(161) = -2.1, p=.037. While 41% of cluster 1 (improved) subjects were involved with
police at some point over the two year follow-up, in comparison, 75% were in cluster 2,
X²(1, N = 170) = 16.94, p<.001. It is notable that this involvement was at least in part due
to violent (e.g., assaultive) behaviour almost half (47.9%) of the time in cluster 2
subjects, but only 15.2% of the time in cluster 1 subjects, X²(1, N = 170) = 19.21, p<.001.
While these cluster types appear to differ substantially in difficult behaviours and
negative life events experienced, we found numerous population variables that typically
distinguish developmental outcomes in the general population were not significant
predictors in this unique sample. For example, age, sex, income, child welfare status, and
IQ scores were not statistically different on significance tests of mean comparisons across
the two clusters. Interestingly, age of onset of mental health concerns as reported
retrospectively by parents also did not distinguish the two groups. The median age of
service entry was 6 years for both cluster groups, with standard deviations of 2.2 and 2.7
in the two clusters.
CPRI-UWO 2008
52
In terms of predicting cluster membership based on pre-admission evaluations, at
referral teacher report of problems (SSRQ Total Problem score) significantly predicted
group membership, F(1, 120) = 14.92, p<.001. This indicates that teacher‘s reporting of
the highest degree of internalizing and externalizing symptoms at pre-admission, was
predictive of those children who did not make health and functioning gains over time.
Likewise, an extremely high degree of parental stress at intake was predictive of
membership in the poor outcome group (PSI total score; F(1, 142) =4.67, p=.032).
One construct that has had some importance in the literature is the level of family
engagement while the child is in group care. Unfortunately this was not measured directly
through staff ratings. However a proxy for this construct may be found in questions asked
of parents on the satisfaction scale at discharge. First ―were you involved in treatment
planning‖ revealed no significant relation to cluster membership. Second ―did you find
staff helpful?‖ was significantly predictive (F(1, 78) = 5.53, p=.021). A final area of
interest, could staff living on an inpatient unit with their clients predict long term
outcomes? Green et al. (2007) in a sample and treatment model very similar to the
present one indicated that status on the inpatient unit was related to outcome at one year
post discharge. However this has not been a common finding in the literature. Staff
opinions regarding the outcomes were not asked directly in the present study, but staff
did rate the child on the SSRQ Total Problem Scale. There was a trend for the nonimproved group to receive higher pathology ratings from inpatient staff who knew them
well, F(1, 137) = 2.95, p=.08. These exploratory results will require replication in another
sample of youth with severe psychopathology, as category membership is probabilistic
only.
CPRI-UWO 2008
53
Table 11 Latent Class Cluster Service Differences
Percentage of Children Receiving Mental Health Services
Latent Cluster
Receiving services?
Out of home?
If so, inpatient/resident?
If so, group/foster care?
If so, youth corrections?
Psychotropic
medication? a
Pre-Treatment
Improved
(n=116)
100
68.1
42.2
45.7
6
93.9
Pre-Treatment
Not-Improved
(n=54)
100
77.8
55.6
46.3
11.1
96.3
Two-Years Post
Improved
(n=116)
86.8
50.9
14.0
14.9
9.6
65.2
Two-Years Post
Not-Improved
(n=54)
90.4
69.8
28.3
13.5
17.3
76.9
Note. The values represent the percentage of residents answering YES with available data (valid
percentages). Out of home percentages do not add to 100 as children experienced more than one out of
home placement.
a
10% of clients were reportedly refusing their prescribed medications at follow-up (adolescents primarily).
Table 12 Latent Class Cluster differences at end of study
Latent Cluster
Could something have been done the past 2 years
to help your child?
At follow-up, is your child in school?
In the past 2 years, has your child been involved
with the police for property crimes?
In the past 2 years, has your child been involved
with the police for violent behaviour?
Do you feel your child requires further care (end of
study)?
Improved
(n=116)
49.1
Not-Improved
(n=54)
73.1
82.6
79.2
12.5
35.4
15.2
47.9
59.3
88.7
Note. The values represent the percentage of residents answering YES with available data (valid
percentages).
CPRI-UWO 2008
54
DISCUSSION
Descriptions of the level of need in the present sample are striking. Children, youth and
their caregivers in this tertiary sample experience numerous stressors and repeated
setbacks over time. Ongoing or chronic use of social support, mental health, child
welfare, and special educational services is evident both pre and post admission, the
documentation of which is missing from much of the research on tertiary treatment
currently available. The results confirm that a period of four month intensive inpatient
psychiatric milieu therapy combined with community/caregiver supports and full access
to a treatment classroom did have a significant impact in reducing symptomatology as
reflected in multi-informant, multi-modal tracking measures. This reduction was evident
in a clinically meaningful and statistically significant reduction in symptom measures as
reported by both caregivers and teachers, improved functioning and reduced youth risk as
rated by clinicians and measures of parent satisfaction, increased observed social skills as
reported by both the teacher and caregiver, and reduced parental distress and familial
conflict over time.
The present investigation can be viewed as a replication and extension of the
work of Green et al., (Green et al., 2007) in the U.K. in offering clear evidence that
children and youth with severe externalizing problems (troubling) and requiring
psychiatric attention (troubled) can markedly benefit from a structured, community
linked psychiatric inpatient treatment program with gains that can be seen within a
system of care model for two years. Benchmark effect sizes exist for non-control group
repeated measure inpatient treatments of varying characteristics: ―the mean weighted
effect sizes for children in residential care concerning general, internalizing, and
externalizing problem behaviors are +.60, +.45, and +.60‖ (Knorth et al., 2008, p. 136).
Presently the larger effect sizes (over .70) seen in our externalizing and total mental
health symptom improvement scores, maintained over a much longer follow-up period,
aligns with the intensive nature of our psychiatric milieu facility, and the primary reasons
for which children and youth are referred to these specialized tertiary beds.
In a rare random assignment comparison of effect sizes in child and youth tertiary
treatment, Wilmshurst in London Ontario (2002) and Henggeler, et al., (2003) in the U.S.
found parent report of behaviour problem scores reduced by approximately 10 points on
the total T score over one year for both inpatient and intensive outpatient programs. The
present study corroborates this benchmark intensive treatment impact of 10 standard
score points on average on the BCFPI Externalizing Scale, seven months post-discharge.
Our numbers indicate outpatient services prior to the target admission show little mean
change, short term residential impact shows substantial behaviour problem reduction
(effect size = .92), while long term system of care and developmental change on average
for all subjects over three years also denotes strong behaviour problem reduction (effect
size = .78). Monitoring the average outcomes is essential practice for any tertiary
CPRI-UWO 2008
55
intervention that is considered a long term and costly practice. We feel there is now
sufficient replication of these mean score outcomes to report that agencies working with
comorbid mental health disordered populations should consider medium to large pre-post
effect sizes a realistic goal for many clients.
Outcome research over the past 20 years has demonstrated that rather than ―no
evidence‖ of the benefits of out-of-home care, there is clear evidence that substantial
reductions in referral symptoms can be seen at home and school. However, attempts to
demonstrate that clients are ―cured‖ or no longer in the clinical range of symptoms have
not been at all successful in comorbid tertiary samples. Our results add to the growing
evidence that intensive tertiary care programs can expect to see extreme or crisis
symptom scores at referral (T>80), followed by mean symptom reduction to a level of
outpatient treatment needs (T = 70) at the end of service. This is also clearly seen in the
data tables of several independent studies, where ―high symptom‖ referents show gains of
similar magnitude (T80 —> T70) to ―low symptom‖ referents (T70 —> T60), such that
the end point for those most in distress equals the intake data for others5. This data adds
to the growing necessity for coordinated system of care models. In the current example,
there is the clear evidence that inpatient programs should ―step down‖ into outpatient and
in-home supports, following the successful reduction of symptoms in the crisis period
which often reflects aggression and/or emotional instability.
Frensch and Cameron (2002) focused on the importance of ―after care‖, as their
review found that gains made during out-of-home treatment dissipated over time. In the
face of the high ongoing needs in the present sample, the continued report of ―outpatient
level‖ scores over the following two years after discharge is not surprising. It was
somewhat surprising to the authors upon our initial analysis of repeated measures, that
over the course of the many months between post-discharge and the follow-up time
frames, externalizing and internalizing symptom reports did not show a steady increase,
nor did parental stress or depression, or increased teacher complaints of class behaviour.
The only strong indicator of nonlinear change was on the CAFAS, which monitors and
directly ranks life events such as school suspensions, police contacts, aggression to
others, self-harm, etc, where one such negative event can in and of itself ―spike‖ a score
into the clinical range. This observation prompted the analysis of between subject scatter,
which demonstrated that measured standard deviations grew larger after the intensive
target treatment, which points strongly to a differential treatment effect. This would
indicate that the depiction of a large drop in response to inpatient treatment, followed by
maintenance of gain but no worsening, may be an example how the average ‗masks‘ the
true story. An attempt to delineate clients into subsets based on differential trajectories of
5
See for example Halliday-Boykins, Henggleler, Rowland, & DeLucia, 2004; for the same effect in
CAFAS score changes across severe and less severe clients see Hodges, Xue & Wotring, 2004
CPRI-UWO 2008
56
change over time was indicated (Halliday-Boykins et al., 2004; Hodges et al., 2004;
Rosenblatt & Furlong, 1998)6.
Two distinct groups were delineated through latent class analysis once the
between subject differences were categorized. In contrast to previous research, we
clustered change trajectories over time on two measures simultaneously: parent report of
symptoms and clinician rating of functioning. Group 1 consisted of two thirds of subjects
reflecting large initial symptom reduction and long term improvement in parent and
teacher reports of symptoms and clinician rating of adaptive functioning. This group also
reflected the importance of ongoing service usage post discharge, with periodic negative
life events remaining well above non-clinical literature controls. This observation
indicated the ongoing use of services as a key variable not tracked adequately in the
mental health literature. We are now undertaking research to understand long term
service use patterns in child and adolescent mental health in Ontario (Centre of
Excellence grant, Reid et al., 2008-2010). While our present tracking of service usage
was retrospective based on parent/guardian report, we note that Henggeler et al., (2003)
utilized a monthly telephone service utilization interview, a prospective protocol
recommended for future investigations.
The second group identified through latent class cluster analysis indicated that a
third of children and youth did not appear to benefit significantly from either inpatient or
outpatient treatment efforts over time. This group tended to have the highest level of
referral symptoms of psychopathology according to parent report, teacher report, and
higher initial caregiver distress and preadmission CAFAS risk ratings. This group was
discharged from inpatient treatment with higher rates of psychotropic medication therapy,
a proxy for pathology. At follow-up this group was marked by concerning levels of
aggressive behaviour, and double the rate of inpatient mental health treatment and
incarceration.
Attempts to distinguish the characteristic features of the improved and not
improved groups indicated that common markers such as age of onset, child protection
status, gender, income and IQ identified in the broad epidemiological literature were not
useful predictors. Halliday-Boykins et al. (2004) note that factors determining initial
levels of symptom severity may not be identical to predictors of symptom chronicity or
treatment responsiveness in tertiary care. It is hypothesized that some of these variables,
such as an early age of onset of psychopathology, would be a key predictor of a more
heterogeneous client population. Presently, high comorbid symptoms with previous
placement breakdowns at home and school were the norm, with minimal variability. Our
initial analysis of symptoms (e.g., conduct disorder versus bipolar disorder) did not
6
The reader unfamiliar with the potential utility of identifying trajectory subgroups using latent class
cluster analysis in non-experimental longitudinal data is referred to Mun et al., (2008); see also Haviland,
Nagin, Rosenbaum & Tremblay (2008) for an interesting discussion of these powerful new methods of
distinguishing developmental trajectories with differential response to treatment.
CPRI-UWO 2008
57
indicate this readily distinguished improved or not-improved status, but the investigators
are continuing to analyze this data in a more detailed fashion.
Despite the similarities in broad static risk factors, the early onset of significant
behavioural problems, family dysfunction, punitive school and community contacts, and
admission to inpatient mental health treatments, these two groups of children and youth
have not experienced the same number of negative life events in the three years in which
they were followed. The fact that they are distinguished immediately by lowered
symptoms may indicate some factor that determines differential treatment
responsiveness, which was not monitored. The cluster analyses do reveal striking
differential outcomes in multivariate longitudinal outcomes which require replication. For
the two thirds of subjects in cluster 1, large treatment effect sizes were indicated across
various measures, with gains sustained over time. In contrast, for cluster 2, the BCFPI
total mental health mean change was close to 0, and the CAFAS change reflected a
marked quadratic function, with small improvement followed by marked deterioration
accompanied by negative life events. The majority of children/youth in cluster 2 required
out-of-home services and were in conflict with police over the long term. These
differences would certainly hold clinical significance and have implications for resource
allocation should predictors be determined. It is however supported by similar findings
from Halliday-Boykins and colleagues (2004) who reported that high symptom tertiary
clients showed two outcomes; those that do not respond to intensive treatment and those
that do both initially and over time. If replicated, it would be very helpful to
administrators and service planners to know that roughly two thirds of those in tertiary
child mental health care who begin contact with the mental health system at a very young
age and grow to develop complex comorbid symptomatology requiring inpatient
treatment, are likely to receive periodic special education, child welfare and mental health
attention, but can benefit over time from these intensive services. The other third of the
children/youth will show few gains and require high levels of ongoing support and
expensive legal and hospital-based resources. This points the mental health literature in
the direction of the life-course persistent versus time limited offenders theoretical
literature.
Statistically, the authors of this study considered two explanations for the
improvement, corroborated by parent, teacher and clinician reports, seen in the majority
of clients. These gains coincided with the time frame of the intensive inpatient
multidisciplinary treatment at CPRI. First, this could indicate a dose-response
relationship where the most intensive 24 hour treatment available for comorbid
symptoms in southern Ontario demonstrated a large impact. Second, this could indicate a
common pattern of measurements taken at the time of referral tending to be inflated, as
they reflect a crisis, and subsequent measurements will necessarily decrease or ―regress
to the mean‖. Methods to rule out the second option include either the use of alternate
treatment control groups, or more frequent measurement of symptoms and adaptive
CPRI-UWO 2008
58
behaviours not linked to referral and discharge time points. As reported, 18% of
parents/guardians in the four waves repeated measures analysis had been interviewed on
the BCFPI twice before the targeted admission. For these subjects there was no
significant symptom decrease over the first two measurements, then a large symptom
decrease coinciding with the CPRI inpatient stay, then little significant change during the
―after care‖ phase. As well, latent class cluster 2, the non-improved group, demonstrated
no regression to the mean in symptoms, and this was related to higher pre-admission
symptom ratings, greater teacher concerns, and more chronic life difficulties. In the
future it is strongly recommended that research of intensive treatment programs require
repeated measurements of symptoms and functioning during the referral process, in an
attempt to develop wait-list control effect sizes for tertiary care.
Clinically, the authors of this investigation who collectively represent many years
of experience in the tertiary care mental health, correctional, and educational service
systems, did not expect to be surprised by the needs found in a complex tertiary inpatient
sample of children and youth. However, many of these children are experiencing more
distress and receiving more services than we had envisioned. For example, prior to
admission, 71% of these young children had already spent time away from their parents
due to safety or mental health needs. Equally striking, at follow-up, a major neuroleptic
medication (with side effect concerns) was used in low doses to control agitation and
aggression and was prescribed to 85% of those contacted. The high percentage of
children/youth involved with the children‘s aid society was expected. However the sheer
number of out-of-home moves, comorbid diagnoses, amount of medication, familial
stress, school suspensions and police contacts was overwhelming. There is an obvious
indication that these children, youth, and their families require a carefully organized,
coordinated system of care over the long term.
The first author was reminded that in teaching undergraduates, explaining the
classic 35 year follow-up of the Perry preschool longitudinal study is always a highlight,
as 36% of the treatment (good preschool) group had 5 or more arrests by the age of 40!
(see www.highscope.org). In relative terms, this is a good outcome as 55% of the control
group had 5 or more arrests by 40 – it was a high risk sample! The High Scope
foundation goes on to claim very large savings to the taxpayer due to increased
employment and decreased police and court costs over many years. In the same vein,
41% of the improved group had police contact over our two year follow-up, which is a
relatively good outcome when compared to the 75% in the non-improved cluster of
individuals. Clearly, any attempt to understand this sample in one snap shot, target
unitary symptoms, assess outcomes after a single trial of evidence-based treatment, or
ignore broader systems issues would seem misguided. Similar to the substance abuse
literature on harm reduction, there appears to be a small number of mental health child
and youth consumers where the long term service outcome goal will be reduction of harm
and improved quality of life rather than any form of ―cure‖ or removal from the service
CPRI-UWO 2008
59
system. In the adult mental health literature, clustering of subtypes within a tertiary
population has now attempted to distinguish these trends (W. V. Rubin & Panzano,
2002).
Social scientists interested in developmental psychopathology in comorbid
tertiary samples will want to extend our scientific method (Foster & Kalil, 2008); clinical
administrators our understanding of the coordination of support systems. Keiley et al.
(2003) and Haviland et al. (2008) offer an example of the level of sophistication required
by scientists to further research in this area, while a well funded federal government
initiative in the U.S. (http://www.systemsofcare.samhsa.gov/) offers an example of the
type of system of care planning that is required by senior clinical administrators.
In tertiary care, both research and clinical endeavours must adopt longitudinal
research methods in reporting on continuity of care models. For a comprehensive history
of residential care for children and youth, and the manner in which standardized
measurement can inform good clinical decision making in the public service system and
greatly reduce costs, readers are referred to the work of John Lyons (2004) and his
excellent book on improving the children‘s mental health system. Some of the reforms
Lyons recommends and demonstrates with empirical support have already occurred in
Ontario. For example, implementing a service wide system of objective assessment of
who requires the most intensive and expensive treatments, can both save dollars and offer
more appropriate community supports to families. Anecdotally, at CPRI the number of
children referred to the inpatient psychiatry units has been significantly reduced over the
past 5 years. It is presumed that this reduction represents a shift to a greater use of
community resources, now that standard screening is being emphasized within each
community.
The present investigators, having reviewed the literature on comorbid child and
youth developmental psychopathology and intensive inpatient treatments, and now
mapped the life stories of 170 children and youth over three years, offer the following
recommendations for tertiary care, with a check-mark offered where we feel the southern
Ontario service system is particularly well-positioned:
 Triage or screen low symptom profiles away from the intensive, expensive
tertiary care system 
 Measure symptoms, functioning, skill change and service usage over time
using structured, standardized measurement tools
 Include at least two pre-measures and two post-measures to allow for a true
wait-list ―control‖ comparison to periods of intensive treatment or placement
change
 Research in different regions across Ontario is needed to begin to track the
nature of treatments provided, including access to evidence based treatment
 Track service usage in tertiary care with a ‗quick‘ phone check-in every three
months
CPRI-UWO 2008









60
For applied research, record multiple perspectives on valid standardized scales
to fully understand needs and outcomes. Presently, risk ratings on the CAFAS
identified different trends from the BCFPI symptom scales 
Use prospective samples, with consent to track and contact involved agencies
obtained at the outset, using a common unique identifier across the Ministry
of Children and Youth, Ministry of Health, Ministry of Education, and
Ministry of Community and Social Services
For school services (a key service provider), a report card and IEP analysis
would be beneficial to understand academic outcomes, days absent,
suspensions, and social functioning.
There is evidence that treatment classrooms are required that can both weather
periodic aggression and program for academic progress that will change from
one day to the next. This is due to ongoing changes in availability for learning,
which is caused by active psychiatric symptoms and executive functioning
deficits, and in-home conflicts
Planners must build community wide linkages in forming a system of care
A number of tertiary care clients require long term, ongoing care and do not
benefit from discrete, disjointed service episodes, which place families back
on wait lists for support
Children and youth referred to CPRI for inpatient services typically access
out-of-home care a number of times in their lives. At least half will again do
so after leaving CPRI. This indicates that community services must consider
the nature and costs of tertiary follow-up services they are prepared to offer at
the time of referral. For these clients, a short stay in treatment is clearly not
able to eliminate long term mental health and child welfare service access, but
is geared to stabilize and support crisis symptoms safely. Other models of
intensive services (e.g., multidimensional treatment foster care, multisystemic
therapy) must be considered, as Ontario currently has no clear system to
handle ongoing aggression in children and youth without relying on
fragmented community service systems or Children Aid Societies to fund
placements.
Staff training in tertiary inpatient mental health should focus on linkages and
continuity of care, with some attention paid to ongoing staff morale knowing
that presently one third of clients will show little response to treatment and
will return to treatment providers to utilize intensive services on more than
one occasion
Graduate level social science students should be welcomed into applied
clinical-research settings, to experience messy ―real world‖ data sets. Students
can thus enhance their awareness of the need for multivariate statistical
CPRI-UWO 2008

61
methods that can handle data in multiple forms, change over time, and more
than one related covariate with differential attrition
Scientist – practitioner relationships, such as the current one between CPRI
and the University of Western Ontario that was fostered by the Provincial
Centre of Excellence for Child and Youth Mental Health, should be assisted in
terms of time allocation and dollars 
Strengths and Limitations
This study provides support for positive effects in the majority of consecutive admissions
to a regional tertiary centre up to two years following discharge. However, conclusions
regarding long-term effectiveness of the system of care in southern Ontario must remain
tentative due to methodological issues in the course of the evaluation. These issues
include:
1. There was no random assignment or control group.
2. Treatment protocols were not controlled, monitored or checked for fidelity.
3. Issues related to maturation could have impacted outcomes, although the
between-subject variance would indicate this was not a uniform threat to
validity. Concerns that gains were simply a function of normal development
are also tempered by the sudden change in slope seen during the most
intensive inpatient service.
4. Due to some missing data, certain analyses could not be conducted.
5. Fourth, rater changes (across teachers or across guardian changes) indicate
that this repeated measures study is not necessarily a within persons repeated
informant study. We hope to understand the potential impact of rater changes
over time in future analyses of this data.
6. As cluster membership is probabilistic, the finding that a majority of clients
benefit from treatment requires replication.
With a one sample design, interpreting the large initial gains seen on all outcomes
is under threat of regression to the mean. In this event, the authors assume that referral
measures will indeed emphasize the negative and minimize the positive, in order to
impress upon the system of care the need for immediate assistance. This would account
for the extreme scores, reflected in averages more than three standard deviations above
the population mean. While we were unable to routinely collect two baseline measures
(no overall ―wait list‖ control), on the subset of subjects for whom this was available, the
slope of change was markedly different only following the period of inpatient treatment,
indicating regression to the mean or maturational arguments for change would not fully
account for the findings.
Other limitations should be noted in reference to this study. While in the present
sample chronic needs are highlighted, we recognize this is a restricted sample chosen for
CPRI-UWO 2008
62
its homogeneity in terms of severity of presentation. The primary concern at the time of
application for funding was that all data were obtained from a single tertiary agency. This
data source limitation is likely to have an impact on the ability to generalize our findings
across other treatment providers, although the subjects were referred from a large, diverse
base of communities, each with their own treatment services during the pre and postdischarge years of the study. This regional status offered an opportunity to examine
outcomes in different regions of the Province. Analyses of outcomes based on county of
referral, did indicate some trends in referral differences, but did not indicate that long
term outcomes were statistically different across the large catchment area. This is
somewhat surprising given the large discrepancy in service allocation between urban and
rural areas. It may be that small sample sizes in some areas hindered the statistical
analysis. We are in the process of further analyzing this data. It is positive news that
diverse communities across southern Ontario are offering treatment support and seeing
gains in children and youth who pose the highest level of need.
The factors our agency would claim as clinical strengths, such as flexible and
individualized treatment planning, the ability to handle diverse co-morbid clinical
children/youth, supporting reluctant clients with high staffing levels to reduce worker
burn-out, all hinder applied research measurement and treatment control (Hair, 2005).
Presently, there was the use of manualized treatments on some units of service for some
clients, but no coded archive of treatment provision or measurement of treatment fidelity
was taken, a major obstacle to accounting for the variability in outcomes.
There is no definitive form of out-of-home treatment, with centres varying widely
in terms of the services offered, the size, structure and children served (e.g., Wells,
1991). This diversity makes it difficult to draw conclusions and generalize findings from
studies of residential psychiatry and inpatient populations. It also highlights the
importance of conducting program evaluation in each setting and providing descriptive
information and details related to the type of services offered and children served (Brady
& Caraway, 2002). Our results might be less relevant to units that do not treat children
and youth with highly complex, co-morbid mental health problems, or those with no
onsite psychiatrist, no onsite school, or with more intact parental guardianship.
As well, one obvious theoretical predictor to help distinguish our cluster 1 from
cluster 2, family and child treatment engagement, was not measured directly in this
investigation. As a proxy to this, we did look at client satisfaction, with questions such as
―did you participate in treatment planning; were you comfortable speaking with staff‖ but
these ratings tended to be uniformly positive and with such little variance they made for
poor predictors of long term wellness. In a tertiary sample with similarly severe histories
of psychosocial adversities and comorbid psychiatric symptoms, Hussey and Guo (2002)
charted other family characteristics, such as parental mental illness and incarceration, that
could be mapped in future attempts to determine the predictors of different trajectories in
developmental psychopathology.
CPRI-UWO 2008
63
This is a single group repeated measures analysis of consecutive referrals. We
have reported the effect size and utilized the pooled standard deviation, a conservative
estimate of change. It should be noted that our large treatment effect sizes will be
inflated from what would be found in a randomized control group analysis7. To address
this we compared our effect size to other one-sample designs in the literature, and
demonstrated a larger than average treatment impact.
At the assessment stage, there are no universal weighted equations or algorithms
guiding clinicians in making recommendations to clients about the use of intensive,
restrictive treatment. Investigating placement determination, which reflects service usage
decisions in both the tertiary mental health and school intervention literatures, indicates
that predicting who requires segregation from their class, school and/or home is a
difficult task. This is understood within the framework of contextual variables, where it
is not possible to predict intensity, duration, or restrictiveness of interventions based on
child characteristics alone (Glassberg, 1994; Snowden, Leon, Bryant, & Lyons, 2007).
Outcome studies for residential treatment have been limited historically by poor
methodology and difficulties with appropriate comparison groups. This study
demonstrates clearly that tertiary treatment is a complex and dynamic intervention where
the active change ingredients are not easily identified. Hence, there is continued
uncertainty as to the effective components of the treatment, optimal length of admission,
suitability of treatment type and length given presenting problems, and identification of
for whom long-term positive effects are sustained (Bickman, Foster, & Lambert, 1996;
Green et al., 2007). In their commentary on evidence-based psychotherapy (EBT)
compared to ―usual care‖, Weisz and Gray (2007) cite examples of control treatments
that aided children and youth as well as effective EBT. Unfortunately they note that
written protocols describing the successful ‗real world‘ treatments did not exist. They
recommend ―studying usual care with care‖, not just as a control condition. Due to the
complex logistics regarding tertiary treatments, it is often difficult to identify the key
factors that facilitate positive change. For example while we know that CPRI focuses
heavily on behavioural parent training and social work family counselling and support,
we do not have the data presently to know which families accepted and engaged fully in
this support, versus those which focussed solely on child psychiatric symptom reduction.
The lack of randomized clinical trials and specific controls to ensure treatment integrity
has lead to inconsistencies in results and recommendations in the past. This study was
not designed to demonstrate the efficacy of specific treatments. Given the absence of a
control group and the lack of treatment fidelity measurements we cannot make definitive
conclusions as to the key ingredients in the success of our patients. In the future, process
analyses that focused on specific differences in residential approaches such as describing
different assessment and treatment plans, the percentage of time spent with family, the
types of individual and group therapy, acceptance of psychiatric medications, ―dosage‖
7
For more information regarding this issue see Carlson & Schmidt, 1999.
CPRI-UWO 2008
64
levels of all forms of therapy will no doubt be useful. It is only by showing an empirical
relation between outcomes and well-described interventions that true insight will be
obtained with respect to what makes the difference in tertiary inpatient child and youth
care (Harder, Knorth, & Zandberg, 2006; Kendrick, 2007).
Finally, it should be noted that there is also a lack of consensus on how to define
improvement following residential care. Symptoms among children and youth can wax
and wane, suggesting that simply looking at symptom presentation is too narrow a
perspective (Sourander et al., 1996). Hence, the multi-modal, multi-informant approach
taken in the current investigation should be seen as a real strength. The addition of
qualitative data should also be considered by all investigators. It is interesting that in our
qualitative interviews and satisfaction scales, even in the face of poor symptom outcome
reports reflected in the cluster 2 analysis, adult caregivers reported the service as
beneficial. Is this an outcome of treatment, an increased parental awareness of realistic
goals given the diagnoses and prognoses that have been shared with them over time?
Among the strengths of this study was the anticipated use of multiple informants
at critical points in a continuum of service to report on symptom status of the
children/youth. The dual coding of symptoms (BCFPI) and impairment (CAFAS) was
essential. Investigations that simply measure symptoms cannot claim adequate
comprehension of impairment. For example, in ADHD literature reported by Gordon et
al. (2006), it was found that symptom measurement was unable to predict more than 25%
of the variance in impairment. However, due to poor return rates on questionnaire data
from teachers and youth self-report presently, our results are more limited in both scope
and generalizability than we had hoped. Attrition in teacher and self-report data hindered
the full test of our latent class predictor model, although the subset of teacher reports did
support the other observations statistically and clinically. The outcome measures we
came to rely on, the BCFPI and CAFAS, are well-situated given their status as mandated
measures in Ontario child mental health centres. The present findings offer benchmark
outcome results to all agencies in Ontario interested in evidence based practice and
longitudinal investigation. Presently, we have added considerable supporting data to the
use of the BCFPI and CAFAS battery adopted by the province as beneficial to both triage
efforts to sort the level of intensity of service required and as outcome monitoring tools.
To our knowledge this is the first published account of the validity of the BCFPI as a
measure of change over time.
Future Directions
From an empirical perspective, the gains made in growth mixture statistical modelling in
the field of child and adolescent delinquency (e.g., the distinction between life-course
persistent offenders and adolescent limited offending), can be adopted by the field of
tertiary mental health care (Foster & Kalil, 2008; Halliday-Boykins et al., 2004). General
CPRI-UWO 2008
65
improvement over the short term following intensive tertiary treatment was well
documented in our literature review, but with the caveat that not all youth made gains,
and not all gains were maintained. Previous work has readily distinguished front line and
tertiary type referrals. This investigation eliminated low risk individuals in order to more
clearly delineate whether complex, high needs referents could be distinguished in terms
of the course of long term functional status and symptoms. We found that differential
outcome trajectories should be a focus of future research in tertiary care.
Our review identified two published reports in a tertiary mental health youth
sample that clearly mapped different youth trajectories on symptom change reflected in
CBCL Total parent report profiles over time (Halliday-Boykins et al., 2004; Rosenblatt &
Furlong, 1998). Their samples differed from the present one in terms of treatment and
sample characteristics. Halliday-Boykins found their highest needs subjects also divided
into improved versus unimproved symptom trajectories, but did not differ strongly on
baseline symptom levels, nor did treatment offered distinguish the symptom outcomes.
Despite the presence of inpatient treatment and outpatient supports, Halliday-Boykins
report that half of their participants experienced chronic symptom elevations over 16
months. Their non-improved group tended to be younger, a finding not consistent in the
literature, and to have caregivers who perceived themselves as capable and empowered
by the service system. Interestingly, they found that risk predictors of baseline symptoms
levels such as poverty did not predict long term outcomes, a finding which was
reinforced in the present findings. Finally, they too report that unimproved symptoms
were predictive of increased out-of-home placements over 16 months, a finding
replicated in the present study.
We also found that for one third of our sample, symptoms based on the BCFPI
Total parent report did not diminish and poor functional outcomes reported on the
CAFAS were confirmed over three years, in a population admitted with a more
externalizing comorbid problem focus and higher overall symptoms scores than the
studies from the Rosenblatt and Henggeler groups. Many of our clients that did improve
in symptoms reports and functional risk appeared at an ―outpatient level‖ of
symptomatology and support post-discharge, apparently benefitting from extensive,
ongoing service use including medication therapy and special education supports. Only
12% of our entire sample reported no ongoing involvement in the 2 years post-discharge.
In comparing Figure 7a above, with Halliday-Boykins (2004) data trends, total
parent report of symptoms over time was remarkably similar in our 2 cluster model to
two of their clusters which reported the high symptoms improved and high symptoms
unimproved groups. As well as replicating and lengthening the developmental time line
on their trajectory profiles, our effect sizes over time were also remarkably similar. Our
BIC scores did not indicate adoption of more complex cluster models; this may be due to
the fact that our sample was less heterogeneous, not including the lower symptom
subpopulations identified in Halliday-Boykins et al., (2004), Hodges et al., (2004) and
CPRI-UWO 2008
66
Rosenblatt and Furlong (1998), which accounted fully for the extra clusters of subjects
they identified. 8
The extensive use of ongoing services found in Halliday-Boykins in the U.S. and
in our Ontario research points strongly to the need to consider chronic care models of
support, beyond discrete or disjointed treatment episodes. For example, the Life Needs
Model of Service Delivery, developed in London Ontario, envisions services to support
community participation and quality of life for children and youth with disabilities (King
et al., 2002). This developmental model of disability support, which crosses individual
needs, interpersonal skills, and community spheres of service, would aid theoretical and
practical service planning, and guide mental health clinicians working with young people
disabled by more chronic neurological or mental health symptoms in considering a more
holistic view of ongoing supports. Within this broader perspective, research could then
examine particular client, treatment and socio-demographic variables that might predict
the most efficient, effective treatment responsiveness in tertiary care (Gorske, Srebalus,
& Walls, 2003). Presently, there is an indication that those with fewer school problems at
pre-admission, and those with lower parent stress, benefitted most from support and
maintained gains over more than two years following a psychiatric milieu therapy
admission and system of care services.
At this point in the application of tertiary care for youth, it would appear that
inpatient and residential treatments remain of significant utility. However, it is assumed
on a theoretical level by many that there must be a better (less expensive, less intrusive)
way to appropriately treat this group of children/youth. It is unknown how many inpatient
referrals could be safely diverted to intensive in-home community alternatives, which are
often unavailable and considered intrusive by some families (versus inpatient which is
intrusive to the child!). Models of how to reduce the use of treatment ―beds‖ can be found
in the parallel literature on youth incarceration. Intensive community alternative
programs have now demonstrated their success in the area of adolescent delinquency,
with well developed trials of multisystemic therapy (MST) reported by Scott Henggeler‘s
treatment and research group at the Medical University of South Carolina (Henggeler et
al., 1999) and multidimensional treatment foster care (MTFC) reported by Chamberlin
and Smith (2003) in Oregon. Both of these community-based interventions have
demonstrated their benefits to youth and their families in a cost effective manner.
Unfortunately, in a well designed randomized, prospective, longitudinal follow-up study
of 156 youth, an attempt to move tertiary psychiatric care to the community did not
demonstrate enhanced success (Henggeler et al., 2003). However, the Henggeler group
did offer evidence that while hospitalization led to more rapid behaviour symptom
improvement according to parents, MST resulted more rapidly in less time out of home
8
Note that at admission Halliday-Boykins had some patients with very low CBCL scores while our
multiple gating intake appropriately diverted these clients to outpatient supports as recommended by Lyons
et al. (2001).
CPRI-UWO 2008
67
and time out of class, but did not prevent re-hospitalization. At the close of the study, one
year after the target psychiatric crisis, both hospitalization with regular care follow-up
and MST led to equally positive gains in symptoms and functioning. The authors noted
however that the gains were difficult, and that the model of MST that was successful for
delinquency required substantial modification for mental health targets, and was still
unable to prevent re-hospitalization.
Clearly, all out-of-home or group care agencies should routinely monitor
outcomes on readily obtained rating scales; examples of which include those developed
by John Lyons and Kay Hodges. Lyons‘ state-wide analyses of aggregated outcome
trajectories allows for a direct comparison of efficacy across tertiary agencies. Mapping
trajectories of change over time allows a visual comparison of treatment programs
accepting similar levels of entry severity in client symptoms (comparing beginning
intercepts on a graph) yet differing treatment effect sizes (comparing slopes of change on
a graph) (Helgerson, Martinovich, Durkin, & Lyons, 2005; Lyons et al., 2001). We offer
the present long term effect sizes as the beginning of this bench marking process in
Ontario. This type of outcome research is more attainable than randomized assignment
(RCT) comparisons of inpatient care versus outpatient care, which is hampered by some
guardians specifically requesting a safe or respite environment.9 An alternative to the
RCT is to delineate the ―natural selection bias‖ found when comparing tertiary
community and tertiary residential services, thus allowing for a comparison of outcomes
without random selection; see the propensity analysis research by Haviland et al., (2008).
It is anticipated that inpatient group families may differ on levels of family stress or
severity of presenting problems when contrasted with families who accept intensive inhome support.
It is also underscored that, rather than avoidance of tertiary care placement based
on the ―last resort‖ concept, it is important to evaluate in which situations a temporary
out-of-home or group care strategy can be most meaningful (Whittaker & Maluccio,
2002). Long term outcomes should not only focus on symptom reduction but also
measure other substantial outcomes such as remaining in an academic setting, extracurricular involvement, maintaining employment, staying out of the criminal justice
system and improving the overall quality of life.
Another future direction is the investigation of the impact on mental health
workers when improved triaging of clients is utilized (Lyons, 2004). At CPRI, our
inpatient staff report that the children seem more damaged and their problems more
complex over the past decade. A check of archived data indicates there is evidence that
this is true of our referrals. For example, the average T scores for presenting problems at
intake for CPRI inpatient referrals are indeed increasing over the past 15 years. If only
children and youth with the most intensive needs are accepted into a program, workers in
that program will require ongoing professional support and education that is focused
9
Exceptions to this are represented by the studies by Henggeler et al., 2003 and Wilmshurst, 2002.
CPRI-UWO 2008
68
specifically on the vicarious trauma that may be experienced. For example the present
data set was shared broadly within CPRI, and will be shared directly by the lead authors
with our sister agencies, in order to promote an understanding regarding the ramifications
of such ongoing high support needs. This same education is important for administrators.
Managers in the service system who insist that success be defined only as less service
usage in the future would be labelling many inpatients seen at CPRI over the past four
years as treatment failures, a perception that cannot be shared with an appreciation of the
data from the present study. Rather, directing children and youth with the most severe
needs to an intensive level of ongoing mental health and school support, while expensive,
is simply acknowledging that a chronic care model is implicated for some children/youth
according to the current empirical findings. Insisting these clients have no case
management continuity, repeatedly closing service and proceeding through new intake
procedures to re-enter service is a source of unnecessary frustration with little evidence of
success.
Deliverables
To CPRI
Internal presentations and newsletters were provided to all staff in order to utilize the
present research to impact clinical planning and staff training. As well, this data now
serves as a benchmark for treatment effectiveness; as such open access to this data has
been offered to the CPRI Research Coordinator and Senior Management. CPRI has
recently altered programming in the inpatient services it provides, and program
evaluation efforts are now utilizing this comparison data, for example see Appendix E
page 94.
To the Ministry
1/
The Province of Ontario has chosen to adopt a commonly used clinical rating
measure (CAFAS), and a new parent report measure (BCFPI). Given the lack of validity
research on the BCFPI, we analyzed the BCFPI for:
i.
Integrity of the original factor structure in a more severe clinical sample
- Confirmed (Table A)
ii. Concurrent validity with a well established measure, the Conners parent
rating scale - confirmed
iii. Capacity to distinguish outpatient and inpatient referrals (Figure 2)
iv.
Utility as an outcome measure (Figures 5, 6, 7)
Our analyses offer evidence on the construct validity of BCFPI. Adoption of the
BCFPI for CPRI inpatient services was of concern to the first author, given the brief
screening purpose resulting in a diminished item pool. Fears of a ―ceiling effect‖ were
erased in that the present severe sample did score significantly higher than outpatient
CPRI-UWO 2008
69
referrals. Likewise, the measure showed sensitivity to change over time, with the greatest
degree of symptom reduction occurring during the most intensive period of treatment, as
would be predicted, and convergent teacher reports were found.
This data offers the Ministry evidence that the BCFPI can play the intended role
of triaging referrals for access to more expensive, intensive resources, and agencies may
wish to adopt this same measure for outcome monitoring – with benchmarks available
based on the present study. The telephone interview format had significant advantages
over questionnaire mail-outs at follow-up, as attrition was greatly reduced.
These BCFPI validity results are in preparation by the authors for submission to a
clinical child psychology journal.
We also confirmed previous research indicating the CAFAS measure can triage
and predict service need, and be utilized for outcome monitoring that is distinct from but
correlated to symptoms. The present results indicate that the current use of this measure
province wide could be used to map trajectories of adaptive functioning in children and
youth receiving mental health services. Hodges has already demonstrated the utility of
this measure to aid broad system-wide planning.
2/
The present study offers benchmark effect sizes for tertiary care and longitudinal
outcomes, along with recommendations as to the need for a chronic care model. These
results and the recommendations contained therein (Executive Summary) should be
shared with other tertiary agencies in Ontario. Unfortunately our submission to present
this data at the Children‘s Mental Health Ontario conference was declined. We will
explore other means to share our results within the child and youth clinical community in
Ontario, with the assistance of the Centre of Excellence.
3/
As a regional centre, the chart reviews indicated the difficulty in understanding
service provision across a large catchment area. Ideally, a unique, common identifier
would depict client access to services throughout children‘s mental health Ontario, with
storage in a common database. This is a large undertaking that would have to be directed
by Ministry efforts.
A simple recommendation that can be adopted by each agency: Identify the
agency, location, nature of involvement and role clearly on all documentation. Some
agency reports found on file contained just a logo letterhead; no full name, no address, no
indication of service role, and at times not even a date on the report. When serving the
local community they may assume that ―everybody knows ABC Services‖, but for
coordination at a broader level this knowledge should not be assumed, and failing to
identify services and dates breaches professional responsibility.
4/
Our analysis of county of origin, indicated no large difference in long term
outcomes. We are undertaking more detailed analyses. A description of differential
trends across the 17 individual counties will be offered to each of the 10 Single-Points-ofAccess that refer to CPRI, in order to assist them in resource planning and boost moral
CPRI-UWO 2008
70
over positive findings in the face of much personal client distress. We will de-identify
other counties during these presentations given to any one county.
5/
We will ask our Community Advisory Board (see Appendix F) to consider the
implications of the present investigation and the best means of knowledge dissemination
to a broad association of community partners. Local presentations at CPRI and to
Ministry personnel have already begun. Presentations at scientific conferences have been
undertaken (see Appendix D). The bulk of the outcome results are under manuscript
preparation for journal submission, and we would ask that the Centre not post these full
results online before we hear the outcome of our submissions.
CPRI-UWO 2008
Appendix A
BCFPI Full Data Set
BCFPI Reliability Statistics
CPRI-UWO 2008
71
72
BCFPI
T Score
N
Pre-Admit
Mean (SD)
N
Post
Mean (SD)
N
Follow-up
Mean (SD)
Attention,
Impulsivity
224
74.75 (8.61)
162
68.43 (9.63)
169
69.15 (10.63)
Cooperation
224
76.75 (8.26)
162
70.16 (11.57)
168
69.95 (11.55)
Conduct
224
92.11 (29.28)
162
74.49 (28.23)
170
72.78 (27.09)
Externalizing
224
83.02 (9.57)
162
73.56 (12.45)
169
73.37 (12.82)
Separation
anxiety
222
63.60 (14.85)
160
61.44 (15.48)
169
60.64 (15.30)
Manage
Anxiety
224
60.79 (15.36)
162
63.81 (14.19)
170
59.66 (13.91)
Manage
Mood
223
75.94 (17.53)
161
66.22 (18.40)
168
66.0 (16.55)
Mood +
Self-harm
221
79.90 (19.32)
160
67.99 (19.12)
167
68.11 (18.20)
Internalizing
223
70.66 (14.06)
159
67.38 (14.32)
168
64.70 (13.71)
Total Mental
Health
224
80.56 (9.87)
160
73.14 (12.23)
169
71.87 (12.72)
Social
participation
222
84.66 (16.07)
158
72.12 (18.91)
167
72.60 (18.50)
Quality
relationships
214
76.17 (10.13)
147
65.43 (13.13)
158
65.32 (13.45)
School
219
80.04 (14.37)
154
67.50 (17.87)
164
66.93 (15.51)
Global Child
Functioning
222
86.16 (12.18)
156
71.79 (16.64)
166
72.09 (15.40)
Family
activities
204
114.84 (34.64)
136
92.24 (38.59)
137
92.58 (40.56)
Family
Comfort
196
82.69 (13.13)
137
74.04 (17.07)
119
71.81 (14.97)
Global
Family
206
101.52 (20.65)
139
84.60 (26.23)
141
84.71 (26.16)
CPRI-UWO 2008
73
Reliability Statistics as a measure of Internal Consistency on symptomatology subscales
in the Brief Child and Family Phone Interview (BCFPI)
Subscale
Cronbach‘s Alpha
Items
Regulation of Attention,
.75
6
Impulsivity, and Activity
Cooperativeness
.77
6
Conduct
.68
6
Externalizing
.78
18
Separation from parents
.79
6
Managing Anxiety
.83
6
Mood
.82
6
Internalizing
.83
18
Total Mental Health
.82
36
CPRI-UWO 2008
Appendix B
Clinical admissions by county of referral
CPRI-UWO 2008
74
75
CPRI Catchment Area Map of Southern Ontario with referent numbers by county
N=9
N=3
N = 10
N=6
N=9
N=9
N=2
N=8
N = 72
N = 25
N = 21
N = 17
N = 12
N = 10
N=7
N=4
N= 225*
*1 client came from out of catchment area
CPRI-UWO 2008
Appendix C
CPRI data collection
CPRI-UWO 2008
76
Measurement Tools used at Different Time Periods throughout Evaluation
Appendix D
Knowledge Transfer
79
Curriculum Vitae for the ―Role of Residential Treatment for High Needs, High Risk
Children and Youth: Evaluating Outcomes and Service Utilization‖ Grant
Publications
St. Pierre, J. (2007). The use of the BCFPI at CPRI. Psychology Ontario, Special Issue:
Screening and outcomes measures in children’s’ mental health services, 13-14.
St. Pierre, J., Cook, S.F., Johnson, A.M. (manuscript in preparation for journal
submission). Validation of the Brief Child and Family Phone Interview in a tertiary
mental health treatment sample.
St. Pierre, J., Leschied, A., Cullion, C., Stewart, S.L. (manuscript in preparation for
journal submission). Differentiating three year outcomes following tertiary child and
youth inpatient psychiatric treatment.
Presentations
St. Pierre, J., Cook, S.F., Leschied, A., Cullion, C., Johnson, A.M., & Stewart, S. (2008).
Influencing policy and programs in systems of care: How can data collection and
analysis methods impact our decisions? Presented at the 21st Annual Research
Conference of the University of South Florida Research and Training Center for
Children‘s Mental Health, Tampa, Florida, February.
St. Pierre, J. (May, 2007) Situating the Role of Residential Treatment for High Needs,
High Risk Children and Youth. Ministry of Children and Youth Services Southwest
Region Centre of Excellence Research Day, Windsor.
St. Pierre, J. (Feb, 2007). BCFPI/CAFAS outcomes at CPRI/MCYS. Ontario
Psychological Conference, Toronto, Invited Workshop.
Refereed Poster Presentations
Cook, S.F., Cullion, C., Noftle, J.W., St. Pierre, J., Leschied, A., Stewart, S.L., Johnson,
A.M. (2008). Construct validation of the Brief Child and Family Phone Interview in a
population of high-risk, high-needs children. Presented at the 2008 Convention of the
Canadian Psychological Association, Halifax, Nova Scotia, June.
Cook, S.F., Cullion, C., Noftle, J.W., St. Pierre, J., Leschied, A., Stewart, S.L., Johnson,
A.M. (2008). Situating the role of residential treatment for high-needs, high-risk children
and youth: Preliminary outcomes. Presented at the 2008 Convention of the Canadian
Psychological Association, Halifax, Nova Scotia, June.
Cook, S.F., Cullion, C., Stewart, S.L., & Johnson, A.M. (2007). Showing ―restraint‖:
Understanding the management of aggressive behaviour in a residential care facility.
CPRI-UWO 2008
80
Presented at the 2007 Convention of the Canadian Psychological Association, Ottawa,
June.
Cordick, J., Cullion, C., Cook, S.F., St. Pierre, J., Stewart, S., Johnston, K., Schepens, T.,
& Johnson, A.M. (2007). A longitudinal assessment of the trajectory of change for
children and youth in mental health treatment. Presented at the 2007 Convention of the
Canadian Psychological Association, Ottawa, June.
Cullion, C., Cook, S.F., Stewart, S.L., Swart, G.T., & Johnson, A.M. (2007). The
proactive use of medications to manage aggressive behaviour in residential care.
Presented at the 2007 Convention of the Canadian Psychological Association, Ottawa,
June.
Tanel, N., St. Pierre, J., Swart, G.T., Wahab, A., Cullion, C., & Johnson, A.M. (2007).
Longitudinal Assessment of Global Functioning among High-Risk High-Need Children:
The Impact of Stimulant Medications. Presented at the 2007 Convention of the Canadian
Psychological Association, Ottawa, June.
Non-Refereed Poster Presentations
Stewart, S.L., Cullion, C., Hamza, C., St. Pierre, J., Cook, S.F., Leschied, A., & Johnson,
A.M. (2008). Short-term and Long-term Inpatient treatment: Effectiveness of Intensive
Intervention for Children with Complex Mental Health Needs. Presented at the
Children's Mental Health Ontario Meetings, May.
St. Pierre, J., Leschied, A., Cullion, C., Stewart, S.L., Cook, S.F., Noftle, J., & Johnson,
A.M. (2008). Situating the Role of Inpatient Treatment for High Needs, High Risk
Children and Youth. Presented at the Children's Mental Health Ontario Meetings, May.
St. Pierre, J., Chung, J., Stewart, S. L. & Cullion, C. (July, 2006). Situating the Role of
Residential Treatment for High Needs, High Risk Children and Youth. 19th Biennial
Meeting of the International Society for the Study of Behavioural Development,
Melbourne, Australia.
Completed Student Theses (abstracts attached below)
Cordick, J. (2007). A Longitudinal Assessment of the Trajectory of Change for Children
and Youth in Mental Health Treatment. King‘s University College Undergraduate
Psychology Thesis.
King, A. (2006). Relations between Academic Achievement and Emotional and
Behavioural Disorders in a Residential Population. University of Western Ontario
Undergraduate Psychology Thesis.
CPRI-UWO 2008
81
Oosterveen, J.A. (2008). Client Satisfaction with Mental Health Services: A Qualitative
Approach to Understanding the Family‘s Experience of Residential Care. University of
Western Ontario M.Ed Counselling Psychology Thesis.
Tanel, N. (2007). The ability of a brief global functioning scale to identify psychological
and pharmacological treatment effectiveness. King‘s University College Undergraduate
Psychology Thesis.
Current Student Theses (abstracts attached below)
Gerrits, J. (2009). Examining the Impact of Residential Treatment: Relevance of Family
and the Out-of-Treatment Environment. University of Western Ontario M.Ed
Counselling Psychology Thesis.
Noftle, J. (2009). The Trajectory of Change for Children and Youth within Residential
Treatment. University of Western Ontario M.Ed Counselling Psychology Thesis.
Skrodzki, D. (2009). Wards of the State: Examining Residential Treatment Outcomes
for Children Involved in the Welfare System. University of Western Ontario M.Ed
Counselling Psychology Thesis.
Yaremko, S. (2009). Gender Informed Analysis of Residential Treatment Outcomes.
University of Western Ontario M.Ed Counselling Psychology Thesis.
CPRI Newsletters: ―Bringing It All Back Home‖
Dr. St. Pierre gave periodic presentations to all clinical staff and then made summary
newsletters available to all staff.
#1 Mental health residential client satisfaction at CPRI, Fall 2006
#2 Is it just me, or do the children and families in residential treatment at CPRI really
require more supports than they used to?, Spring 2007
#3 Outcomes, Spring 2008
#4 Dr. St. Pierre will summarize aspects of this final report for all CPRI staff in a
―Bringing It All Back Home‖ Newsletter in late 2008
CPRI-UWO 2008
82
University Theses Based on CPRI Outcome Data supported by Funding from the
Centre of Excellence in Children’s Mental Heath
(Abstracted)
CPRI-UWO 2008
83
CLIENT SATISFACTION WITH MENTAL HEALTH SERVICES:
A QUALITATIVE APPROACH TO UNDERSTANDING THE FAMILY’S
EXPERIENCE OF RESIDENTIAL CARE
Julia Oosterveen M.Ed (Counselling Psychology)
2008
INTRODUCTION: One avenue of assessing the treatment impact of children‘s mental
health services is to explore the perceptions of parent‘s of the clients' mental health
services in the context of the perceived quality of care that the children and youth
received This study explored through qualitative methodology the following: (1) The
forces most salient in affecting children's well-being after they are discharged. (2) The
relationship between satisfaction and measures of the severity of the service recipient's
problems at admission. (3) The relationship between levels of satisfaction and current
level of functioning. (4) The extent to which client's perceive the treatment as effective.
(5) The predictors of satisfaction.
METHOD: A purposive sample of seven parents comprised of both ‗successful‘ and
less ‗successful‘ graduates of treatment were interviewed. Participants were the parents of
children and youth who were mental health residents admitted to the Child and Parent
Resource Institute (CPRI) in London Ontario between January 14th, 2003 and February
26th, 2007. The nature of the presenting problem at in-take for these children and youth
included conduct/behavioral difficulties, mood disorders, anxiety, depression, family
problems and developmental disorders. Standardized screening prior to each child/youth
admitted into CPRI included the Brief Child and Family Phone Interview (BCFPI).
Symptoms of the children admitted to CPRI based on the BCFPI were, on average, two to
three standard deviations above the mean compared to a non- treated population of
children and youth of similar age. A standardized interview format was used for data
collection in meeting with consenting parents. All Interviews were held in the homes of
the families.
RESULTS: Content analysis served as the basis of analysis, with the unit of analysis
being the themes that were identified within the transcribed interview content. Consistent
themes identified by parents - in descending order by frequency count - included: the
emotional impact on the family of having a special needs child; the nature of the
treatment their child/youth received; concerns about medication; the impact of treatment;
future concern for their child; the need for parental support; school adjustment problems;
and the search for understanding regarding the cause of their child‘s problems.
DISCUSSION: The most significant finding in this study is that regardless of whether
the child who entered into treatment was considered to be high or low functioning upon
discharge from the facility, parents viewed the intensive short-term treatment as an
extremely valuable service that should continue in the future. Families reported
benefiting from having their child in treatment even if the focal child showed no
measurable or quantifiable improvements in their overall functioning. This finding held
CPRI-UWO 2008
84
true even for those families whose child showed little change in functioning postdischarge. In addition, regardless of their child‘s functioning post-discharge families
indicated that not only would they recommend services to another family, but they would
return to the residential treatment facility if they perceived their child needed intensive
services in the future. These results are similar to research conducted by De Boer,
Cameron & Frensch, 2007 who suggested that families within their study perceived their
child‘s residential treatment program as beneficial despite the fact that the focal child did
not achieve a comparable level of positive change. The benefits of utilizing qualitative
methodologies in exploring parent‘s perceived benefit from their child/youth‘s treatment
program are described as well as recommendations for future research.
References
De Boer, C., Cameron, G., & Frensch, K. (2007). Siege and Response: Reception and
Benefits of Residential Children‘s Mental Health Services for Parents and
Siblings. Child and Youth Care Forum, 36, 11-24.
CPRI-UWO 2008
85
THE TRAJECTORY OF CHANGE FOR CHILDREN AND YOUTH WITHIN
RESIDENTIAL TREAMENT
John Noftle
INTRODUCTION: Despite several large scale reviews indicating that a period of
residential treatment, on average, improves functioning for children and youth (Frensch
& Cameron, 2002; Hair, 2005; Knorth et al., 2008), uncertainty remains regarding the
trajectory through which change occurs (Lyons et al., 2001). Gaining insight into not just
that this service works, but also how it works is of significance to both practitioners and
policy makers (Knorth et al., 2008). Therefore, the goal of this study is to identify the
characteristics associated with positive or negative trajectories of symptom change for
children and youth who are receiving treatment within this system of care program.
METHOD: Using a consecutive sampling strategy, participants will be children and
youth age 6 to 16, admitted voluntarily into residential treatment at the Child and Parent
Resource Institute (CPRI), in London Ontario from October 1st, 2002 to July 1st, 2005
(N=225). The Conners‘ Global Index-Parent (CGI-P) and the Conners‘ Global IndexTeacher (CGI-T) were selected as the primary outcome measures to examine individual
trajectories of change in externalizing behavior over time. These scales consist of 10
items that provide a sensitive method for monitoring response to treatment and
determining when behaviors of interest are within normal limits (Conners, 1999). Given
that the primary interest is in modeling both whether the child‘s externalizing behavior
improved, and when the greatest change occurred, event history analysis (i.e. survival
analysis) is the optimal statistical technique (Singer & Willett, 2003), and will be
employed in this thesis.
RESULTS: To be completed.
DISCUSSION: To be completed.
CPRI-UWO 2008
86
EXAMINING THE IMPACT OF RESIDENTIAL TREATMENT: RELEVANCE
OF FAMILY AND THE OUT-OF-TREATMENT ENVIRONMENT
Julie Anne Gerrits
INTRODUCTION: Despite the fact that residential treatment is the most intensive and
expensive form of treatment a child or adolescent can undergo, there is relatively little
research on the long-term outcomes, or effectiveness, of residential treatment. Also,
although it is known that not all youth maintain initial treatment gains (Frensch &
Cameron, 2002), little is known regarding whom those children are, or more specifically
the context of the out-of-treatment environment to which they return. When one is
discharged from residential treatment, he or she is typically returned to a family
environment, which will likely influence the maintenance of treatment gains. Hence the
current study examined various family system variables to determine whether they
predict or influence the maintenance of treatment gains. In addition, even though
residential treatment is typically both a child and family treatment program, very little
research has examined whether there are any changes in parents or family functioning
indicators (Knorth, et al., 2008). Thus the objectives of the current research are to
examine: (1) the short-and long-term outcomes of residential treatment, (2) whether
family system variables predict or influence whether or not residential treatment gains are
maintained post-discharge, and (3) whether residential treatment produces any changes in
parental and family functioning indicators.
METHOD: Participants consist of 98 children and adolescents ranging in age from 6 to
16 who were consecutively admitted to the Child and Parent Resource Institute (CPRI)
between October 2002 and July 2005. The youth in the current study were purposefully
selected from a larger sample of children and adolescents (N = 225) at CPRI to include
only those who were both discharged to a family and lived with a family two-years postdischarge. Data for the current study were collected with the Brief Child and family
Phone Interview (BCFPI), the Parenting Stress Index- Short Form (PSI-SF), as well as
from one question on the CPRI satisfaction questionnaire. The BCFPI was selected as the
primary outcome measure when examining changes over time in youth‘s internalizing
and externalizing problems as well as family functioning indicators. Caregivers
completed the BCFPI at admission, 6-months post-discharge and 2-years post-discharge.
The PSI-SF was selected as a measure of parental stress, and is used to examine whether
caregiver stress predicts or influences youth‘s outcomes post-discharge. Caregivers
completed the PSI-SF at admission, discharge, and 6-months post-discharge. Given that
the current research is examining various trajectories of change from admission to twoyears post-discharge, a series of repeated measures ANOVA‘s, ANCOVA‘s, and MixedModel ANOVA‘s will be conducted.
CPRI-UWO 2008
87
GENDER INFORMED ANALYSIS OF RESIDENTIAL
TREATMENT OUTCOMES
Sarah Yaremko
INTRODUCTION: The etiology, risk factors, and functionality of female adolescent
aggression tends to differ from that of male adolescent aggression. These differences may
impact treatment outcome. Unfortunately, many treatment programs for girls have been
informed by male specific research (Hipwell & Loeber, 2006) with a ―one-size fits all‖
approach being applied in the treatment of adolescent females (Graves, 2007). This
singular treatment modality also exists within residential treatment programs that were
originally established to serve the needs of boys with aggressive and delinquent behaviors
(Kirigin, 1996). It is undetermined as to whether these models of treatment for aggression
and delinquent behavior are an effective and viable option for female adolescents with
disruptive behavior (Handwerk, Clopton, Huefner, Smith, Hoff, & Lucas, 2006). The
purpose of the this study is to examine the outcome of treatment for aggressive girls
within a residential treatment setting who were followed for a period of up to two years
post discharge from a residential treatment center in London, Ontario. Specifically, the
association of gender, externalizing and internalizing behaviors, and treatment will be
explored.
METHOD: This study will consist of a convenience consenting sample of children and
youth between the ages of 6 and 16. All children and youth will be accepted in the Child
and Parent Resource Institute (CPRI), which is a large, regional children‘s mental health
center operated directly by the Ministry of Children and Youth Services in London
Ontario. Children involved with the study will be those accepted into CPRI between the
years 2002 and 2005. The Brief Child and Family Phone Interview (BCFPI) will be used
as a measure of residential treatment outcome. Caregivers completed the BCFPI at
admission, as well as six months and two years post-discharge. Specifically, the
Externalizing and Internalizing subscales will be used to infer treatment efficacy for
adolescent females.
CPRI-UWO 2008
88
WARDS OF THE STATE: EXAMINING RESIDENTIAL TREATMENT
OUTCOMES FOR CHILDREN INVOLVED IN THE CHILD WELFARE
SYSTEM
Donna Skrodzki
INTRODUCTION: Children involved with the child welfare system who have
experienced maltreatment often suffer a variety of emotional, behavioural, cognitive and
social difficulties that require intensive forms of treatment that far surpass outpatient,
community-based resources. In such cases, residential treatment frequently becomes the
next viable alternative for chronically disturbed children. Although the benefits and
utility of residential treatment have been well documented in literature (e.g., Hair, 2005;
Frensch & Cameron, 2002), very little research has examined how treatment outcomes
may vary as a function of involvement with the child welfare system. The present study
will examine the trajectory of change in the problematic behaviours of children served in
residential treatment. More specifically, differences among children involved with child
welfare system, children with no involvement with the child welfare system and children
with no CAS involvement with at least one past incident of CAS involvement will be
compared on internalizing and externalizing problems from time of admission to two
years post-discharge.
METHOD: Participants will include a consenting, convenient sample consisting of
consecutive referrals accepted into residential treatment at the Child and Parent Resource
Institute (CPRI) in London, Ontario between October 1, 2002 and July 1, 2005. Two
hundred and twenty five residents between the ages of six and 16 will be investigated:
children with CAS involvement at the time of admission, children with no CAS
involvement at the time of admission and children with no CAS involvement at the time
of admission with at least one past incident of CAS involvement. Child emotional and
behavioural problems will be assessed using scores on two subscales measuring several
internalizing and externalizing functioning factors of the Brief Child and Family Phone
Interview at admission, six months post-discharge and two years post-discharge (BCFPI,
Cunningham, Pettingill, & Boyle, 2004). Data will be analyzed using repeated measures
analysis of covariance (ANCOVA).
CPRI-UWO 2008
89
A LONGITUDINAL ASSESSMENT OF THE TRAJECTORY OF CHANGE FOR
CHILDREN AND YOUTH IN MENTAL HEALTH TREATMENT
J. Cordick
INTRODUCTION: Attempts at tracking changes in functioning throughout residential
treatment and post-discharge have yielded equivocal results from a demonstration of
significant improvement, to decline. A dose-effect relationship has been used to explain
how treatment can improve functioning, suggesting that if a service is effective, then
increasing the intensity of the service will produce even better outcomes (Lyons, Terry,
Martinovich, Peterson & Bouska, 2001). In this study, children / youth in residential
treatment showed a decrease in high-risk behaviours, with reductions in suicidal ideation,
self- mutilation, and aggression towards others. Asarnow, Aoki and Elson‘s (1996)
identified that boys released from treatment had a 32%, 53%, or 59% risk for readmission at one, two, and three years post-discharge, respectively. The present study
investigated the change status of children and youth who received treatment in one large
residential treatment centre over a two year period, improving on reported outcomes by
using a standardized multidimensional rating scale, and multiple assessment points.
METHOD: This study drew on data involving 585 participants, aged 5 to 18 years, who
were referred to the Child and Parent Resource Institute (CPRI) in London, Ontario,
between 2002 and 2006. A standardized testing battery at CPRI including multiple
administrations of various instruments included the Child and Adolescent Functional
Assessment Scale (CAFAS; Hodges, 2000). The CAFAS was rated at multiple points
during treatment, including every three months during treatment and at six-month followup. The CAFAS (Hodges, 2000) measures impairment in functioning in eight areas of the
child‘s life (higher scores indicate more impairment of function): School/Work, Home,
Community, Behaviour towards Others, Moods/Emotions, Self-Harmful Behaviour,
Substance Use, and Thinking.
RESULTS: A continuous, normally-distributed, age-independent variable was created by
taking the standardized residual from the regression of age onto each scale score of the
CAFAS. Time was then regressed onto this residual score using both linear and quadratic
regression models. All eight subscales demonstrated significant quadratic regression
equations.
DISCUSSION: Results indicate that treatment at CPRI was successful at reducing
functional impairment over time, across all subscales and for the aggregate impairment
index. Results also indicate that every scale, with the exception of Substance Use,
experienced some post-discharge worsening. While it is encouraging to note the
improvement in functioning in the youth after treatment, it is clear that not all gains are
maintained after treatment ends. Future research should investigate the impact of postdischarge environment, to identify factors that impact on an individual‘s ability to
maintain benefits of treatments
CPRI-UWO 2008
References
Asarnow, J. R., Aoki, W., & Elson, S. (1996). Children in residential treatment: A
follow-up study. Journal of Clinical Child Psychology, 25, 209–214.
Hodges, K. (2000). Child and Adolescent Functional Assessment Scales, 2nd Revision.
Ypsilanti, Michigan: Eastern Michigan University.
Lyons, J.S., Terry, P., Martinovich, Z., Peterson, J., & Bouska, B. (2001). Outcome
trajectories for adolescents in residential treatment: A statewide evaluation.
Journal of Child and Family Studies, 10, 333-345.
CPRI-UWO 2008
90
91
LONGITUDINAL ASSESSMENT OF GLOBAL FUNCTIONING AMONG HIGH
RISK HIGH NEED CHILDREN: THE IMPACT OF STIMULANT
MEDICATION
N. Tanel
King’s University College
INTRODUCTION: Residential care facilities provide relief for exhausted and frustrated
parents, a consistent and predictable environment for the child, and special programs
designed to shape increasingly positive behavioural and emotional responses (Hair,
2005). In addition, since many children showing signs of problem behaviour are
prescribed a complex array of medications, residential treatment centres additionally
provide guidance for the pharmacological treatment of behavioral disorders. The use of
stimulant medication in particular has demonstrated improvement in social and classroom
behavior and academic abilities, decreases in oppositional and aggressive behaviour and
influence alertness, inhibition focus and integration of sensory information (Tucha et al.,
2006). The goal of this study was to report on the impact of medication on the behaviour
of children/youth in residential treatment.
METHOD: Participants for this study included 200 clients at the Child and Parent
Resource Institute (CPRI), all of whom demonstrated severe behavioral disturbances. All
of the residents had been admitted to the program between 2002 and 2005 ranging in age
from 6 to 16 years. Approximately 85% of the sample was male. The Conners‘ Global
Index Scale (CGI) was used as the behavioural measure. Data was collected on a biweekly basis using parent, teacher, and staff reports. Medication including dosage was
tracked for each client. Each resident was evaluated to determine the medication change
that occurred at two week intervals as a function of whether the medication/dosage
remained constant, increased or decreased.
RESULTS: Sensitivity of the CGI to changes over time yielded a significant multivariate
effect [F(9,46) = 3.26, p < .05] with a significant effect of time on parent ratings
[F(2,155) = 4.78, p < .05]; a significant improvement between admission and week six,
and between admission and week eight; a significant effect of time on teacher ratings
[F(2,131) = 3.61, p < .05]; significant decline between admission and week eight. No
effect was found regarding time on staff ratings [F(2,154) = 1.96]. In assessing sensitivity
of CGI to dosage changes over time, CGI teacher ratings used and scores were
categorized by the extent of change within the previous two weeks. These results
reflected a behavioral decline and in relevant areas a behavioral improvement greater
than 0.5 SD decline. A significant relationship was identified between medication status
(stayed the same, increased or decreased) and teacher CGI, [2(4) = 15.902, p < .05],
consisting of the following: weeks in which no changes were made to medications were
significantly less likely to result in behavioral improvement; weeks in which dosages
were increased were significantly more likely to result in behavioral improvement, and
significantly less likely to result in behavioral decline; no significant effects demonstrated
for weeks in which dosages were decreased
CPRI-UWO 2008
92
DISCUSSION: Results suggest that the CGI is sensitive to treatment effects measured
over time within a population of children who have severe behavioral disturbances, with
a differential outcome based on the context within which the ratings were generated.
Parent‘s observations reflected a significant improvement in behaviour with teachers
observing a significant deterioration of behavior. Results also suggest that the CGI is
sensitive to changes in medication titration, reflecting that behaviour changes do occur in
a predictable / explainable direction within the medication trials.
References
Tucha, O., Prell, S., Mecklinger, L, Bormann- Kischkel, C., Sabine, K., Linder, M.,
Walitza, S. & Lange, K. (2006). Effects of methylphenidate on multiple
components of attention in children with attention deficit / hyperactivity
disorder. Psychopharmacology, 185, 315-326.
CPRI-UWO 2008
93
AN INVESTIGATION OF THE RELATIONSHIP BETWEEN ACADEMIC
ACHIEVEMENT AND EMOTIONAL AND BEHAVIOURAL DISORDERS IN A
RESIDENTIAL POPULATION
Aarynn S. King
INTRODUCTION: Children and youth who access intensive mental health treatment
present with challenges across numerous domains including both behaviour and learning
problems. Gresham, Lane, and MacMillan (1999) found that students with internalizing
and externalizing disorders failed to meet the social and behavioural standards and
expectations of their teachers, resulting in teacher rejection and referral to special
education. It is of interest whether teacher observations and subsequent ratings of
academic achievement may be influenced by the negative classroom behavior a student
with externalizing behavioural disorders display.
In other words, do the extreme behavioural deficits of children/youth interfere with
teachers accurately judging the academic competence of these students?
METHOD: The clinical charts of 202 children/youth consisting of 48 females and 154
males at the Child and Parent Resource Institute (CPRI) were reviewed in order to obtain
parent and teacher reports on standardized measures, diagnostic profiles, and academic
test scores.
Specifically, the present archival review assessed whether students with lower Conner‘s
teacher rating of behavioral problems would obtain higher (better) standardized academic
test scores on Woodcock-Johnson Tests of Achievement (WJIII). Also of interest was the
level of correspondence between teacher ratings of academic achievement on the
Conner‘s and Social Skills Rating System (SSRQ) and the students‘ actual academic
achievement on the WJIII in a clinical population.
RESULTS: Results indicate that students with behavior problems struggle in both
reading and writing, but higher academic achievement did not lead to significantly
improved Conner‘s teacher ratings of oppositional behavior. Teacher ratings of academic
achievement on the Conner‘s and SSRQ were significantly related to students actual
academic skills.
DISCUSSION: These findings illustrate the usefulness of teacher ratings of clinical
student‘s achievement, which may be used as an alternative to extensive WJIII and IQ
testing.
References
Gresham, F. M., Lane, K. L., MacMillan, D. L. (1999). Social and academic profiles of
externalizing and internalizing groups: Risk factors for Emotional and Behavioral
Disorders. Behavioral Disorders, 24, 231-245.
CPRI-UWO 2008
Appendix E
Capacity Building: Further Funded Research Based on the Centre of Excellence Grant
CPRI-UWO 2008
94
95
Investigating the Impact of Intensive Residential Treatment on Children Referred
from the Child Welfare System: The Nature of Maltreatment
Shannon L. Stewart, PhD
Child and Parent Resource Institute
and
Alan Leschied, PhD
Funder: Canadian Institutes for Health Research
London Ontario CANADA 2008-2009
Currently, funding through the Centre of Excellence in Children‘s Mental Health
(CHEO) is providing support to investigate the impact of intensive residential treatment
on children and youth referred from a 17 county catchment area in southwest Ontario.
This study is unique in several ways. First, it examines follow–up at three time periods
for children and youth who spend, on average, four months in an intensive residential
treatment program. These follow-ups are; immediately upon discharge, six months
following discharge and again at two years. While previous research, most notably by
John Lyons and his colleagues, suggests that such intensive intervention can show a short
term effect, it is the on-going generalization of treatment gains over time which is of
concern in this study (St. Pierre & Leschied, 2006). Second, it uses a standard set of two
measures, the aforementioned BCFPI and the Child and Family Functional Assessment
Scale (CAFAS). Early analysis of our follow-up data suggests that treatment gains are
significant at discharge reflected in a lessening of symptomatology as measured on the
Brief Child and Family Phone Interview (BCFPI), and continue to show improvement
over and above the levels of distress measured again at the two year follow-up compared
to intake levels. (St. Pierre, Cook, Leschied, Cullion, Johnson & Stewart, 2007).
We are however aware that considerable within group variation in symptom
improvement within our target group is being accounted for by a variety of factors.
Among the more clinically, systemically and theoretically relevant are those children and
youth who come into residential treatment having already been found victimized through
one or more forms of maltreatment. The current sample in our follow-up study includes a
substantial number of CAS to treatment referrals that have been priorized and vetted
through the region‘s single point access system. These cases then will have been
investigated and verified as maltreated, provided service within the child welfare system,
typically through traditional foster, group or outside paid institutional placements. It is
the breakdown or inability of these child welfare-based services that in turn trigger the
need for, what is arguably, the most intensive and expensive form of service provided
within the children‘s service delivery system, namely in patient residential treatment. It
is the impact of this intensive treatment for children and youth who have previously
experienced maltreatment and services within the child welfare system who are the focus
for this proposal.
CPRI-UWO 2008
96
Currently, child and family – based data for the current study is restricted to that
which is either on-file at the time of referral or provided following the child or youth‘s
admission to CPRI. Follow-up evaluations of the child‘s progress are made using both the
BCFPI and the CAFAS. However, critical information related to the nature and degree of
maltreatment, the family‘s history within mental health and child welfare services and
previous treatments attempted is lacking. As well, the percentage completion of followups with the CAS subsample is less than that which is collected with the children and
youth who are placed back with their families‘ of origin.
There are two aspects to this proposal. The first is to provide greater detail with
respect to who the children and youth are who come into intensive residential treatment
from the child welfare system. We propose to utilize the Child Welfare Data Retrieval
Instrument (CWDRI) (Leschied, Hurly, Chiodo and Whitehead, 2003) to gain greater
detail on these cases by accessing the child welfare file from the referring child welfare
system. The CWDRI is a structured series of items that guide data extraction from child
welfare files. It has been used in a number of previous studies by the second author in
identifying relevant child welfare data (Leschied, Chiodo, Whitehead, Hurley, D. &
Marshall, L, 2003; Whitehead, Chiodo, Leschied, & Hurley, 2004a; Leschied,
Whitehead, Hurley, & Chiodo, 2004b). It also has imbedded within the instrument, items
to the Ontario Eligibility Spectrum Code which identifies previous forms and degree of
maltreatment along with other family-related difficulties (Ontario Ministry of Children
and Youth‘s Services, 2000). The second aspect of this proposal is to increase our
numbers from the child welfare sample at the two year follow up period.
Timeline
January 2008 – Ethics completed and submitted to UWO ethics committee.
- Research assistant hired and trained on Eligibility Spectrum Code and
Data Retrieval Instrument
- Children‘s Aid Societies contacted and engaged in project
February 2008 – data collection begins
August 2008 – data collected, entered into larger data set
September – December 2008- data analyzed, update on literature provided; write-up of
final report
February 2009 – Final report submitted
Budget Justification
Research Assistant: Two part-time research assistants are required to collect and code data,
conduct casebook reviews, format, enter and check data into a larger data set. Research
assistances will also assist with literature searches, literature reviews, data cleaning and
preliminary analyses, report writing.
CPRI-UWO 2008
97
2 Research Assistants at 13 hours per week for 52 weeks at a rate of pay of $17.00 per hour =
$22,984.
Data Analyst
Provide statistical consultation, conduct data analyses, etc.
Estimated cost: $5000
Travel: Travel costs directly related to research assistant travel to and from Children Aids
Societies across the South West Region. $2000
Total: $29,984
References
Ontario Ministry of Children and Youth‘s Services (2000). Risk assessment model for
child protection in Ontario. Toronto ON.
Leschied, A.W., Whitehead, P.C., Hurley, D. & Chiodo, D. (2004b). Protecting children
is everybody‘s business. Ontario Association of Children’s Aid Society
Journal, 47, 3, 10-16.
Leschied, A.W., Chiodo, D., Whitehead, P.W., Hurley, D. and Marshall, L. (2003) The
Empirical Basis of Risk Assessment in Child Welfare: Assessing the Concurrent
and Predictive Accuracy of Risk Assessment and Clinical Judgment. Child
Welfare, 32, 527-540.
Whitehead, P.C., Chiodo, D.. Leschied, A.W., & Hurley, D (2004a). Referrals and
Admissions to the Children‘s Aid Society: A Test of Four Hypotheses. Child and
Youth Care Forum, 33, 6, 425-440.
St. Pierre, J. & Leschied, A. (2006). Situating the role of residential treatment for high
needs, high risk children and youth: Evaluating outcomes and service utilization.
Proposal supported by the Centre of Excellence in Children‘s Mental Health.
Ottawa, ON.
St. Pierre, J., Cook, S., Leschied, A., Cullion, C., Johnson, A., & Stewart, S. (2007). How
can data collection and analysis choices aid outcome evaluations. University of
South Florida Systems of Care Conference (Submitted).
CPRI-UWO 2008
98
Short-Term Residential Treatment: Effectiveness of Intensive Intervention for
Children with Complex Mental Health Needs
Dr. Shannon Stewart (P.I.), Centre of Excellence Program Evaluation Grant, 2007
Data is collected on children and youth participating in CPRI‘s residential program on an
ongoing basis. The database housing various outcomes measures is being used to assess
the effectiveness of different types of residential services within our institution. For
example, a recent study compared the outcomes of children/youth in longer-term (3.5-7
months) vs. short-term (4-6 weeks) residential care. Findings suggested that both the
short- and longer-term residential treatment had a significant, positive impact. Parents
and clinicians reported major improvements with respect to overall severity of child
dysfunction and externalizing problems, with superior outcomes for longer stays. At 6month follow-up, results suggested that both the shorter and longer term stays were
effective but the effect sizes still remain higher for longer term stays. However, these
results were not statistically significant in the small sample size analyzed. Additionally,
client satisfaction was high regardless of type of stay.
Plans are also underway to use this outcome data to evaluate the effectiveness of a new,
innovative service delivery for younger male residential clients. In January 2007, CPRI
opened a 9-bed capacity unit for assessment/ treatment of boys aged 9-12 who could not
be served in less restrictive settings due to complex, high risk needs. This unit has been
influenced by the assessment/treatment philosophy and approach called Collaborative
Problem-SolvingSM (CPS) originated by Dr. Ross Greene (see www.thinkkids.org). CPS
emphasizes transactional or reciprocal processes between parents and children (with the
goal of improving compatibility between children and adults) and offers a detailed
formulation about child characteristics or cognitive factors that lead to explosive
behaviour. These cognitive factors are grouped in 5 clusters of skills that are needed for
the development of flexibility, frustration tolerance and problem solving: executive skills,
language processing skills, emotion regulation skills, cognitive flexibility skills and social
skills (Greene & Ablon, 2006). Outcome data that reflects these program goals will be
compared for those residential clients who receive CPS versus the previous treatment.
Finally, the data collected about residential clients is used by clinical staff, through
incorporation of data in clinical assessments of clients and subsequent use for treatment
planning, and for assessing aspects of care such as parent satisfaction.
CPRI-UWO 2008
99
Diversion from Inpatient Child Psychiatry
Dr. Alan Leschied and Dr. Jeff St. Pierre
UWO Internal SSHRC Research Grant Application
Fall 2008 (announcement expected late fall)
A small percentage of children and youth who experience mental illness, severe
behaviour problems, or familial abuse enter into expensive hospital and voluntary out-ofhome psychiatric treatments. These children are screened to ensure only those with the
most severe presenting problems access the inpatient beds. Our own research and a small
literature indicate treatment benefits within ongoing needs to access multiple mental
health, corrections, child welfare and special education services. Given the involvement
of multiple professionals, and the expense of inpatient services, it is striking how little we
know about who benefits, and what happens to those who are eligible but do not access
services. The present investigation proposes to look at a unique population: children and
youth referred and accepted to a psychiatric inpatient unit, but not admitted despite their
severe presentation. The goal of this study is to review the archival record, contact these
individuals, appreciate why they did not enter inpatient psychiatric treatment, and
identify what services they did receive and their status with respect to their current
psychiatric symptoms and overall adaptive functioning. This presentation will be
compared to our long term follow-up data on those who did enter psychiatric inpatient
service.
CPRI-UWO 2008
Appendix F
Advisory Committee
CPRI-UWO 2008
100
101
Members of the Advisory Committee –
Child and Parent Resource Institute Follow-up Study on the Impact of Residential
Treatment on High Risk, High Needs Children
(Funded through the Centre of Excellence in Children’s Mental Health, The
Children’s Hospital of Eastern Ontario. Ottawa Ontario)
Community Members
Sharon Skutovich—Consumer (Parent) Representative
Larry Langan, Director of Education, Huron Catholic Board of Education
Leo Massi, Executive Director, Haldimand Norfolk REACH,
Steve Novosedlik, Executive Director, Contact Niagara- Child and Youth Services
Rod Potgeiter, Executive Director, St. Thomas and Elgin Children‘s Aid Society
Liz Prendergast, Executive Director Children‘s Services Coordination Network – London
Rick Shields, Executive Directory, St. Clair Child and Youth Services,
Dr. Margaret Steele CHWO – London
Child and Parent Resource Institute Members
Ann Stark, Executive Director
Peter Ferguson, Principal, Madeline Hardy School
Ex Officio Members and Consultants
Dr. Melanie Barwick (CAFAS Ontario),
Dr. Peter Pettingill (BCFPI)
CPRI-UWO 2008
102
References
Abidin, R. R. (1995). Parenting Stress Index: Third Edition: Psychological Assessment
Resources, Inc.
Ainsworth, F., & Hansen, P. (2005). A dream come true-no more residential care. A
corrective note. International Journal of Social Welfare, 14(3), 195-199.
Albano, A. M., Chorpita, B. F., & Barlow, D. H. (2003). Childhood anxiety disorders. In
E. Mash & R. Barkley (Eds.), Child psychopathology (pp. 279-329). New York:
Guilford Press.
APA Task Force on Evidence-Based Practice for Children and Adolescents. (2008).
Disseminating Evidence-Based Practice for Children and Adolescents: A Systems
Approach to Enhancing Care. Washington, DC: American Psychological
Association.
Barnes, L. L. B., & Oehler-Stinnett, J. J. (1998). Review of the Parenting Stress Index,
third edition. In C. Impara & B. S. Plake (Eds.), The thirteenth mental
measurements yearbook. . Lincoln, NE: Buros Institute of Mental Measurements.
Barth, R. P. (2005). Residential care: From here to eternity. International Journal of
Social Welfare, 14, 158-162.
Bastiaansen, D., Koot, H. M., & Ferdinand, R. F. (2005). Psychopathology in children:
Improvement of quality of life without psychiatric symptom reduction? European
Child & Adolescent Psychiatry, 14(7), 364-370.
Bickman, L., Foster, E. M., & Lambert, E. W. (1996). Who gets hospitalized in a
continuum of care. Journal of the American Academy of Child & Adolescent
Psychiatry, 35(1), 74-80.
Birmaher, B., & Axelson, D. (2006). Course and outcome of bipolar spectrum disorder in
children and adolescents: A review of the existing literature. Development and
Psychopathology, 18(4), 1023-1035.
Birmaher, B., Williamson, D. E., Dahl, R. E., Axelson, D. A., Kaufman, J., Dorn, L. D.,
et al. (2004). Clinical presentation and course of depression in youth: Does onset
in childhood differ from onset in adolescence? Journal of the American Academy
of Child & Adolescent Psychiatry, 43(1), 63-70.
Boyle, M. H., Cunningham, C. E., Georgiades, K., Cullen, J., Racine, Y., & Pettingill, P.
(in press). The Brief Child and Family Phone Interview (BCFPI): 2. Usefulness in
screening for child and adolescent psychopathology. The Journal of Child
Psychology and Psychiatry.
Brady, K. L., & Caraway, S. J. (2002). Home away from home: Factors associated with
current functioning in children living in a residential treatment setting. Child
Abuse & Neglect, 26(11), 1149-1163.
Briggs-Gowan, M. J., Owens, P. L., Schwab-Stone, M. E., Leventhal, J. M., Leaf, P. J., &
HoRwitz, S. M. (2003). Persistence of psychiatric disorders in pediatric settings.
Journal of the American Academy of Child & Adolescent Psychiatry, 42(11),
1360-1369.
Broidy, L. M., Nagin, D. S., Tremblay, R. E., Bates, J. E., Brame, B., Dodge, K. A., et al.
(2003). Developmental trajectories of childhood disruptive behaviors and
adolescent delinquency:A six-site, cross-national study. Developmental
Psychology, 39, 222–245.
CPRI-UWO 2008
103
Burke, J. D., Loeber, R., & Birmaher, B. (2002). Oppositional defiant disorder and
conduct disorder: A review of the past 10 years, part II. Journal of the American
Academy of Child & Adolescent Psychiatry, 41(11), 1275-1293.
Carlson, K. D., & Schmidt, F. L. (1999). Impact of experimental design on effect size:
Findings from the research literature on training. Journal of Applied Psychology,
84(6), 851-862.
Caspi, A. (2000). The child Is father of the man: Personality continuities from childhood
to adulthood. Journal of Personality and Social Psychology, 78(1), 158-172.
Chamberlain, P., & Smith, D. K. (2003). Antisocial behavior in children and adolescents:
The Oregon Multidimensional Treatment Foster Care Model. In A. E. Kazdin & J.
R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents
(pp. 282-300). New York: Guilford.
Conners, C. K. (1997). Conners' Rating Scales-Revised. Toronto: Multi-Health Systems.
Connor, D. F., Miller, K. P., Cunningham, J. A., & Melloni, R. H. (2002). What does
getting better mean? Child improvement and measure of outcome in residential
treatment. American Journal of Orthopsychiatry, 72(1), 110-1177.
Cunningham, C. E., Boyle, M. H., Hong, S., Pettingill, P., & Bohaychuk, D. (in press).
The Brief Child and Family Phone Interview (BCFPI) 1: Rationale, development,
and description of computerized children's mental health intake and outcome
assessment tool. The Journal of Child Psychology and Psychiatry.
Cunningham, C. E., Pettingill, P., & Boyle, M. (2004). The Brief Child and Family Phone
Interview (BCFPI-3): A computerized intake and outcome assessment tool.
Interviewer's manual. Retrieved February 7, 2005 from
http://www.bcfpi.ca/bcfpi/downloads.html.
Curry, J. F. (1991). Outcome research on residential treatment: Implications and
suggested directions. American Journal of Orthopsychiatry, 61(3), 348-357.
Curry, J. F. (2004). Future directions in residential treatment outcome research. Child and
Adolescent Psychiatric Clinics of North America, 13(2), 429-440.
Dodge, K. A., & Pettit, G. S. (2003). A biopsychosocial model of the development of
chronic conduct problems in adolescence. Developmental psychology, 39(2), 349371.
Duchnowski, A., Hall, K., Kutash, K., & Friedman, R. (1998). The alternatives to
residential treatment study. In M. H. Epstein, K. Kutash & A. J. Duchnowski
(Eds.), Outcomes for children and youth with emotional and behavioral disorders
and their families: Programs and evaluation best practices (pp. 55–80). Austin,
TX: Pro-ed.
Epstein, M. H., Kutash, K., & Duchnowski, A. J. (Eds.). (2005). Outcomes for children
with emotional and behavioral disorders and their families: Program and
evaluation best practices (2nd ed.). Austin: TX: Pro-Ed.
Fergusson, D. M., & Horwood, J. L. (1995). Predictive validity of categorically and
dimensionally scored measures of disruptive behavioural adjustment and juvenile
offending. Journal of Abnormal Child Psychology, 34(4), 477-487.
Fergusson, D. M., Horwood, J. L., & Ridder, E. M. (2005a). Show me the child at seven
II: Childhood intelligence and later outcomes in adolescence and young
adulthood. Journal of Child Psychology and Psychiatry, 46(8), 850-858.
CPRI-UWO 2008
104
Fergusson, D. M., Horwood, J. L., & Ridder, E. M. (2005b). Show me the child at seven:
The consequences of conduct problems in childhood for psychosocial functioning
in adulthood. Journal of Child Psychology and Psychiatry, 46(8), 837-849.
Foster, E. M., & Kalil, A. (2008). New methods for new questions: Obstacles and
opportunities. Developmental Psychology, 44(2), 301-304.
Frensch, K. M., & Cameron, G. (2002). Treatment of choice or a last resort? A review of
residential mental health placements for children and youth. Child and Youth
Care Forum, 31(5), 307-339.
Gianarris, W. J., Golden, C. J., & Greene, L. (2001). The conners' parent rating scales: A
critical review of the literature. Clin Psychol Rev, 21(7), 1061-1093.
Glassberg, L. A. (1994). Students with behavioral disorders: Determinants of placement
outcomes. Behavioral Disorders, 19, 181-191.
Goodyer, I., Psych, F. R. C., Herbert, J. O. E., Tamplin, A., Secher, S. M., & Pearson, J.
(1997). Short-term outcome of major depression: II. Life events, family
dysfunction, and friendship difficulties as predictors of persistent disorder.
Journal of the American Academy of Child & Adolescent Psychiatry, 36(4), 474480.
Gorske, T. T., Srebalus, D. J., & Walls, R. T. (2003). Adolescents in residential centers:
Characteristics and treatment outcome. Children and Youth Services Review,
25(4), 317-326.
Green, J., Jacobs, B., Beecham, J., Dunn, G., Kroll, L., Tobias, C., et al. (2007). Inpatient
treatment in child and adolescent psychiatry-a prospective study of health gain
and costs. Journal of Child Psychology and Psychiatry, 48(12), 1259-1267.
Gresham, F. M., & Elliott, S. N. (1990). Social Skills Rating System: Manual. Circle
Pines, MN: American Guidance Service Inc.
Hair, H. (2005). Outcomes for children and adolescents after residential treatment: A
review of research from 1993 to 2003. Journal of Child and Family Studies,
14(4), 551-575.
Halliday-Boykins, C. A., Henggeler, S. W., Rowland, M. D., & DeLucia, C. (2004).
Heterogeneity in youth symptom trajectories following psychiatric crisis:
Predictors and placement outcome. Journal of Consulting and Clinical
Psychology, 72(6), 993-1003.
Hammen, C., & Rudolph, K. (2003). Childhood Mood Disorders. In E. Mash & R.
Barkley (Eds.), Child psychopatology (pp. 233-278). New York: Guilford Press.
Handwerk, M. L., Larzelere, R. E., Soper, S. H., & Friman, P. C. (1999). Parent and child
discrepancies in reporting severity of problem behaviors in three out-of-home
settings. Psychological assessment, 11(1), 14-23.
Harder, A. T., Knorth, E. J., & Zandberg, T. (2006). Residential child and youth care in
the picture. Amsterdam: SWP Publishers.
Haviland, A., Nagin, D. S., Rosenbaum, P. R., & Tremblay, R. E. (2008). Combining
group-based trajectory modeling and propensity score matching for causal
inferences in nonexperimental longitudinal data. Developmental Psychology,
44(2), 422.
Heijmens Visser, J., Koot, H. M., & Verhulst, F. C. (1999). Continuity of
psychopathology in youths referred to mental health services. Journal of the
American Academy of Child & Adolescent Psychiatry, 38(12), 1560-1568.
CPRI-UWO 2008
105
Heijmens Visser, J., Van der Ende, J., Koot, H. M., & Verhulst, F. C. (2003). Predicting
change in psychopathology in youth referred to mental health services in
childhood or adolescence. Journal of Child Psychology and Psychiatry, 44(4),
509-519.
Helgerson, B. S., Martinovich, Z., Durkin, E., & Lyons, J. S. (2005). Differences in
outcome trajectories of children in residential treatment. Residential Treatment for
Children and Youth, 22(4), 67-79.
Henggeler, S. W., Rowland, M. D., Halliday-Boykins, C., Sheidow, A. J., Ward, D. M.,
Randall, J., et al. (2003). One-year follow-up of multisystemic therapy as an
alternative to the hospitalization of youths in psychiatric crisis. Journal of the
American Academy of Child & Adolescent Psychiatry, 42(5), 543.
Henggeler, S. W., Rowland, M. D., Randall, J., Ward, D. M., Pickrel, S. G., Cunningham,
P. B., et al. (1999). Home-based multisystemic therapy as an alternative to the
hospitalization of youths in psychiatric crisis: Clinical outcomes. Journal of the
American Academy of Child & Adolescent Psychiatry, 38(11), 1331-1339.
Hill, J. (2002). Biological, psychological and social processes in the conduct disorders.
Journal of Child Psychology and Psychiatry, 43(1), 133-164.
Hodges, K. (2000). Child and Adolescent Functional Assessment Scale (2nd Rev. ed.).
Ypsilanti, MI: Eastern Michigan University.
Hodges, K., & Wotring, J. (2004). The role of monitoring outcomes in initiating
implementation of evidence-based treatments at the state level. Psychiatric
Services, 55, 396-400.
Hodges, K., Xue, Y., & Wotring, J. (2004). Use of the CAFAS to evaluate outcome for
youths with severe emotional disturbance served by public mental health. Journal
of Child and Family Studies, 13(3), 325-339.
Hofstra, M. B., Van der Ende, J. M. S., & Verhulst, F. C. (2000). Continuity and change
in psychopathology from childhood to adulthood: A 14-year follow-up study.
Journal of the American Academy of Child and Adolescent Psychiatry, 39, 850–
858.
Hurley, S., & Goldsmith, T. (2007). Outcomes for youth receiving intensive in-home
therapy or residential care: A comparison using propensity scores. Paper
presented at the 20th Annual Research Conference – A System of Care for
Children‘s Mental Health: Expanding the Research Base.
Hussey, D. L., & Guo, S. (2002). Profile characteristics and behavioral change
trajectories of young residential children. Journal of Child and Family Studies,
11(4), 401-410.
Jensen, P. S. (2003). Comorbidity and child psychopathology: Recommendations for the
next decade. Journal of Abnormal Child Psychology, 31(3).
Kazdin, A. E. (2006). Arbitrary metrics: Implications for identifying evidence-based
treatments. American Psychologist, 61(1), 42-49.
Kazdin, A. E., & Weisz, J. R. (1998). Identifying and developing empirically supported
child and adolescent treatments: Empirically supported psychological therapies.
Journal of consulting and clinical psychology, 66(1), 19-36.
Keiley, M. K., Lofthouse, N., Bates, J. E., Dodge, K. A., & Pettit, G. S. (2003).
Differential risks of covarying and pure components in mother and teacher reports
CPRI-UWO 2008
106
of externalizing and internalizing behavior across ages 5 to 14. Journal of
Abnormal Child Psychology, 31(3), 267-283.
Kendrick, A. J. (Ed.). (2007). Residential child care: prospects and challenges. London:
Jessica Kingsley Publishers.
Kim-Cohen, J., Caspi, A., Moffitt, T. E., Harrington, H. L., Milne, B. J., & Poulton, R.
(2003). Prior juvenile diagnoses in adults with mental disorder: Developmental
follow-back of a prospective-longitudinal cohort. Archives of General Psychiatry,
60(7), 709-717.
King, G., Tucker, M. A., Baldwin, P., Lowry, K., LaPorta, J., & Martens, L. (2002). A
life needs model of pediatric service delivery: Services to support community
participation and quality of life for children and youth with disabilities. Physical
and Occupational Therapy in Pediatrics 22(2), 53-78.
Knorth, E. J., Harder, A. T., Zandberg, T., & Kendrick, A. J. (2008). Under one roof: A
review and selective meta-analysis on the outcomes of residential child and youth
care. Children and Youth Services Review, 30(2), 123-140.
Koot, H. M. (1995). Longitudinal studies of general population and community samples.
In F. C. Verhulst & H. M. Koots (Eds.), The epidemiology of child and adolescent
psychopathology (pp. 337–365). New York: Oxford University Press.
Lavigne, J. V., Arend, R., Rosenbaum, D., Binns, H. J., Christoffel, K. K., & Gibbons, R.
D. (1998). Psychiatric disorders with onset in the preschool years: I. Stability of
diagnoses. Journal of the American Academy of Child & Adolescent Psychiatry,
37(12), 1246-1254.
Leon, S. C., Lyons, J. S., Uziel-Miller, N. D., Rawal, P., Tracy, P., & Williams, J. (2000).
Evaluating the use of psychiatric hospitalization by residential treatment centers.
Journal of the American Academy of Child & Adolescent Psychiatry, 39(12),
1496-1501.
Leschied, A. W., Chiodo, D., Nowicki, E., & Rodger, S. (2007). Childhood predictors of
adult criminality: A meta-analysis drawn from the prospective longitudinal
literature.
Libby, A. M., Coen, A. S., Price, D. A., Silverman, K., & Orton, H. D. (2005). Inside the
black box: What constitutes a day in a residential treatment centre? International
Journal of Social Welfare, 14(3), 176-183.
Lilienfeld, S. O. (2003). Comorbidity between and within childhood externalizing and
internalizing disorders: Reflections and directions. Journal of Abnormal Child
Psychology, 31(3), 285-291.
Loeber, R., Green, S. M., Lahey, B. B., Christ, M. A. G., & Frick, P. J. (1992).
Developmental sequences in the age of onset of disruptive child behaviors.
Journal of Child and Family Studies, 1(1), 21-41.
Lyons, J. S. (2004). Redressing the emperor: Improving our children's public mental
health system. Westport, CT: Praeger.
Lyons, J. S., & McCulloch, J. R. (2006). Monitoring and managing outcomes in
residential treatment: Practice-based evidence in search of evidence-based
practice. American Academy of Child and Adolescent Psychiatry, 45(2), 247-251.
Lyons, J. S., Terry, P., Martinovich, Z., Peterson, J., & Bouska, B. (2001). Outcome
trajectories for adolescents in residential treatment: A statewide evaluation.
Journal of Child and Family Studies, 10(3), 333-345.
CPRI-UWO 2008
107
Mash, E. J., & Dozois, D. J. A. (2003). Child psychopathology: A developmentalsystems perspective. In E. Mash & R. Barkley (Eds.), Child psychopathology (2nd
ed., pp. 3–71). New York: Guilford Press.
Moffitt, T. E., & Caspi, A. (2001). Childhood predictors differentiate life-course
persistent and adolescence-limited antisocial pathways among males and females.
Development and Psychopathology, 13(2), 355-375.
Moffitt, T. E., Caspi, A., Harrington, H., & Milne, B. J. (2002). Males on the life-coursepersistent and adolescence-limited antisocial pathways: Follow-up at age 26
years. Development and Psychopathology, 14(1), 179-207.
Mun, E. Y., von Eye, A., Bates, M. E., & Vaschillo, E. G. (2008). Finding groups using
model-based cluster analysis: Heterogeneous emotional self-regulatory processes
and heavy alcohol use risk. Developmental Psychology, 44(2), 481-495.
Nelson, A., Stage, S., Duppong-Hurley, K., Synhorst, L., & Epstein, M. H. (2007). Risk
factors predictive of the problem behavior of children at risk for emotional and
behavioral disorders. Exceptional Children, 73(3), 367-379.
Oosterveen, J. (2008). Client satisfaction with mental health services: A qualitative
approach to understanding the family's experience of residential care. The
University of Western Ontario, London, ON.
Phillips, S. D., Hargis, M. B., Kramer, T. L., Lensing, S. Y., Taylor, J. L., Burns, B. J., et
al. (2000). Toward a level playing field: Predictive factors for the outcomes of
mental health treatment for adolescents. Journal of the American Academy of
Child & Adolescent Psychiatry, 39(12), 1485-1495.
Reid, G. J., Stewart, S. L., Barwick, M., Cunningham, C., Carter, J., Evans, B., et al.
(2008-2010). Exploring patterns of service utilization within children's mental
health agencies (2 years). Grant from The Provincial Centre of Excellence for
Child and Youth Mental Health at CHEO. $148,549
Roeder, K., Lynch, K. G., & Nagin, D. S. (1999). Modeling uncertainty in latent class
membership: A case study in criminology. Journal of the American Statistical
Association, 94(447), 766-776.
Rosenblatt, A., & Furlong, M. J. (1998). Outcomes in a system of care for youths with
emotional and behavioral disorders: An examination of differential change across
clinical profiles. Journal of Child and Family Studies, 7(2), 217-232.
Rosenblatt, A., & Rosenblatt, J. A. (2002). Assessing the effectiveness of care for youth
with severe emotional disturbances: Is there agreement between popular outcome
measures? The Journal of Behavioral Health Services & Research, 29(3), 259273.
Roy, J. (2007). Latent class models and their application to missing-data patterns in
longitudinal studies. Statistical Methods in Medical Research, 16(5), 441.
Rubin, K. H., Burgess, K., Kennedy, A. E., & Stewart, S. L. (2003). Social withdrawal
and inhibition in childhood. In Child psychopathology (2nd ed., pp. 372-406).
New York: Guilford.
Rubin, K. H., & Stewart, S. L. (1996). Social withdrawal in childhood. In E. Mash & R.
Barkley (Eds.), Child psychopathology (pp. 277–307). New York: Guilford Press.
Rubin, K. H., Stewart, S. L., & Chen, X. (1995). The parents of aggressive and
withdrawn children. In M. Bornstein (Ed.), Handbook of parenting (Vol. 1, pp.
255-284). New Jersey: Lawrence Erlbaum Associates, Inc.
CPRI-UWO 2008
108
Rubin, W. V., & Panzano, P. C. (2002). Identifying meaningful subgroups of adults with
severe mental illness. Psychiatric Services, 53(4), 452-457.
Rutter, M. (2005). Multiple meanings of a developmental perspective on
psychopathology. European Journal of Developmental Psychology, 2(3), 221252.
Sheras, P. L., Abidin, R. R., & Konold, T. R. (1998). Stress Index for Parents of
Adolescents: Professional Manual. Odessa, FL: Psychological Assessment
Resources.
Singer, J., & Willett, J. (2003). Applied longitudinal data analysis: Modeling change and
event occurrence: Oxford University Press, USA.
Snowden, J. A., Leon, S. C., Bryant, F. B., & Lyons, J. S. (2007). Evaluating psychiatric
hospital admission decisions for children in foster care: An optimal classification
tree analysis. Journal of Clinical Child and Adolescent Psychology, 36(1), 8-18.
Sourander, A., Helenius, H., Leijala, H., Heikkilä, T., Bergroth, L., & Piha, J. (1996).
Predictors of outcome of short-term child psychiatric inpatient treatment.
European Child & Adolescent Psychiatry, 5(2), 75-82.
Speltz, M. L., McClellan, J. O. N., DeKlyen, M., & Jones, K. (1999). Preschool boys
with oppositional defiant disorder: Clinical presentation and diagnostic change.
Journal of the American Academy of Child & Adolescent Psychiatry, 38(7), 838845.
Stanger, C., MacDonald, V. V., McConaughy, S. H., & Achenbach, T. M. (1996).
Predictors of cross-informant syndromes among children and youths referred for
mental health services. Journal of Abnormal Child Psychology, 24(5), 597-614.
Sterba, S. K., Prinstein, M. J., & Cox, M. J. (2007). Trajectories of internalizing problems
across childhood: Heterogeneity, external validity, and gender differences.
Development and Psychopathology, 19(2), 345-366.
Tremblay, R. E., Masse, B., Perron, D., Leblanc, M., Schwartzman, A. E., & Ledingham,
J. E. (1992). Early disruptive behavior, poor school achievement, delinquent
behavior, and delinquent personality: Longitudinal analyses. Journal of
Consulting and Clinical Psychology, 60(1), 64-72.
Vermunt, J. K., & Magidson, J. (2005). Latent gold 4.0 user's guide. Belmont, MA:
Statistical Innovations Inc.
Weisz, J. R., & Gray, J. S. (2007). Evidence-based psychotherapy for children and
adolescents: Data from the present and a model for the future. Child and
Adolescent Mental Health, 1-10.
Wells, K. (1991). Placement of emotionally disturbed children in residential treatment: A
review of placement criteria. American Journal of Orthopsychiatry, 61(3), 339347.
Whittaker, J. K., & Maluccio, A. N. (2002). Rethinking'child placement': A reflective
essay. Social Service Review, 76(1), 108-134.
Wilmshurst, L. A. (2002). Treatment programs for youth with emotional and behavioral
disorders: An outcome study of two alternate approaches. Mental Health Services
Research, 4(2), 85-96.
Youngstrom, E. A., Findling, R. L., & Calabrese, J. R. (2003). Who are the comorbid
adolescents? Agreement between psychiatric diagnosis, youth, parent, and teacher
report. Journal of Abnormal Child Psychology, 31(3), 231-245.
CPRI-UWO 2008
Zeitlin, H. (1986). The natural history of psychiatric disorders in children. England:
Oxford University Press.
CPRI-UWO 2008
109

Three year outcomes in tertiary care

Transcription

Similar documents

Open living room

Get the pdf HERE.

Report for Parkview Residential Hot2000 Modelling

Residential Fire Safety Kits

Floodwatch®

Specialist Residential Risks

REAL PROPERTY TAX RATES IN HAWAII

multifamily development and investment

Brotoloc South Brochure

Miraco – Carrier Laboratories