Bonnie CLAC Year-End Report

Transcription

COMPREHENSIVE EVALUATION
OF
MORE THAN WHEELS
FINAL REPORT
Vol. 1. Narrative Report
Prepared for:
More Than Wheels
250 Commercial Street, Suite 4021
Manchester, NH 03101
Prepared by:
Barbara Wauchope, Ph.D., Director of Evaluation
Eleanor Jaffee, M.S.W., Evaluation Research Associate
The Carsey Institute
University of New Hampshire
Durham, NH 03824
December 23, 2011
COMPREHENSIVE EVALUATION OF
MORE THAN WHEELS
FINAL REPORT
Contents
EXECUTIVE SUMMARY.................................................................................................................................. 1
Findings ............................................................................................................................................................. 2
Significant Changes in More Than Wheels Clients Over Time .................................................... 2
Other Findings Over Time ..................................................................................................................... 3
Evidence for Changes Attributable to More Than Wheels ........................................................... 4
Conclusions ....................................................................................................................................................... 5
BACKGROUND.................................................................................................................................................. 6
OUTCOME/IMPACT EVALUATION ............................................................................................................ 7
1. More Than Wheels Program .................................................................................................................. 7
1.1 Eligibility ............................................................................................................................................. 7
1.2 Programming ..................................................................................................................................... 7
1.3 Program Theory of Change ........................................................................................................... 8
2. Evaluation Questions and Methodological Approach ..................................................................10
3. Data Sources..............................................................................................................................................11
3.1. Client Survey Questionnaire ......................................................................................................11
3.2. Client Focus Groups ......................................................................................................................11
3.3. Financial Fitness Knowledge Test ............................................................................................12
3.4. Client Credit Information ............................................................................................................12
4. Evaluation Design and Analytical Approach ...................................................................................13
4.1. Longitudinal Outcome Evaluation............................................................................................13
4.2. Quasi-Experimental Impact Evaluation .................................................................................14
5. Description of Client Survey Samples ...............................................................................................16
5.1. Client Samples ................................................................................................................................16
5.2. Comparison Group Samples .......................................................................................................17
6. Findings: Longitudinal Outcomes and Program Impact .............................................................20
ii
6.1. Financial Management Outcomes ............................................................................................22
6.2 Health And Stress Outcomes.......................................................................................................32
6.3 Health-Access Related Outcomes ..............................................................................................43
6.4. Food Insecurity Outcomes ..........................................................................................................48
6.5. Transportation Outcomes...........................................................................................................52
6.6. Community Involvement Outcomes ........................................................................................57
SUMMARY AND CONCLUSIONS ..........................................................................................................61
Personal Money Management ............................................................................................................62
Health and Stress....................................................................................................................................63
Access to Health Care ............................................................................................................................64
Food Insecurity .......................................................................................................................................64
Transportation ........................................................................................................................................65
Community Involvement .....................................................................................................................66
Conclusions ..............................................................................................................................................66
iii
Acknowledgements
Every evaluation of this size and time commitment is a team effort and this one was no exception. The
authors of this report wish to acknowledge the support and assistance of the many people over the last
three years who helped us successfully complete this project and its final report.
For assistance with the design of the evaluation and its primary data collection instrument, the Client
Survey, our thanks go to Dr. Denise Anthony of the Department of Sociology at Dartmouth College. For
helping us through the ups and downs and the zigs and zags of our various data collection efforts - Mary
Burnett, Donelle Wright, Sherry Maher, Millie Barry and all the other More Than Wheels staff who
worked so hard to persuade clients to participate in the evaluation and to ensure that we had enough
data: we cannot thank you enough! For her patient and always thorough work with us to understand
and use the More Than Wheels databases and in the management of the Client Surveys, a very special
thanks to Kathy Menard of More Than Wheels. Our appreciation especially goes out to all the clients
who took the time to answer our questions. This evaluation would not have happened without them.
Thank you Martha Belanger, Tracy Keirns, and Chad Novak of the University of New Hampshire Survey
Center for taking on the evaluation telephone and online surveys. Our wonderful Graduate Research
Assistant Meghan Mills put in long hours to clean up, manage and analyze the very complicated
datasets, and Genevieve LaCon, Undergraduate Research Assistant, cheerfully entered, transcribed, and
analyzed our Client Satisfaction Survey questionnaires for two years – thank you both. Brenda Henry
and Celeste Torio of the Robert Wood Johnson Foundation provided invaluable advice during the design,
data collection and analysis phases of the evaluation.
Our special thanks to Terri Steingrebe of More Than Wheels who throughout this project always
demanded a high quality evaluation but also one that she could understand and use. Terri asked
important and often challenging questions, provided insights that contributed to some important
decisions about the evaluation, remained supportive throughout and, possibly most gratifying to us, was
really curious and interested in what we were doing and how she could help.
Finally, Dr. Sally Ward of the Sociology Department at the University of New Hampshire was responsible
for getting this project off the ground with an initial one year evaluation for More Than Wheels. She
helped to design this longer, more comprehensive evaluation, and provided ongoing support and advice
to project staff throughout the project. Thank you, Sally, for all the many ways that you contributed.
iv
In 2008 More Than Wheels1 contracted the Carsey Institute at the University of New Hampshire to
conduct a comprehensive process and outcome evaluation of its program. More Than Wheels is a
nonprofit organization that provides financial education and counseling and car buying assistance to
eligible clients interested in obtaining a low interest guaranteed car loan and the purchase of a new car.
The purpose of the study was to obtain a third party evaluation of the implementation, outcomes, and
impact of the program. This report presents the results of the analyses of almost three years of the
outcome/impact data collected from More Than Wheels clients in New Hampshire, Vermont, and
Massachusetts, and two similar comparison groups.
The purpose of the Carsey evaluation was to answer two questions: 1) Do participants in the More Than
Wheels program change over time, and 2) is there evidence suggesting that these changes are a result of
participation in the program, i.e., that the program has an effect on participants? Over the years
program staff had observed that not only did clients report improvements in their financial
circumstances that they attributed to the program and ownership of a new car, they also reported
positive health and social outcomes. These observations became the basis for the outcome areas
targeted for investigation by the evaluation: money management knowledge and behavior; mental and
physical health; utilization of health care, health insurance, and food assistance; employment and
income; and transportation-related activities, including interaction with family, friends and community.
The primary source of data was a survey administered to More Than Wheels clients at program
enrollment, completion, and six months or more after completion. Clients who completed the program
were compared for changes over time. The survey was also given to two comparison groups: a group of
people who contacted but never enrolled in the program, and a group of former clients who enrolled
but withdrew before finishing. Clients were compared to these two groups to see if they changed while
the comparison groups, with limited or no exposure to the program, remained the same. Credit scores
obtained from clients at two points in time, scores on a knowledge test taken by clients before and after
they completed the program’s Fin Fit financial literacy course, and anecdotal data obtained from several
focus groups conducted with former program clients provided data supplementing the surveys.
The key findings from these analyses are that
clients who completed the More Than Wheels program did change over time, improving their
knowledge and behaviors related to personal money management, increasing their credit
scores, and improving their perceived mental and physical health; and that
the evidence from comparing these clients to comparison groups suggests that these changes
can be attributed to their participation in the More Than Wheels program.
Based on these findings, it appears that More Than Wheels is a successful model for helping people
struggling with severe financial problems gain financial stability and control, improve their credit, and
obtain a new car, which can result in reduced stress and other positive changes to their quality of life.
1
At the time Carsey was contracted to conduct the evaluation, the More Than Wheels organization was called Bonnie CLAC (Car Loans And
Counseling). In 2010, the organization was renamed to More Than Wheels.
1
The following is a summary of the results of the evaluation for clients who completed the program. 2
Significant Changes in More Than Wheels Clients Over Time
Clients who completed the More Than Wheels program reported statistically significant improvements
in a number of areas3 while they were in the program, from time of enrollment to program completion,
and for a longer period of time, from enrollment to post-completion, approximately six months or more
after they finished the program. Outcomes reported after they left the program could be evidence of
lasting change. In analyses of the responses of clients matched over time, the evaluation found that
clients who completed the program:
1. Increased their knowledge of personal money management.
Fin Fit financial fitness course test scores increased by 55% between pre- and post-test.
2. Improved their money management behaviors.
On average, clients reported an increase in regular savings activity, by two-thirds (66%)
between enrollment and program completion.
Clients reporting that they usually or always paid their bills on time increased, on average, by
13% between enrollment and program completion and by 31% between enrollment and postcompletion.
3. Improved their financial credit scores.
Clients’ credit scores, obtained from national credit rating services, increased on average by
approximately 37 points, from an average of 596 to 633.
4. Gained improved mental health, including reduced stress and increased selfconfidence.
Clients reported a 52% decline in the number of days that their mental health was not good
during the previous month, from 6.6 at enrollment to 3.2 days at post-completion.
In seven specific questions about mental health and stress, clients who completed the program reported
statistically significant improvements between the time of enrollment in the program and postcompletion.
Findings for two of these questions are especially strong, with changes for both the time period of
enrollment to program completion and enrollment to post-completion.
On average, clients felt that things were going their way more often, increasing by 17%
between enrollment and program completion, and by 19% between enrollment and postcompletion.
On average, clients less often felt that difficulties were piling up so high that they could not
overcome them with a decline of 21% from enrollment to program completion, and of 26%
between enrollment and post-completion.
2
Note: Here and throughout the report, bulleted findings that are statistically significant are shown in bold font while bulleted findings that are
not significant are shown in regular font.
3
Results of statistical significance testing and calculations of effect sizes are available in the full report and in Appendix A, the Technical
Appendix.
2
The remaining five questions were also statistically significant but only for the longer time period,
between enrollment and post-completion. Clients reported, on average:
less often feeling unable to control the important things in their life (declined by 29%;
less often feeling helpless dealing with money problems (declined by 22%);
less often feeling little control over financial things that happen to them (declined by 19%);
less often feeling that there was little that they could do to change many of the money issues
in their life (declined by 15%).
more often feeling confidence in their ability to handle personal problems (increased by 8%).
5. Gained improved physical health.
The number of days clients reported that their physical health was not good during the
previous 30 days declined by 60%, from 4.2 days at enrollment to 1.7 days at post-completion.
Other Findings Over Time
The evaluation found other changes in outcomes over time for the larger samples of unmatched clients
who completed the program. These are findings that could not be tested statistically or did not rise to
the level of statistical significance but given the opportunity to extend the evaluation to collect and
analyze more data, might eventually confirm a trend or be found significant. We report them here and
throughout the report because they add to and deepen our understanding of the extent of outcomes for
the More Than Wheels clients. These findings describe responses of clients to a survey at program
enrollment compared to clients who took a second survey at program completion and finally, a third
survey at post-completion, approximately six months or more after finishing the program.
In the area of money management, over time more clients who completed the program:
used longer-term money-building and investment strategies with the money they saved;
used bank-based strategies like checks, debit cards, and online-banking to pay their bills; and
used spending plans or budgets to help them manage their finances.
More clients over time also reported:
feeling in control of their money,
working toward financial goals,
feeling comfortable doing business with banks and credit unions, and
an increase in monthly net work income.
Clients also reported other health outcomes:
an increase in the percentage of clients’ children with health insurance;
and a decrease in the percent of clients reporting not getting and postponing needed medical
care for themselves or their children.
With transportation the expectation was that clients would be able to increase their access to medical
care. The evaluation found that
3
by the end of clients’ participation in the program, no clients reported that they were not able
to obtain needed medical care for themselves or their children due to lack of transportation,
down from 23% at enrollment.
Reliable transportation was a serious issue for More Than Wheels clients but this concern declined:
At enrollment most clients worried about lacking a reliable car; by program completion and in
the months after, most clients did not worry about this at all.
Improved access to transportation was believed to improve clients’ opportunities to become more
involved with family, friends, and community.
Over the longer term clients reported increases in contact with friends or family andattendance
at religious services, social or community organizations.
The evaluation began just as the economy plunged into the Great Recession. For the population of
working poor and lower middle class served by More Than Wheels, the expectation was that many
clients were likely to increase their use of emergency food assistance during this time. However, over
time, fewer clients reported getting food from churches, food pantries, food banks, and soup kitchens.
Although Completed Clients’ use of SNAP benefits and their children’s eligibility for free and
reduced price lunch remained fairly stable, use of emergency food services decreased over time,
suggesting that they had improved management of their limited food budgets.
Clients’ concerns about having enough money for food decreased from over 40% at enrollment
to approximately 20-30% at follow-up.
Evidence for Changes Attributable to More Than Wheels
The findings above establish that clients who completed the program experienced significant changes in
their lives during the time they were in the evaluation. However, to know whether those changes were
due to their participation in the program, those results were statistically compared to the results of two
comparison groups over time. For most outcomes a significant finding was one in which clients changed
and comparison groups did not, suggesting that the program rather than other factors was the agent of
change. The following are the outcomes with statistical evidence of impact due to the program.4
1. Better money management behaviors.
By the time they completed the program, clients increased their regular savings activity more
than either of the two comparison groups.
Clients were significantly more likely to say they usually or always pay their bills on time than
one of the comparison groups; there was no significant difference with the other comparison
group.
2. Improved mental health.
Seven indicators that clients’ feelings of stress declined and self-confidence and optimism
increased over time, in marked contrast to the findings from the comparison groups. Both
groups reported either no change or change in the opposite direction over the same time
period for all but one indicator.
4
Results of statistical significance testing and calculations of effect sizes are available in the full report and Appendix A, the Technical Appendix.
4
3. Fewer barriers to using health care.
Participating in the More Than Wheels Bridge temporary car program or by completing the program and
obtaining a new car was expected to remove transportation as a barrier to health care. The evaluation
found some evidence for this assumption. At program completion or Time 2,
more comparison group participants than Completed Clients said that lack of reliable
transportation was affecting their ability to get to doctor’s appointments;
more comparison group participants than Completed Clients said they were postponing
needed medical care due to lack of transportation; and
4. Less food assistance utilization.
Clients who completed the program reported less use of food assistance than participants in comparison
groups.
At program completion or Time 2, More Than Wheels clients reported using emergency food
services like food pantries, food banks, and soup kitchens significantly less than one
comparison group, and the federal Supplemental Nutrition Assistance Program (SNAP)
program less than the other.
Conclusions
More Than Wheels is an innovative model for working with a challenging population: people coping
with often severe personal financial problems, often in tandem with other challenges such as illness and
disability, underemployment, low education and histories of crisis and loss. Our evaluation found strong
evidence that this model is having success at turning its clients’ lives around, helping them learn new
skills and how to change old habits and behaviors in the areas of personal money management and
health. Clients who completed the program were able to gain control of their personal finances and
maintain stability long enough to improve their credit, obtain a car loan and new car. Their physical and
mental health improved, and their reliance on emergency food assistance decreased. For many of these
clients, the changes were found to be long term, lasting well beyond the end of their involvement with
the program.
Because of the methods used to select client and comparison group participants and the resulting small
sample sizes, we cannot say conclusively that clients made these changes because of their participation
in More Than Wheels. Also, a number of changes that we expected to find, for example, in
employment, transportation, and community engagement, either were not found or were not
statistically significant , possibly because they required more time for impact to become measurable.
Nonetheless, it is clear from the results that there were notable improvements in several areas and that
for at least this one group of clients, the program was successful.
5
More Than Wheels is a nonprofit organization started in 2001 for the purpose of providing car buying
assistance, low interest guaranteed car loans, and temporary transportation primarily to low income
and vulnerable individuals. The program requires that clients participate in a program of financial
education and counseling as a prerequisite to applying for a low interest loan and obtaining a new car.
As part of this program clients learn about managing personal finances, how to improve their credit
scores, and ways to save money through smart food purchasing and developing a healthy lifestyle. Since
the program began, More Than Wheels has facilitated over $15 million in car loans, helping almost 1,500
people purchase reliable, fuel efficient vehicles.5
Headquartered in New Hampshire, the program initially served clients primarily from New Hampshire
and some clients from Vermont through sites in Concord, Keene, Lebanon, Manchester, and
Portsmouth. With additional funding a site was added in Boston, Massachusetts in 2008. Most staff
work in virtual offices, communicating with clients by telephone and computer. When needed,
arrangements are made with local human service organizations that provide space and computers faceto-face meetings with clients or for staff meetings or events. The addition in 2010 of a new
headquarters in Manchester added a centralized meeting space.
The Carsey Institute, a social science policy research and evaluation center at the University of New
Hampshire, has carried out several evaluations of the program. In 2007, Carsey evaluators conducted a
preliminary evaluation of the More Than Wheels program based primarily on a self-report survey of
clients and client credit reports. That study found that participants’ experiences with More Than Wheels
were very positive, giving support to the testimonials that More Than Wheels staff collects, some of
which are available on their website (http://www.morethanwheels.org/success-stories). Clients
reported that obtaining cars through the program helped them get to jobs, to do so on time, and
maintain steady work schedules; their overall financial situation and habits improved; they had
improved access to health care; and their credit scores improved.
In 2008 More Than Wheels received funding from the Robert Wood Johnson Foundation for the Carsey
Institute to conduct a more comprehensive and more rigorous process, outcome and impact evaluation
of the program. Because the organization collects, maintains, and analyzes substantial amounts of data
about its own operation as a tool for program planning and management, the primary purpose of this
evaluation was to obtain an independent assessment of the program’s impact on clients, particularly its
impact on health outcomes that had been reported anecdotally to the program by its clients.
Secondarily, the evaluation was to look at implementation of the program and identify any
characteristics that might affect program outcomes. The design that Carsey proposed for the process,
outcome and impact evaluation relied on both qualitative and quantitative methods and compared
More Than Wheels clients that participated in the evaluation with similar groups of Non-Clients and
Withdrawn Clients over time. The following report describes the results of this evaluation conducted
from over a period of almost three years with data collection beginning in August 2008 and ending in
May 2011.
5
Numbers from the More Than Wheels website: http://morethanwheels.org
6
The More Than Wheels program is a voluntary financial fitness and counseling program for people
struggling with personal credit and financial management problems. Its mission is “To help struggling
individuals and families break the cycle of poor financial decision making by using the car buying process
to catalyze lasting change, financial stability and control.6”
1.1 Eligibility
Individuals apply online or by calling the organization, completing an application form that asks for
information about household demographics, their current car, employment, credit history, and income
and expenses and paying a small fee. After assessing the information on income and monthly expenses,
staff members estimate the amount available to the individual at the end of the month. If that amount
is over $300, typically between $300 and $400, applicants are income-eligible. Once accepted,
applicants pay a program fee that is subtracted from the cost of the new car obtained at the end of the
program.
Income is only one program requirement. Eligibility also requires applicants to demonstrate through an
interview evidence of their willingness to change their money management habits and behaviors. This is
measured using four key determinants of success in the program that were identified by More Than
Wheels staff through a review of over 500 successful and unsuccessful clients. The resulting indicators,
Willingness to change, Ability to change, Motivation, and Initiative (WAMI) are included in an instrument
that staff developed for screening applicants, the WAMI Scale Document. The scale is used to rate
applicants Very Low, Some, Moderate, or Strong on these four dimensions and staff use the ratings as a
guide to their decision-making about an applicant. Because the Willingness to change dimension has
been found to be the most important of the four dimensions, sometimes applicants are screened for
that dimension alone.
1.2 Programming
Applicants who meet the eligibility requirements are invited to enroll, pay the program fee, and begin
the “Steps to Success” process.7 This process includes the requirement to attend six Financial Fitness
financial literacy classes and to meet monthly with a Client Consultant staff member for financial
counseling. The goals are to educate and counsel the client in personal money management, and to
help the client restore and stabilize their financial situation.
The Client Consultant works with clients to help them obtain and interpret their credit reports and other
documentation needed, and to learn how to budget, save money, be an educated consumer –
particularly purchases and activities that impact diet and health - and to practice other financial
strategies including reducing their personal debt. Throughout this process the Client Consultant
provides encouragement and psychological support to motivate and promote confidence in the clients.
If temporary transportation is needed during this period the client may also apply to the More Than
Wheels Bridge Program which supplies used cars as a transportation “bridge” until the new car is
obtained. Client Consultants conduct a progress review at 60 days prior to the estimated completion of
6
7
http://www.morethanwheels.org/mission
http://www.morethanwheels.org/steps-success
7
the program to determine the client’s progress, complete information needed to by the program to help
clients select a new car, and determine other information or actions needed to complete the program.
Once it is determined that the client is on track to complete the program, a process that takes from four
months to a year, the client moves from counseling to the Car Services staff where they work with the
client to choose a new cost and fuel-efficient car appropriate to the client’s budget and needs. Car
Services also works with the client to complete and submit a low-interest loan package to the bank and,
once approved, helps clients with the loan agreement and car delivery process. More Than Wheels
follows up with alumni of the program, monitoring loan payments and credit reports, and encourages
further counseling if needed.
1.3 Program Theory of Change
More Than Wheels says on its website: “At More Than Wheels, we are committed to promoting longlasting social change. With our program, clients not only get a solution to their immediate
transportation problem, they are also able to make permanent changes in their lives as a result of the
newfound knowledge and self-esteem they gain from our focus on improving their financial decision
making and physical health.” The program believes it is having a larger impact on clients’ lives because
of the anecdotes and testimonials obtained from clients and because of the positive results of an initial
evaluation of clients conducted by the Carsey Institute in 2007. The anecdotal mental and physical
health impacts piqued Robert Wood Johnson Foundation’s interest and led to support of the current
evaluation.
The Carsey evaluators understand the More Than Wheels theory of change as a theory of financial
stressors mediated by the program’s education and counseling (see Figure 1 below). This social support
helps clients reduce the stress of living with poor credit and an unreliable or no car by giving them
information, skills, counseling and positive reinforcement. As a result, clients gain capacity to change
their money management behaviors, improve their credit rating, and increase their financial stability.
Both the process of taking control over their financial circumstances and the tangible results of that
process – obtaining a car loan and new car – can have a number of different outcomes: reduced mental
stress and possibly better health overall; increased ability to seek out medical care; improved ability to
conduct daily activities requiring transportation; increased contact with friends, relatives, and others in
the community; increased ability to afford food and other necessities; and increased employment
opportunities resulting in increased income and greater access to employer-sponsored health insurance.
8
Figure 1. More Than Wheels Program Theory of Change Model
Typical Client
MTW Program Intervention
Poor credit
rating
Large debt
Financially
unstable
Underemployed
Low income
No insurance
Illness or
disability
Unreliable or
FIN FIT CLASS
Personal money
management education
& skill building
ONE-ON-ONE
COUNSELING
Case management with
social support & positive
reinforcement for
improved financial
behavior and successes
BRIDGE CAR PROGRAM
Short-term loan of a
used car if needed
Client Personal Money Management Outcomes
SHORT-TERM
Increased
money
management
knowledge &
skills
Increased
selfconfidence
and selfesteem
INTERMEDIATE
Improved
management of
personal debt
and financial
circumstances
Improved credit
LONG-TERM
Financial
stability
Self-sufficiency
New car:
improved
transportation
More engaged
with family &
community
Client Health-related Outcomes
Reduced mental stress
Improved mental health
Improved physical health
Increased access to health care
Increased ability to afford
insurance
Decreased food insecurity
9
The purpose of the comprehensive evaluation of More Than Wheels has been to obtain findings on the
key outcomes for clients predicted in the More Than Wheels theory of change. Our approach was to
investigate the outcomes of the program for clients, that is, to learn what changes occurred to clients
who participated in the program but particularly for those clients who completed the program. Because
the goal of More Than Wheels is to make permanent changes in clients’ knowledge, attitudes, and
behaviors about themselves and money, a primary objective of the evaluation was to track changes in
these outcomes over time, particularly to see if the changes are sustained after clients finish and leave
the program. Thus, the first question for the evaluation to answer was:
1) Do participants in the More Than Wheels program change over time?
This question can be investigated with a longitudinal design, by tracking clients over time and observing
their changes from one time to another, testing to see if the changes are large enough to attain
statistical significance. Results from our longitudinal analyses that are statistically significant are
important findings because they describe the actual changes in participants at various points in time in
relation to their program participation.
Because many factors in peoples’ lives other than participation in the More Than Wheels program could
account for the changes found in the clients, this evaluation took the additional step of asking a second
evaluation question:
2) Is there evidence suggesting that changes in More Than Wheels participants are a result of
participation in the program, i.e., that the program has an effect on participants?
This second question, concerning the impact or effects of the program, requires a more rigorous
research design, one as close to a controlled experiment as possible, to minimize the factors other than
the program that could explain changes in participants. Controlled experiments require randomly
assigning participants into treatment and control groups. Because this was not an option in this
evaluation, we chose the next best approach, a quasi-experimental design using two comparison groups
composed of people as similar as possible to the More Than Wheels clients.
In this design, the desired result is that over time the clients change and the comparison groups do not.
Such a finding is strong evidence – although not proof – that the program has had an effect on clients.
For many programs this is the best evidence of their impact that can be obtained. Results from our
analyses comparing clients to a comparison group that show statistically significant differences between
them are very important findings. They are evidence that program is having an effect on the
participants. Further, by calculating effect sizes for these statistically significant results, the extent of
the difference of between participants and the non-participant comparison group can be estimated.
The two designs used in this evaluation, longitudinal, to answer questions about outcomes, and
comparison group, to answer the question about evidence of program impact, are described in more
detail in Chapter 4 below.
10
Staff from More Than Wheels and Robert Wood Johnson Foundation (RWJF) worked with Carsey’s
evaluators to identify a set of client outcomes hypothesized to result in positive changes as a result of
exposure to the program based on the theory of change described above. These included dimensions of
the clients’ access to health care, health-related behaviors; transportation; financial “fitness” or literacy
and behaviors related to money management; credit scores; and involvement with community.
The Carsey and More Than Wheels staff collaborated to collect the data on these outcomes from several
sources. The primary source was a self-report Client Survey. Additional outcome data included
anecdotal data collected from several client focus groups, a test of knowledge clients completed in the
program’s Financial Fitness class, and client credit scores from credit rating services.
3.1. Client Survey Questionnaire
The purpose of the Client Survey questionnaire (see Appendix C for a copy) was to collect data on a set
of specific participant outcomes in each of the outcome areas described above. An initial version of the
questionnaire was developed in 2008 by a faculty member at the Dartmouth Institute for Health Policy
and Clinical Practice in collaboration with Carsey Institute evaluators, RWJF staff and More Than Wheels
staff. Items in this questionnaire were taken directly or modified from the National Health And
Nutrition Examination Survey (NHANES) 2005-2006 survey,1 Commonwealth Fund 2005 Biennial Health
Insurance Survey,2 Community Tracking Study Household Survey,3 Social Capital Community Benchmark
Survey,4 and a social contacts survey.5 The questions were designed to measure dimensions of the
client’s transportation; health status and health insurance utilization; food insecurity; mental and
emotional stress as it relates to money; personal money management strategies; and community
engagement. In 2010, after preliminary analyses found that some items were not relevant to the study
as it was developing, the Carsey evaluators revised the questionnaire, removing those items and adding
some untested questions relevant only to the More Than Wheels program.
3.2. Client Focus Groups
Carsey conducted two focus groups (see Appendix C for the protocol) in August 2009 in two different
locations in New Hampshire: Lebanon and Manchester. Lebanon is the oldest program site, serving
clients from Vermont and western New Hampshire, and at that time Manchester was comparatively
new, serving clients in central and southern New Hampshire and Massachusetts. Thirteen former
clients, four males and nine females, participated in the focus groups, all recruited by More Than Wheels
staff. Most were from New Hampshire with two from Vermont and one from Massachusetts. All had
finished the program in the past year or earlier which means that the experiences of most participants
referred to the More Than Wheels program as it was prior to the start of our evaluation in August 2008.
The selection of former clients for the focus groups was intentional. We wanted to hear from clients
who could provide insight into the longer term outcomes of the program as supplemental data to
follow-up interviews with Client Survey respondents which we expected to be challenging because of
the relatively short length of time we had available to follow up with them and likely difficulties in
1
Centers for Disease Control and Prevention, http://www.cdc.gov/nchs/nhanes.htm
The Commonwealth Fund, http://www.commonwealthfund.org/Content/Surveys/2005/2005-Biennial-Health-Insurance-Survey.aspx
3
Center for Studying Health System Change, http://www.hschange.com/index.cgi?data=02.
4
John F. Kennedy School of Government, Harvard University, http://www.cfsv.org/communitysurvey/docs/survey_instrument.pdf.
5
Donald, C.A. and Ware, J.E., 1982. The Quantification of Social Contacts and Resources. The Rand Corporation.
http://www.rand.org/pubs/reports/2006/R2937.pdf.
2
11
tracking them after they left the program. A few focus group participants had been through the More
Than Wheels program more than once, buying several cars over the years.
The findings from these focus groups were previously reported in Carsey’s Interim Report on the
evaluation.6 For this report we include summarized findings from that report to provide a more
complete picture of the data collected and findings for the outcome evaluation.
Given that the focus group participants were selected by More Than Wheels staff and had stayed in
touch with More Than Wheels so were easily contacted suggests that they were a biased group. In fact,
all the focus group participants in both groups described a number of positive changes to their lives as a
result of their involvement with More Than Wheels and/or the car that they had obtained through the
program. Even so, some participants criticized some aspects of program implementation, comments
that were included in the Interim Report.
Many of the client stories reported during the focus groups are similar to the testimonials that can be
found on the More Than Wheels website. They were analyzed and fall into four types of impacts:
health access, mental health, financial, and social. Each of these findings is included as supplemental
data in each relevant section of the Findings in Chapter 6.
3.3. Financial Fitness Knowledge Test
More Than Wheels requires program enrollees to participate in Fin Fit, a financial literacy or “fitness”
class, a series of classes that teach participants about a variety of personal financial management
strategies. The purpose is to educate them as consumers and borrowers so they have the information
and skills needed to improve their own financial circumstances and success in applying for a new car
loan.
The Fin Fit classes began prior to the evaluation and some data on participant satisfaction was collected
at that time. With the implementation of the evaluation, Carsey worked with staff to develop an
objective knowledge test. During 2010 this test was revised to reflect changes in the class (see Appendix
C for a copy of the test). The test was administered prior to the first class and after the end of the final
class in the course. Results of the data collected from the pre- and post-test administrations during a
two and a half year period of evaluation are included in the Financial Management section of Chapter 6
Findings below.
3.4. Client Credit Information
More Than Wheels staff members obtained permission from clients to contact credit agencies for their
credit reports at program enrollment. During the last year of the evaluation, staff again asked to
request follow-up credit reports on clients who had completed the program to provide data for an
evaluation of change in clients’ credit status. Follow-up scores were also available for some Withdrawn
Clients. The results of the analysis of these data are presented in the Financial Management section of
Chapter 6 Findings below.
6
Wauchope, B. 2010. Comprehensive Evaluation of Bonnie CLAC Interim Report. Carsey Institute, University of New Hampshire.
12
As noted above, this evaluation used two complementary designs, longitudinal and quasi-experimental
comparison group, to structure the Client Survey data collection and analysis to answer the two major
evaluation questions. Each of these is described in more detail below.
4.1. Longitudinal Outcome Evaluation
Carsey chose a longitudinal design as the primary means to assess the outcomes of clients who
completed the program as measured by data collected from the Client Survey questionnaire. This
design required collecting data over multiple points in time from as many clients as possible at each
time point, ideally with the same clients taking the survey repeatedly. In reality, some attrition did
occur, and with each survey administration, fewer clients participated (for details of sample attrition see
the Attrition Analyses in Appendix A, the Technical Appendix). The resulting longitudinal dataset
allowed for comparisons of each client group that participated in the survey at each point in time:
at program enrollment (referred to throughout the text and figures in this report as
“Enrollment,” “Time 1” or “Time 1”);
up to 3 months after program completion (referred to as “Program completion,” “Time 2” or
“Time 2”); and
at 6 months or more after they had completed and left the program with their car loan and new
car (“Post-Completion,” “Time 3” or “T3”).
Because enrollment in the program was rolling, i.e., clients came in one at a time rather than as a cohort
group, client participants were tracked individually for their continued enrollment in the program over
time through the More Than Wheels database system. As a result, we could not conduct analyses over
time with the clients as a single cohort, i.e., individuals starting and completing the program together.
Instead, three large samples were created from the pool of all clients who took the Client Survey based
on their participation in 1) the enrollment survey, 2) a follow-up survey conducted at program
completion or Time 2, and 3) a second follow-up survey conducted at post-completion. From the group
that participated in the enrollment survey a subset of clients was identified that had successfully
completed the program. This group, the second group that had completed the program completion
survey, and the third group that had finished the program and completed a second follow-up survey at
least six months after completing the program, became the three Completed Client groups whose
responses to the Client Survey were the basis for the analysis and findings in this report.
To analyze client change over time, we selected two subsets of matched clients from the larger samples
of Completed Clients described above.
To measure short-term change, from time of enrollment through program completion, we
selected the subset of Completed Clients who had participated in both the enrollment survey
and the follow-up survey at program completion.
To measure longer term, presumably more lasting change, from time of enrollment to at least
six months beyond program completion, we selected the subset of Completed Clients who had
participated in both the enrollment survey and the follow-up survey at post-completion.
Very few clients participated in both follow-up surveys so we were not able to analyze change for the
period between program completion and post-completion. For further information on how we arrived
at the samples for the analyses, see the Data Management and Analysis section of Appendix A, the
13
Technical Appendix. Results of the longitudinal analyses are presented in Chapter 6 with detailed tables
of statistically significant findings in Tables of Statistical Tests in Appendix A, the Technical Appendix.
4.2. Quasi-Experimental Impact Evaluation
The second design used in this evaluation was quasi-experimental, collecting the same Client Survey
data from two comparison groups as we collected from the Completed Clients for the purpose of
analyzing the similarities and differences with the Completed Clients on their responses to the Client
Survey questions. The resulting comparisons allowed us to test statistically for differences between the
Completed Client group and the two other groups and estimate the size of those effects.
We had available to us two comparison groups for this evaluation and they included:
Non-Clients – a volunteer group of survey participants who had contacted More Than Wheels
but after learning more about the program either decided they did not want, were not ready, or
were not eligible to enroll in the program; and
Withdrawn Clients – former clients who enrolled in the program but subsequently withdrew
from the program before completion.
Both groups participated in at least one initial Client Survey (Time 1) and both were contacted for a
follow-up survey (Time 2). The initial survey for Non-Clients was conducted during the second year of
the evaluation with a follow-up approximately a year later. Withdrawn clients participated in the
enrollment survey along with other clients who completed and were contacted about a follow-up survey
after dropping out.
Ideally comparison groups should be as similar to the clients and to each other as possible for a quasiexperimental design (see Chapter 5 for tables comparing the groups). We did not have large enough
samples in either the Non-Client or Withdrawn Client groups to select a subset of the individuals that
best matched the characteristics of clients for use in our analyses. Instead from the entire sample of
Non-Clients and the entire group of Withdrawn Clients we selected subsets of matched individuals for
use in the analyses. These individuals had completed both the initial survey and a follow-up survey.
As noted above, if the Completed Clients and a comparison group responded to survey questions
similarly at the beginning of the clients’ exposure to the program but over time the clients’ answers
changed while those of the comparison group remained the same, then the statistically significant
difference between the two groups is strong evidence for program impact, i.e., that the program
contributed to the clients’ change. To look for evidence of program impact we first carried out the
longitudinal analyses for the Completed Clients to determine if there was statistically significant change
between enrollment (Time 1) and program completion (Time 2) or between enrollment (Time 1) and
post-completion (Time 3) for the matched Completed Client samples. We did the same for each of the
comparison group matched samples for change between Time 1 and Time 2. Finally, we did a similar
comparison between the clients in the Time 1-Time 2 and Time 1-Time 3 groups with each of the
comparison groups. The statistical methods and associated tests we used to conduct these comparisons
included:
Paired samples t-tests for change over time in one group, pairing Completed Client Time 1-Time
2, Completed Client Time 1-Time 3, Non-Client Time 1-Time 2, Withdrawn Client Time 1-Time 2,
and in some circumstances the complete sample at two points in time.
Wilcoxon signed-rank tests, the nonparametric alternative to the paired samples t-test for
ordinal variables, i.e. those resulting from survey questions for which participants selected a
14
response from a scale. This type of question comprises the majority of the Client Survey items.
Nonparametric tests in general are more suitable for response distributions that are skewed or
peaked, which is more likely to occur with smaller samples.
Independent samples t-tests for differences between groups at one point in time, comparing
Completed Client Time 1-Withdrawn Client Time 1, Completed Client Time 1- Non-Client Time 1,
Completed Client T2- Withdrawn Client Time 2, and Completed Client Time 2-Non-Client Time 2.
Mann-Whitney U tests, the nonparametric alternative to the independent samples t-test.
Mixed between-within analysis of variance (ANOVA) for comparing change over time and
differences between groups as well as the interaction effects of time and group. Because of the
types of questions in the surveys there were limited occasions to use the ANOVA, which in most
instances should not be used for ordinal or non-normally distributed variables.
In addition, effect sizes were calculated for many of the comparisons using Cohen’s d for scale and
ordinal variables in the longitudinal and cross-sectional analyses, and partial eta squared for ANOVAs
only.
The results of these analyses are presented in Chapter 6 Findings. Details of the statistical tests and
estimated effect sizes are found in the Tables of Statistical Tests in Appendix A, the Technical Appendix.
15
The following sections describe the demographic characteristics of client and comparison samples used
in the outcome/impact analyses for the Findings in Chapter 6. For a summary description of the
characteristics of each of the Completed Client samples only, see Comparison of Samples of Completed
Clients in Appendix A, the Technical Appendix.
5.1. Client Samples
Participation Rates. Over the three years of the evaluation, 41% of eligible enrolled clients participated
in the enrollment survey (350 of 847), which, along with signing the consent forms, enrolled them in the
evaluation. More Than Wheels staff members recruited participants into the evaluation in several ways,
e.g., distributing the questionnaire at the beginning of the Fin Fit class and later at the initial meeting
with the Client Consultant. The latter proved to be more effective strategy, yielding a participation rate
of 60% compared to the rate of 28% when it was distributed in the class.
Evaluation Participant and Non-Participant Comparisons. To learn how generalizable our findings for
the evaluation were, we compared the clients in the evaluation with the entire dataset of More Than
Wheels enrolled clients to find out how similar or different our evaluation participants were from the
overall population of program clients. If the result showed that evaluation clients were significantly
different on many characteristics, then findings about client change and program impact might not
apply to program clients overall. See Appendix A for a table that summarizes these demographic data
for all the program participants and for all the clients in the evaluation at enrollment. We found no
significant differences for sex, race/ethnicity, marital status, monthly net income or initial credit score.
We did find significant differences in education, age, and employment status.
Figure 2. Comparison of education level between the subset of evaluation participants and all More
Than Wheels clients.
100%
80%
60%
Evaluation non-participant (n=442)
40%
Evaluation participant (n=283)
20%
0%
Less than HS
HS/GED
Vocational
College
Evaluation participants were significantly different*** in their education from those who did
not participate. Evaluation participants were slightly less well-educated than other clients.
Employment status was also significantly different* between survey participants and nonparticipants. A greater percentage of participants were employed full time, employed part
time or part of the year, or were unemployed and looking. A greater percentage of nonparticipants were self-employed, retired, disabled, or “other”.
***
*
p < .001
p < .05
16
Figure 3. Comparison of the evaluation enrollment survey participants with clients who did not
participate.
100%
Enrollment
survey nonparticipants
(n=450)
80%
60%
Enrollment
survey
participants
(n=308)
40%
20%
0%
Employed full time
Self-employed
Retired
Other
Finally, age was significantly different for the evaluation participants compared to non-participating
clients.
Evaluation participants were on average slightly younger (38 years) than other program clients
(41 years).
It is difficult to know how these differences could affect the generalizability of the results of the
evaluation to the program clients overall. However, the practical result of this analysis is a reminder
that we did find several differences and this should be kept in mind when making any inferences about
the program clients based on the evaluation findings.
Client Sample Comparisons. In each of the two follow-up survey administrations we used slightly
different versions of the same survey within three months after program completion and at postcompletion, at least six months after finishing the program. Both were conducted by the University of
New Hampshire Survey Center as either telephone interviews or as an online survey. Out of 135 clients
that participated in at least one survey and completed the program, approximately one-half (51%)
participated in one of the two follow-up surveys. There was no statistically significant difference
between those evaluation participants who completed a follow-up survey and those that did not based
on gender, race/ethnicity, age, education, initial credit score, or net monthly work income.
5.2. Comparison Group Samples
Different strategies were used to recruit each of the comparison groups for the evaluation. Non-Clients
were contacted by telephone for the initial survey. A sample of 100 agreed to participate in the initial
survey. For their follow-up survey, this group was contacted by telephone again about a year later.
The Withdrawn Clients, 171 evaluation participants who completed the enrollment survey and
subsequently withdrew from the program, were invited to participate in a follow-up survey. They were
contacted by telephone and approximately one-quarter participated in this second survey.
Table 1 on the next page summarizes the demographic characteristics of each of these groups and for
the Completed Clients for the purpose of comparing them. One of the assumptions in the impact
evaluation is that if all three of these groups are similar at enrollment (for Completed Clients and
Withdrawn Clients) or at the first survey (Non-Clients), any changes found for the Completed Clients but
not for the comparison groups may be attributable to the program.
17
Table 1. Characteristics of Completed Client and Comparison Group Samples at Time 1
Completed Clients
Non-Clients
Withdrawn Clients
Enrollment
Initial
Survey
Enrollment
Characteristic
(Time 1)
(Time 1)
(Time 1)
(n=64)
(n=97)
(n=52)
Sex
Males
18.8%
24.7%
26.9%
Females
81.3%
75.3%
73.1%
African-American
9.4%
3.1%
11.5%
White non-Hispanic
79.7%
89.7%
84.6%
Other
4.7%
6.2%
3.8%
Missing
6.3%
1.0%
0.0%
High school graduate or less
17.2%
38.2%
13.4%
Some college or vocational school
26.5%
32.0%
50.0%
College or graduate degree
39.0%
25.7%
19.1%
No answer
17.2%
4.1%
17.3%
Less than $1200
17.2%
20.6%
25.0%
$1200 - $2000
35.9%
14.4%
34.6%
More than $2000
39.0%
13.4%
21.2%
No answer/Don’t know
7.8%
51.5%
19.2%
20.3%
23.7%
17.3%
Single
---
44.3%
---
Divorced
---
21.6%
---
79.6%
10.4%
82.7%
Employed full-time
70.3%
28.9%
51.9%
Employed part-time
25.0%
16.5%
28.8%
Self-employed
---
6.2%
---
Unemployed and looking for work
Unemployed and not looking for
work, retired, homemaker or
student
Disabled
---
13.4%
5.8%
1.6%
18.6%
1.9%
1.6%
12.4%
7.7%
No answer
1.6%
4.1%
3.8%
Race/ethnicity
Education
Monthly Net Work Income
Marital Status
Married
Other/No answer
Employment
18
We found one significant difference* between the Completed Clients and the Non-Client
samples in monthly net income, which was lower.
However, over one-half of the clients did not respond to this item so it is difficult to know how accurate
this finding is. Nonetheless, this finding is a reminder to be cautious when interpreting and drawing
conclusions about the findings presented in Chapter 6 regarding program impact. If the Completed
Clients were significantly different on income from one of the comparison groups, they could have been
different in other ways that were not measured. Consequently, a finding of significant differences
between Completed Clients and the comparison groups could mistakenly lead one to draw the
conclusion that the finding indicates program impact, when in fact no such conclusion can be assumed.
*
p < .05
19
In this chapter we summarize the results of statistical analyses of client outcomes to answer the two
evaluation questions:
1) Do participants in the More Than Wheels program change over time?
2) Is there evidence suggesting that changes in More Than Wheels participants are a result of
participation in the program, i.e., that the program has an effect on participants?
The data come primarily from the three Client Surveys described in the previous chapter, with
supporting data from More Than Wheels’ program data, clients’ credit scores, scores on a financial
fitness knowledge test, and findings from several client focus groups.
To answer Question 1, the responses of the following group of survey participants were analyzed:
Completed Clients: the group of clients in the evaluation who completed the program and
participated in the Enrollment survey (referred to below as Time 1) and at least one of two
follow-up surveys, either at program completion or Time 2 or at least six months after
completion of the program (Post-Completion or Time 3).
Two sets of analyses were conducted, between enrollment and program completion (Time 1 to Time 2)
and between enrollment and post-completion (Time 1 to Time 3).1
Question 2 was answered by comparing the results of Question 1 to similar analyses of each of the
following two groups for changes in their responses to the survey over time.2
Non-Clients: a group of potential program applicants who participated in an initial Non-Client
Survey (referred to as Time 1), some of whom then participated in a follow-up Non-Client
Survey (at Time 2);
Withdrawn Clients: a group of clients who enrolled and participated in the Enrollment survey
(Time 1) but withdrew before completing the program and were contacted later to complete a
follow-up survey (Time 2).
In the following sections of this chapter we present the findings from the analyses the survey and
supplementary data organized by outcome topic area. Findings of statistically significant results for
changes in Completed Clients over time provide strong evidence for Question 1: clients who completed
the program did change, and some of those changes were sustained into the months after leaving the
program. The findings that result from comparing clients to similar Non-Client and Withdrawn groups
provide evidence for Question 2: the changes can be attributed to the program intervention, and had
effects on clients.
When interpreting these results, keep in mind that because the two comparison groups were not
randomly selected from the same pool of people as the clients, we do not know precisely how they
differ from the clients. While some of these findings are strong evidence that More Than Wheels had
positive effects on clients, we cannot rule out the possibility that there were other reasons for these
changes that we did not measure.
Findings from three types of analyses are presented:
1
We did not have enough clients who completed both the initial follow-up survey at Time 2 and the second follow-up survey at Time 3 to make
matched Time 2 – Time 3 comparisons.
2
Details of these analyses, including test statistics, p-values, and effect sizes, can be found in tables in Appendix A, the Technical Appendix.
20
1) Longitudinal analyses: Most analyses compared the same groups of Completed Clients or
the Non-Client or Withdrawn Client comparison groups matched over points in time. In
other words, the same people were analyzed at program enrollment or Time 1 and again at
program completion or Time 2. Completed Clients were also analyzed at program
enrollment or Time 1 and post-completion or Time 3. The results of each of these sets of
analyses are presented using line graphs to show average responses for each group at each
point in time.
2) Cross-sectional analyses: Other analyses compared different groups of evaluation
participants at single points in time, such as Completed Clients and Withdrawn Clients, in
order to determine if and to what extent the findings for comparison groups diverge from
those of Completed Clients.
3) Descriptive analyses: In some cases, because of limitations in the data available, we were
not able to match people over time, i.e., different people participated in the enrollment and
follow-up surveys. Instead we compared the results of one group of people responding at
one point in time with the results of a second, similar but not matched group at a second or
third point in time. These findings are presented as bar charts to make it clear that they are
not comparisons of the same people followed over time.
Both statistically significant and non-significant findings are included in these findings. Those that are
statistically significant are noted with footnotes at the bottom of the page and shown in bulleted bold
text.
The sample sizes reported in the figures vary as the number of participants available for inclusion in
each analysis depended on their rate of response to each survey item. Consequently sample sizes are
smaller than the full sample obtained for each group.
21
6.1. Financial Management Outcomes
Much of the work of More Than Wheels is education and counseling on personal money management.
By teaching clients strategies to, for example, save money and pay bills on time, the expectation is that
clients’ financial circumstances will change enough that their credit rating improves and they will be
eligible for a car loan.
In the focus groups that we conducted with past clients, they indicated a strong belief that the program
had been successful in changing their financial management behaviors.
Former clients reported that participation in the program reduced their debt, helped them get
their financial problems under control, and identify solutions to help them improve their credit
score.
These clients also said the program increased their “money awareness” helping them do a
better job of budgeting and managing their money partly as a result of the Financial Fitness
classes but primarily through the counseling and work with their client consultant.
The following sections summarize the responses to the questions asked about this topic in the surveys.
Saving. The Client Survey first asked how often respondents usually save money. This was followed by
a list of options for where they usually put money that they saved.
Figure 4a. In the past six months: “How often did you save money?“ Comparison of the average
responses of Completed Clients over time.
2.00
Completed Client
Enrollment to
Completion
(n=27)**
1.50
2=Regularly
1=As often as I could 1.00
0=Never
Completed Client
Enrollment to
Post-Completion
(n=39)
0.50
0.00
Enrollment
Completion
Post-Completion
Clients increased their savings activity significantly ** by two-thirds (66%), between enrollment
and program completion.1
At enrollment 19% reported regularly saving money, a figure that increased to 35% at program
completion. This number declined somewhat at post-completion, but the difference was not
statistically significant.
When we compared Completed Clients with the Non-Client comparison group, the Completed Clients
had a better record of saving than the Non-Clients at both times.
**
1
p < .01
Technical Appendix Q38
22
Figure 4b. In the past six months: “How often did you save money?” Comparison of the average
responses of Completed Clients with a Non-Client comparison group.
2.00
Completed Client
Enrollment to
Completion
(n=27)**
1.50
2=Regularly
0=Never
Completed Client
Enrollment to
Post-Completion
(n=39)
0.50
Non-Client Time
1 to Time 2
(n=31)
0.00
Enrollment/T1
Completion/T2
Post-Completion
The increase during the same time period for Non-Clients was not large enough to be
significant. Overall Completed Clients were saving significantly more often than Non-Clients
at both enrollment** and at program completion.*2
Comparing Completed Clients to Withdrawn Clients produced a stronger result.
Figure 4c. In the past six months: “How often did you save money?” Comparison of the average
responses of Completed Clients with a Withdrawn Client comparison group.
2.00
Completed Client
Enrollment to
Completion
(n=27)**
1.50
2=Regularly
0=Never
Completed Client
Enrollment to
Post-Completion
(n=39)
0.50
Withdrawn Client
(n=49)
0.00
Enrollment/T1
Completion/T2
Post-Completion
Completed and Withdrawn Clients were similar in their savings at enrollment. However, at
Time 2, significantly** more Completed Clients reported saving than Withdrawn Clients.3
Both of these findings suggest that the program was having an impact on clients’ savings behavior.
**
p < .01
p < .05
2
3
*
23
Bill paying. We also asked about bill-paying behavior.
Figure 5a. “Which statement best describes how you paid your bills, rent and other expenses during
the past six months?” Comparison of the average responses of Completed Clients over time.
4
Completed Client
Enrollment to
Completion
(n=26)*
4=I always paid on
time
3=I usually paid on 3
time
2=I sometimes paid
on time
2
1=I almost never
paid on time
Completed Client
Enrollment to
Post-Completion
(n=38)***
1
Enrollment
Completion
Post-Completion
Completed Clients’ reports of timely payments increased significantly, by 13%, between
enrollment and program completion.* From enrollment and to six months or more after they
finished the program, this behavior increased significantly by 31%.*** 4
At enrollment, 71% of Completed Clients reported that they “usually” or “always” paid bills on
time, and by six months or more after leaving the program, all of the Completed Clients
(100%) gave these responses.
Comparisons with Non-Clients and Withdrawn Clients found patterns similar to those found for savings.
Figure 5b. “Which statement best describes how you paid your bills, rent and other expenses during
the past six months?” Comparison of the average responses of Completed Clients with Withdrawn
Clients and Non-Client comparison groups.
4
Completed Client
Enrollment to
Completion (n=26)*
4=I always paid on
time
3
3=I usually paid on
time
2=I sometimes paid
on time
1=I almost never 2
paid on time
Completed Client
Enrollment to PostCompletion (n=38)***
Withdrawn Client
Enrollment to T2 (n=49)
Non-Client T1 to T2
(n=31)
1
Enrollment/T1
Completion/T2
*
p < .05
p < .001
4
Technical Appendix Q42.
***
24
Post-Completion
On this question, all three groups, the Completed Clients, Non-Clients, and Withdrawn Clients were on
average similar at enrollment. However, by Time 2:
Completed Clients were significantly more likely, on average, to pay bills on time at Time 2**
than Withdrawn Clients but this was not the case in the comparison to Non-Clients.5
With one comparison significant but the other not, the pattern is less clear than it was for the savings
question. Nonetheless, the results for both savings and bill paying suggest that More Than Wheels is
changing some aspects of clients’ money management behavior.
Other financial management topics. Other personal financial management behaviors and a series of
statements about perceptions of respondents’ own financial management understanding and skills were
not found to be statistically significant. The findings presented below are descriptive and crosssectional, showing the results for all the clients who responded to the questions in each of the surveys,
whether or not they completed the program. Although not statistically significant, some suggest
positive trends for clients over time.
Clients were asked where they saved their money.
Figure 6. In the past 6/12 months: “Where did you usually put the money that you saved?”
Comparison of client responses in three surveys.
100%
80%
60%
Enrollment (n=55)
40%
Completion (n=43)
20%
Post-Completion (n=41)
0%
Savings Checking
account account
Savings
bonds
In my
home
IDA
IRA/401K
Other
Savings accounts and checking accounts were the most popular places to put savings across all
three surveys. However, by the last survey, more clients were using them than in either of the
previous surveys and the use of retirement accounts was up to 39%.
By the third survey, the percentage of clients keeping their savings at home was at its lowest.
**
5
p < .01
Technical Appendix Q42.
25
Figure 7. In the past 6/12 months: “How did you usually pay your bills?” Comparison of client
responses to three surveys.
100%
Enrollment
(n=61)
80%
60%
Completion
(n=46)
40%
PostCompletion
(n=42)
20%
0%
Personal
check, debit,
electronic
Cash
Check-cashing Money order Credit card
store
Online
banking
Other
The use of a personal check, debit card, or other electronic payment was the most frequently
reported form of payment among clients in all three surveys.
By post-completion two-thirds of clients (67%) were using online banking, up from one-fifth
(20%) at enrollment.
Respondents were also asked if they used a spending plan or budget to help them meet their monthly
expenses.
More than half (57%) of those surveyed at enrollment said they used a spending plan or budget,
a percentage that increased to just over three-quarters (76%) at post-completion.
Respondents were also asked about changes in their credit card debt.
Over half of respondents, between 54% and 59% across all three surveys, said they had credit
card debt.
At enrollment 22% said that their debt had decreased in the previous six months prior to
enrolling. By the time clients had completed the program, 38% reported that their credit card
debt had decreased in the previous six months. However, this number decreased to 27% by the
Time 3 survey six months or more after program completion.
All survey respondents were asked if they had seen a credit report from a credit-reporting agency in the
past six months.
At enrollment, two-thirds (67%) said they had seen a credit report in the past six months. This
percentage decreased to 52% by six months or more after program completion.
The Client Survey included a set of four statements to obtain respondents’ perceptions of their own
personal financial management.
26
Figure 8. Percentage of respondents answering “agree” or “strongly agree” to statements about how
they felt about managing money during the past six months. Comparison of client responses in three
surveys.
100%
80%
Enrollment (n=60)
60%
40%
Completion (n=46)
20%
0%
I was in control of I had financial I understood the I was comfortable
my money
goals that I was way I spend my doing business
working towards. money enough to with a bank or
make a budget.
credit union.
In the enrollment survey, about half (52 %) of completed clients said they were in control of
their money. By post-completion almost all (93%) of respondents answered this positively. This
pattern held for the other three statements as well although the differences between the two
surveys were not as large.
Over 90% of clients responded positively to all four of the statements at post-completion; 100%
responded positively that they had financial goals they were working toward.
Several cross-sectional analyses were conducted on these items.6 There were significant
differences found in the perceptions of financial management between Completed Clients and
the Non-Client comparison group, several of which were unexpected and difficult to interpret.
Figure 9. Percentage agreeing with statements about financial management: Comparison of
Completed Clients and Non-Clients at Time 1 and Time 2.
100%
Completed Clients
Enrollment (n=60)
80%
60%
40%
Completed Clients
Completion (n=45)
20%
Non-Clients Time 1 (n=94)
0%
I am in control of
my money
I have financial
I understand the I am comfortable
goals that I am
way I spend my
doing business
working towards money enough to with a bank or
make a budget
credit union
Non-Clients Time 2 (n=31)
At Time 1, Non-Clients were more likely to agree that they were in control of their money ***
and that they understood how they spend their money enough to make a budget.**
6
Details of the results of these analyses can be found in Appendix B.
27
Completed Clients were more likely to agree that they had financial goals they were working
toward** and that they were comfortable doing business with a bank or credit union.*7
The differences held at Time 2 only for those items with which Non-Clients were more likely to
agree: that they were in control of their money** and that they understood how they spend
their money enough to make a budget.*
At Time 3 Follow-up, however, Completed Clients agreement with these statements increased
to exceed that of Non-Clients and of previous points, at over 90% agreement for all four and
100% agreement that they have financial goals they are working toward.
There were no significant differences in agreement with these statements between Completed Clients
and Withdrawn Clients at Time 1 or Time 2.
Financial Fitness Knowledge Test. More Than Wheels clients completed a pre and post financial fitness
knowledge test as part of their required Financial Fitness class. The score is a sum of questions
answered correctly. The following shows the results for all clients.
Figure 10. Financial Fitness Knowledge Test: Comparison of averaged scores of all clients pre- and
post-course.
20
15
Fin Fit quiz scores 10
All Clients*** (n=285)
5
0
Pre Quiz
Post Quiz
For all clients, the average post quiz score for the group (17.85) was higher than the pre quiz
score (11.79), a statistically significant increase of 6.06 points indicating that clients had
increased knowledge about financial fitness by the end of the class.***8
We also compared Completed Clients and Withdrawn Clients for their scores on this test to see if the
two groups might have been significantly different in what they learned in the class, thus possibly
predicting client withdrawal.
***
p < .001
p < .01
*
p < .05
7
Technical Appendix Q45-48
***
p < .001
8
Technical Appendix.
**
28
Figure 11. Financial Fitness Knowledge Test: Comparison of averaged scores of Completed Clients and
Withdrawn Clients pre- and post-course.
20
15
Completed Clients*** (n=138)
Fin Fit quiz scores 10
Withdrawn Clients*** (n=79)
5
0
Pre-Quiz
Post-Quiz
There was no significant difference between the two groups: Fin Fit scores increased
significantly whether or not clients ultimately completed or withdrew from the program.***
Completed clients’ scores increased by 55% and Withdrawn Clients’ by 47%.
This finding indicates that clients can benefit from the Fin Fit course and gain financial literacy regardless
of their ultimate program outcomes.
Credit information. More Than Wheels asked clients to request credit score data soon after enrollment.
A second request was made later of some Completed Clients.
Figure 12. Financial credit scores: Comparison of initial and most recent scores for Completed Clients
700
650
600
550
Completed Clients*** (n=58)
500
450
400
Initial
Most Recent
The average credit score for Completed Clients increased significantly over time by 37.93
points, from 595.52 to 633.45.9*** 11
The average initial credit score of all Withdrawn Clients (555.37) was significantly
lower***than the average initial credit score of all Completed Clients (594.17). Most recent
credit scores were not available for Withdrawn Clients in sufficient numbers for analysis.
Income and employment. The evidence presented above regarding savings, bill-paying behavior, and
improvements in credit scores indicates greater financial knowledge and better financial practice.
However, all of these outcomes are to some extent dependent on income. Although there was no
significant change in clients’ average net income from work, there is a positive trend.
***
9
p < .001
See Technical Appendix
29
Figure 13. Monthly net work income over time: Comparison of Completed Clients over time.
$2,500
$2,000
$1,500
Completed Client Enrollment to
Completion (n=23)
$1,000
Completed Client Enrollment to
$500
$0
Enrollment
Completion
Post-Completion
Client’s monthly net work income rose by 10% from enrollment to program completion,
from an average of $1,981 to $2,186.
From enrollment to the post-completion period, income rose by 14%, from an average of
$1,785 to $2,033. However, for this period of time, average income began and ended lower
than the time period between enrollment and program completion.
Because of the data available to analyze for clients’ employment status, the findings are inconclusive
(see the two figures that follow below).
Figure 14a. Client employment status: Comparison of Completed Clients between enrollment and
program completion.
100%
80%
60%
Enrollment
(n=27)
40%
Completion
(n=27)
20%
0%
Employed full Employed part Self-employed Unemployed
time
time or part of
and looking
the year
30
Disabled
Other
Missing
Figure 14b. Client employment status: Comparison of Completed Clients between enrollment and
post-completion, at least six months after program completion.
100%
80%
Enrollment
(n=39)
60%
PostCompletion
(n=39)
40%
20%
0%
Employed full Employed Self-employed Unemployed
time
part time or
and looking
part of the
year
Disabled
Other
Missing
Both charts show decreases in full-time and part-time employment suggesting a negative trend over
time. However, it is likely that this is due to a sharp increase in missing data, from 0% and 3% in the
enrollment survey to 22% in the program completion survey and 18% in the post-compleition survey.10
With no employment data for approximately one-fifth of clients, we cannot draw any conclusions about
trends.
10
Missing data occurred because the employment status data used for Time 1 was collected by More Than Wheels at enrollment intake while
data for Time 2 and Time 3 relied on volunteer response to the Client Survey question on employment status. Approximately one-fifth of
survey participants chose not to respond to this question.
31
6.2 Health And Stress Outcomes
Clients typically come to More Than Wheels overwhelmed by money problems and the stress that
accompanies them. The program strives to improve participants’ physical and mental health by
mediating this financial stress. Program staff provides some educational information about diet and
health through their Financial Fitness course but the primary activity is the social and emotional support
counselors give to clients as they endeavor to improve their financial situations. Theoretically, by
providing activities that encourage clients to change to a healthier and solvent lifestyle, by providing
temporary “bridge” cars to help clients with transportation needs while they progress through the
program, and ultimately, by helping clients obtain a new car, clients’ mental stress declines and health
overall improves.
General health. The Client Survey asked respondents about their perceptions of their own general
physical and mental health. When asked about their general health (whether it was excellent, very
good, good, fair, or poor), there were no statistically significant differences in the Completed Clients’
responses over time although the trend was positive.
Figure 15. “Would you say your general health is: poor, fair, good, very good, or excellent?”
Comparison of client responses to three surveys.
100%
80%
60%
Enrollment (n=64)
Completion (n=46)
40%
20%
0%
Excellent
Very good
Good
Fair
Poor
On average clients reported slightly better health by the last survey in comparison to the
previous two surveys. Average health status on a scale of one to five was 3.66 at program
enrollment but increased to 3.74 by post-completion.
We also asked several more specific questions concerning their mental and physical health over the past
month that did produce significant findings. Two questions concerned the number of days that clients
experienced poor physical or mental health, and a third asked about the impact of these on daily
activities. The figure below shows the initial cross-sectional findings for all clients that suggest a trend.
32
Figure 16. Number of days of poor physical health and mental health: comparison of client responses
to three surveys.
8
7
6
5
4
3
2
1
0
Enrollment (n=64)
Completion (n=46)
Days of poor physical
health
Days of poor mental
health
Days poor physical or
mental health prevented
usual activities
For all three indicators of health, clients reported fewer average days with poor mental or
physical health in the post-completion, six months or more after they had completed the
program, than either at enrollment or at program completion.
Clients reported more days affected by poor mental than physical health in all three surveys. At
enrollment clients reported an average of 7 days of poor mental health and 4 days when
physical health was not good. Although the number of days was considerably lower at the last
survey, the difference between mental and physical health days remained.
When these indicators were analyzed using matched samples, several of these findings were statistically
significant.
Figure 17. “How many days during the past 30 days was your physical health not good?” Comparison
of the average number of days for Completed Clients with Non-Client and Withdrawn Client groups.
14
Completed Client
Enrollment to
Completion (n=26)
12
Average number of days 10
reported by participants
Completed Client
Enrollment to PostCompletion (n=38)*
8
6
Withdrawn Client T1T2 (n=47)
4
2
Non-Client T1-T2
(n=21)
0
Enrollment/T1
Completion/T2
Post-Completion
Completed Clients reported significant declines in the average number of days when their
physical health was not good* between the time they enrolled and at post-completion.1
*
1
p < .05
33
The average number of days of poor physical health declined by 60% from 4.18 to 1.68.
Neither comparison group showed any significant change over time.
The difference in days of poor physical health reported between enrollment and program completion
was not statistically significant. We note this as a reminder that in these and other analyses of
Completed Clients over time, the enrollment-program completion and enrollment-post-completion
analyses describe two different samples of Completed Clients in which some but not all of the clients
overlap both groups. In fact, for this question concerning physical health, the pattern for the
enrollment-program completion Completed Client group more closely resembles that of the two
comparison groups than the other enrollment-post-completion Completed Clients, a finding that raises a
question about attributing to the program the physical health changes reported.
The results for the similar question about mental health are clearer.
Figure 18. “How many days in the past 30 days was your mental health not good? Comparison of
average number of days for Completed Clients with Non-Client and Withdrawn Client groups.
14
Completed Client
Enrollment to
Completion (n=26)
12
10
Average number of days 8
reported by participants
Completed Client
Enrollment to PostCompletion
(n=38)*
6
4
Withdrawn Client
T1 to T2 (n=48)
2
0
Enrollment/T1
Completion/T2
Post-Completion
The number of days when Completed Clients’ mental health was not good was significantly
lower after they had completed the program and been out for six months or more than it had
been at enrollment.*2
The average number of days of poor mental health declined by 52%, from 6.58 to 3.18.
Withdrawn Clients reported significantly more days of poor mental health than Completed
Clients at Time 2.*3
Neither comparison group showed any significant change over time.
Again there were no statistically significant changes for either days of physical health or days of mental
health from program enrollment to program completion for either the Completed Clients or either of
the comparison groups. However, for the Completed Clients during this period the trend in mental
*
p < .05
*
p < .05
3
2
34
health is the reverse of the trend for physical health with the average number of poor mental health
days declining from 8.0 to 5.73.
The third question in the general health series asked for the number of days in the past month in which
physical or mental health prevented respondents from doing their usual activities such as work, school
or recreational activities. The analysis found no significant differences over time. However, there were
significant differences between Completed Clients and Non-Clients on all of these items.
At Time 1, Non-Clients reported significantly more days of poor physical health,*** poor
mental health,*** and more days that poor mental and physical health kept them from their
usual activities** as well as poorer overall health* than Completed Clients. Significant
differences held at Time 2 for all except days of poor mental health.4
Financial and other stress. In focus groups conducted with former clients who had completed the
program from several months to several years before, participants provided anecdotes supporting More
Than Wheels’ belief in its impact on mental stress. Participants reported the following long-term
impacts of the program:
Reduced stress and peace of mind: phrases used repeatedly by participants to describe the
change in mental state they experienced as a result of gaining control over their finances and
having a reliable car that they did not have to constantly worry would break down, particularly
on the isolated rural back roads that many must travel. For most of the participants, this was
the most important impact of the program.
Increased self-respect, self-esteem and pride as a result of successful completion of the program
and taking control over their financial circumstances.
Increased respect from others as a result of owning a new car rather than an old used car.
Feelings of independence and freedom to go wherever they wanted, visiting old friends or
relatives who live at a distance, spontaneously traveling to museums and places never seen
before.
Satisfaction that they were making a contribution to the environment with a car that pollutes
less and uses less fuel.
The Client Survey was an opportunity to investigate some of these attitudes and feelings over time. The
questionnaire includes a series of seven questions that focused on specific aspects of mental health,
stress related to financial problems. The questions ask how often respondents experienced certain
feelings of stress during the last 30 days, particularly around money problems. Five of the items were
questions about negative feelings, two were about positive feelings.
***
p < .001
p < .01
4
**
35
Figure 19. Percentage of respondents answering “fairly often” or “very often” to statements about
how often they felt stress, particularly related to money, in the past 30 days. Comparison of client
responses in three surveys.
100%
80%
60%
Enrollment (n=64)
40%
Completion (n=46)
20%
Post-Completion
(n=37)
0%
Unable to Difficulties Little control Helpless
Little you Things were Confident
control piling up so
over
dealing with could do to going your about ability
important high you
financial
money
change
way
to handle
things in life could not
things
problems
money
personal
overcome
issues
problems
them
For the five negative statements a higher percentage of clients in the enrollment survey than in
post-completion survey responded that they felt this way fairly or very often. The two positive
statements showed an increase from enrollment to post-completion.
For the three statements referencing money, respondents reported more stress than for the
other more general statements.
When these questions were analyzed with matched groups of Completed Clients for statistically
significant change over time,
Completed Clients reported significant positive changes in all seven stress-related items
between enrollment and post-completion.5
The strongest evidence was in two items in which there were significant improvements for the matched
groups over both time periods: Completed Clients with data matched for enrollment to program
completion; and Completed Clients with data matched for enrollment to post-completion. The
following series of figures show these findings.
5
36
Figure 20. In the past 30 days: “How often did you feel like things were going your way?” Comparison
of the average responses of Completed Clients over time.
5
Completed Client
Enrollment to
Completion
(n=27)**
4
5=Very often
4=Fairly often
3=Sometimes 3
2=Almost never
1=Never
2
1
Enrollment
Completion
Post-Completion
Completed Client
Enrollment to
Post-Completion
(n=39)**
Completed clients felt things were going their way significantly more often at program
completion than they had at program enrollment** with a 17% increase in the average.
Over the longer period, between enrollment and post-completion, the average increased 19%.
Of all the significant findings in this series of seven questions on financial and other stress, this
increase had the largest effect size.**6, 7
The percentage of Completed Clients reporting that they felt like things were going their way
fairly or very often increased from 35% at enrollment to 62% at post-completion.
Figure 21. In the past 30 days: “How often did you feel that difficulties were piling up so high that
you could not overcome them?” Comparison of the average responses of Completed Clients over
time.
5
Completed Client
Enrollment to
Completion (n=26)*
4
5=Very often
4=Fairly often
3=Sometimes 3
2=Almost never
1=Never
2
1
Enrollment
Completion
Post-Completion
Completed Client
(n=38)**
How often Completed Clients felt that difficulties were piling up too high decreased
significantly* between enrollment and program completion,8 with the average declining 21%
by completion.
This average declined even more, by 26% over the longer period, between enrollment and
post-completion, six months or more after program completion.
** p < .01
6
Cohen’s d=.819
7
*
p < .05
8
Technical appendix Q37
37
The percentage of Completed Clients answering fairly or very often dropped from 24% at
enrollment to 5% at post-completion.
For the remaining five of the seven stress items, Completed Clients did not demonstrate statistically
significant change while they were in the program, between enrollment and program completion.
However, between enrollment and post-completion, statistically significant change was found for all
five. Although not significant, the findings for enrollment to program completion show the same trend
as the findings for the longer time period.
Figure 22. In the past 30 days: “How often did you feel unable to control the important things in
life?” Comparison of the average responses of Completed Clients over time.
5
Completed Client
Enrollment to
Completion
(n=27)
4
5=Very often
4-Fairly often
3=Sometimes 3
2=Almost never
1=Never
2
1
Enrollment
Completion
Post-Completion
Completed Client
Enrollment to
Post-Completion
(n=39)**
Between enrollment and post-completion, Completed Clients reported that on average they
felt unable to control the important things in life less often,** a decline of 29%.9
At enrollment, 28% of Completed Clients responded that they fairly or very often felt unable
to control the important things in life; at post-completion, 8% responded that way.
The following show the findings for the three questions about stress related to money.
Figure 23. In the past 30 days, “How often did you feel little control over financial things?”
Comparison of the average responses of Completed Clients over time.
5
Completed Client
Enrollment to
Completion
(n=27)
4
5=Very often
4:Fairly often
3=Sometimes 3
2=Almost never
1=Never
2
1
Enrollment
Completion
Post-Completion
Completed Client
Enrollment to
Post-Completion
(n=37)**
The average of how often Completed Clients said they felt little control over financial things
declined significantly, by 19%, from enrollment to post-completion.**10
**
p < .01
Technical appendix Q31
**
p < .01
10
9
38
At enrollment, 40% of Completed Clients fairly or very often felt little control over financial
things, a percentage that dropped to 25% at program completion.
Figure 24. In the past 30 days: “How often did you feel helpless dealing with money problems?”
Comparison of the average responses of Completed Clients over time.
5
Completed Client
Enrollment to
Completion (n=26)
4
5=Very often
4=Fairly often
3=Sometimes 3
2=Almost never
1=Never
Completed Client
(n=39)*
2
1
Enrollment
Completion
Post-Completion
The average of how often Completed Clients felt helpless dealing with money problems
declined significantly, by 22%, between enrollment and post-completion.*11
At enrollment, 31% of Completed Clients fairly or very often felt helpless dealing with money
problems but by program completion that percentage had dropped to 10%.
Figure 25. In the past 30 days: “How often did you feel that there is little that you could do to change
many of the money issues in your life?” Comparison of the average responses of Completed Clients
over time.
5
Completed Client
Enrollment to
Completion (n=26)
4
5=Very often
4=Fairly often
3=Sometimes 3
2=Almost never
1=Never
Completed Client
Enrollment to
Post-Completion
(n=37)*
2
1
Enrollment
Completion
Post-Completion
The average of how often Completed Clients reported feeling that there was little they could
do to change many of the money issues in their life decreased significantly, by 15%, from
enrollment to post-completion.*12
*
p < .05
*
p < .05
12
11
39
At enrollment, 30% of Completed Clients reported feeling this way fairly or very often,
dropping to 20% at post-completion.
Two questions in this series of mental stress items ask respondents about feelings of confidence.
Figure 26. In the past 30 days: “How often did you feel confident about your ability to handle
personal problems?” Comparison of the average responses of Completed Clients over time.
5
Completed Client
Enrollment to
Completion
(n=27)
4
5=Very often
4=Fairly often
3+Sometimes 3
2=Almost never
1=Never
Completed Client
Enrollment to
Post-Completion
(n=39)*
2
1
Enrollment
Completion
Post-Completion
At post-completion the average of how often Completed Clients felt confident about their
ability to handle personal problems had increased significantly over enrollment, by 8%.*13
At enrollment, 63% of Completed Clients said they fairly or very often felt confident about
their ability to handle personal problems. At post-completion, this percentage increased to
86%.
For all seven of stress-related items, there were no statistically significant changes found over time for
the Non-Client or Withdrawn Client comparison groups except for one. Between Time 1 and Time 2
Non-Clients also experienced a significant decline in the percentage that felt helpless. This finding could
raise a question about the impact of the program for this one item but that this item was not also
statistically significant for the Withdrawn Clients suggests otherwise.
In another set of comparisons, Completed Clients and Withdrawn Clients reported similar levels of
feelings about all the items at Time 1, suggesting similarity of the two groups at enrollment. However,
for three of the items at Time 2, there were significant differences between Completed Clients and
Withdrawn Clients, evidence of program impact on at least some aspects of clients’ stress.
*
p < .05
13
40
Figure 27. In the past 30 days: “How often did you feel unable to control important things in your
life?” Comparison of average responses of Completed Clients and Withdrawn Client group over time.
5
Completed Client
Enrollment to
Completion (n=27)
4
5=Very often
4-Fairly often
3=Sometimes 3
2=Almost never
1=Never
Completed Client
(n=39)**
2
Withdrawn Client
Time 1 to Time 2
(n=46)
1
Enrollment/T1
Completion/T2
Post-Completion
At Time 2, Completed Clients felt unable to control the important things in life significantly
less often than Withdrawn Clients with an average response of 2.63, 14% lower than the
Withdrawn Clients’ average response of 3.06.*14
Analyses of two other stress items found similar results.
Figure 28. In the past 30 days: “How often did you feel confident about your ability to handle
personal problems?” Comparison of the average responses of Completed Clients and the Withdrawn
Client group over time.
5
Completed Client
Enrollment to
Completion (n=27)
4
5=Very often
4=Fairly often
3+Sometimes 3
2=Almost never
1=Never
Completed Client
Enrollment to PostCompletion (n=39)*
2
1
Enrollment/T1
Completion/T2
Post-Completion
Withdrawn Client
Time 1 to Time 2
(n=46)
At Time 2, Completed Clients felt confident about their ability to handle personal problems
significantly more often than Withdrawn Clients** with an average response of 3.98 compared
to the Withdrawn Clients’ average response of 3.45, 15% higher.15
*
p < .05
**
p < .01
15
14
41
Figure 29. In the past 30 days: “How often did you feel things were going your way?” Comparison of
average response of Completed Clients with the Withdrawn Client group over time.
5
Completed Client
Enrollment to
Completion
(n=27)**
4
5=Very often
4=Fairly often
3=Sometimes 3
2=Almost never
1=Never
2
Completed Client
(n=39)**
1
Withdrawn Client
Time 1 to Time 2
(n=47)
Enrollment/T1
Completion/T2
Post-Completion
At Time 2 Completed Clients felt things were going their way significantly more often than
Withdrawn Clients* with an average response of 3.63, 15% higher than the 3.17 average
response for Withdrawn Clients.16
The change from enrollment to program completion for the Completed Clients was large enough to
attain statistical significance only for the third item, how often clients felt things were going their way.
However, on all three, the trend was positive, and the differences between Completed Clients and
Withdrawn Clients at Time 2 were statistically significant, suggesting that More Than Wheels had an
impact on clients’ perceptions of their sense of control, their confidence, and their attitude toward their
circumstances. The statistically significant direction of change shown in the Completed Clients from
enrollment to post-completion suggests that these impacts may have extended over time, well beyond
program completion.
For the remaining four items, we did not find any other statistically significant comparisons that would
bolster the evidence for the program as agent of change. Even so, the significant changes found for the
Completed Clients between enrollment and program completion, and between enrollment and postcompletion, combined with the evidence of impact for three of the items, suggest that with larger
samples of Withdrawn Clients at Time 2 and Time 3, we might find impacts for the others.
*
p < .05
16
42
6.3 Health-Access Related Outcomes
More Than Wheels believes that by participating in the program clients’ ability to access health services
when needed improves. Clients’ increased ability to manage their finances is also believed to make it
easier for clients to obtain health insurance, either from new employment or from the increased ability
to buy it themselves. Obtaining reliable transportation also helps clients get to appointments with
health care providers, appointments they might not make if they were without transportation or had to
depend on an old, unreliable car.
Health insurance. Being without health insurance can be a major barrier to obtaining health care. In
the Client Survey, clients were asked a series of questions about their participation in health insurance
to learn the extent of this issue for clients and if, during their time in the program, more were able to
obtain health insurance.
At enrollment 82% of all clients reported having some kind of health insurance. This percentage
increased to 91% at post-completion, at least six months after program completion.
Figure 30. Percentage of all clients reporting health insurance coverage by type. Comparison of
clients in three surveys.
100%
80%
60%
40%
Enrollment (n=53)
20%
Completion (n=38)
0%
Private
Private Medicaid or Medicare Military or
insurance insurance other state
veteran
through
you
medical
coverage
employer or purchased assistance
union
yourself
Other
Types of health insurance remained fairly stable over time aside from a slight increase military
or veteran’s coverage.
We also asked clients specifically about being without health insurance.
At enrollment, 20% of clients reported there were times when they had not had health
insurance during the previous twelve months.
At program completion, the percentage of clients reporting times without insurance during the
previous six months, when they were most likely enrolled in the program, dropped to 10%.
However, the percentage increased to 15% among those responding post-completion.
We also asked those clients who had children if their children were currently covered by health
insurance.
In the enrollment survey 85% of clients reported that their children under 18 were covered by
health insurance. At post-completion, 100% of clients reported that their children were insured.
43
Access to Medical Care. We also asked clients about accessing medical care for themselves and their
children. First we asked if there was any time during the past year when they or their children did not
get medical care they thought they needed.
At enrollment, 16% of clients reported not getting needed medical care for themselves or their
children. At post-completion, less than 10% of clients reported not getting the medical care
they or their children needed.
Next we asked whether they postponed medical care they thought they needed for themselves of for
their child and found a similar pattern.
At enrollment, 25% of clients said they postponed getting needed medical care for themselves
or their children. This percentage dropped to 16% at post-completion.
These are medical care trends in the desired direction, suggesting improved health access outcomes.
There is also evidence of possible program impact on health access, as fewer Completed Clients
reported not accessing needed health care than Withdrawn Clients at Time 2.
Figure 31. Did not get or postponed needed medical care. Comparison of Completed Clients and
Withdrawn Clients at Time 2.
100%
80%
60%
Completed Clients (n=46)
40%
Withdrawn Clients (n=52)
20%
0%
Did not get needed medical care**
Postponed medical care
At Time 2, Withdrawn Clients were significantly more likely to report not getting needed
medical care for themselves or their children,** a difference that was not significant at Time
1.17
Also at Time 2, although the difference was not statistically significant, Withdrawn Clients were
more likely to report postponing needed medical care for themselves or their children.
We followed up with questions about known barriers to health care, including cost, provider refusal to
accept health insurance, inability to obtain an appointment soon enough, inability to get to the doctor’s
office or clinic when it was open, the length of time to get to the doctor’s office or clinic, and lack of
transportation. With a few exceptions reported below, none of these showed any statistically
significant differences. The figure below shows the results of cross-sectional analyses of the barriers to
medical care asked in the Client Survey.
**
17
p < .01
44
Figure 32. Percentage of clients reporting postponing medical care due to various barriers.
Comparison of clients in three surveys.
120%
100%
80%
Enrollment
(n=22)
60%
Completion (n=8)
40%
20%
Post-Completion
(n=4)
0%
Worried about Insurance not Could not get Could not get Takes too long
No
cost
accepted
appt
there when to get there transportation
open
The cost of the medical care was the primary reason given for postponing medical care in all
three surveys. Among those who reported that they experienced barriers to health care, at
enrollment 68% worried about medical care costs, a worry that rose to 100% at the program
completion and then decreased to 75% at post-completion, still higher than at enrollment.
Lacking a means of transportation to get to doctors’ appointments and not being able to get
appointments were the second most frequent responses at enrollment at 23%. Reports of these
two barriers decreased to 0% at post-completion.
In several cross-sectional analyses of transportation as a barrier to health access, Completed Clients and
Withdrawn Clients, who were similar at enrollment, reported significantly different experiences of the
impact of having a reliable car at Time 2. The following describes the results of the first analysis.
At Time 2, Withdrawn Clients were significantly more likely to report postponing needed
medical care due to lack of transportation (12%) than Completed Clients (0%), none of whom
reported this barrier.*18
In a similar analysis the comparison concerned impact of car reliability on doctor’s appointments.
Figure 33. “During the past six months, how much did lack of a reliable car affect how you traveled to
doctors’ appointments?” Comparison of average responses of Completed Clients with Withdrawn
Clients at Time 2.
2
2=A lot
1=Some 1
0=Not at all
0
*
p < .05
18
45
At Time 2, Withdrawn Clients reported that lack of reliable transportation affected their
ability to get to doctor’s appointments significantly more than Completed Clients.*19
Over 50% of Completed Clients reported that lack of reliable transportation did not affect
their doctor’s appointments at all.
The difference could be due to the More Than Wheels Bridge program. When we asked clients how
they traveled to doctor’s appointments, Completed and Withdrawn Clients responded differently.
Figure 34. “During the past six months, how did you most often travel to doctors’ appointments?”
Comparison of average responses of Completed Clients with Withdrawn Client group at Time 2.
100%
80%
60%
40%
20%
0%
Car that you own
Car owned by someone
else
Other/NA
Completed Clients and Withdrawn Clients differed significantly in how they said they traveled
to doctors’ appointments.*** 20 Withdrawn Clients were more likely to rely on their own car
(58%); Completed Clients were more likely to rely on another source of transportation (56%),
primarily More Than Wheels Bridge Program cars (38%).
From the focus groups we learned that most cars owned by clients before enrollment were not reliable.
However, clients who stayed in the program had access to the Bridge program and eventually to new
cars. The figure below shows Completed Clients’ responses to this question for each of the surveys.
Figure 35. “Over the past six months, how did you most often travel to doctor’s appointments?”
Comparison of Completed Clients over time.21
100%
80%
60%
A car that you own
40%
Bridge car
20%
Other/NA
0%
Enrollment (no data)
Completion (n=45)
*
p < .05
***
p < .001
20
21
This item was not included in the Time 1 Enrollment survey until later in data collection. Insufficient data is available for Completed Clients.
19
46
At Completion, 38% of clients said they were using a Bridge car to get to doctors’ appointments.
This percentage declined at Post-Completion, after purchase of a new car. By this point, 91% of
Completed Clients reported that they were using their own car to get to doctors’ appointments.
The evidence presented in this section suggests that clients who completed the More Than Wheels
program were more likely than the Withdrawn Clients, those who left the program early, to overcome
the transportation barrier to obtaining medical care, and this is supported by comments made by the
focus groups:
Former clients reported that between the Bridge Program and eventual purchase of a new car,
the More Than Wheels program made it easier for them to travel to hospitals and doctors
outside the state for their own or their relatives’ medical treatment, particularly to the large
medical centers in the Boston and Worcester, Massachusetts areas.
47
6.4. Food Insecurity Outcomes
Poverty and unemployment are the most often cited causes of food insecurity, i.e., lacking enough food
to eat. However, recent research also links food insecurity to having inadequate access to healthy food
sources because of lack of transportation to retail food outlets that sell fresh food such as
supermarkets1. Participation in More Than Wheels potentially addresses this problem two ways. First,
the program teaches clients ways to save money on food through the Financial Fitness classes and
counsels them on budgeting and better ways to manage their money thus increasing their financial
ability to purchase food. Second, More Than Wheels helps clients obtain cars, increasing their
transportation access to stores that carry healthy food.
In the Client Survey we asked respondents a series of questions about their diet and food security.
Questions included a self-rating of overall diet, questions designed to identify food insecurity or lack of
access to healthy foods, and questions about respondents’ use of emergency food, the Supplemental
Nutrition Assistance Program (SNAP) and the federal school breakfast and lunch programs. In all but a
few notable exceptions mentioned below, we found no statistically significant changes over time for
clients in these areas. However, there were some non-significant trends that merit describing.
The first question asked respondents to characterize their overall diet as poor, fair, good, very good, or
excellent.
Figure 36. “During the past six months, in general, how healthy was your overall diet or the food that
you eat?” Comparison of client responses in three surveys.
50%
40%
30%
Enrollment (n=62)
Completion (n=46)
20%
10%
0%
Excellent
Very good
Good
Fair
Poor
Across all three surveys clients rated their own diet an average of 3.15 to 3.40, between good
(3) and very good (4).
Clients also responded to a set of questions about their food insecurity and their access to food in the
household.
1
Ver Ploeg, M., et al. 2009. Food - Measuring and Understanding Food Deserts and Their Consequences: Report to Congress. Washington, DC:
Economic Research Service, USDA. http://www.ers.usda.gov/publications/ap/ap036/
48
Figure 37. Percentage of clients responding either “always true” or “sometimes true” to statements
about food insecurity. Comparison of client responses in three surveys. 2
100%
80%
Enrollment (n=64)
60%
40%
Completion (n=46)
20%
0%
Worried food
Food did not last Could not afford to Did not have a car
would run out
and I did not have eat balanced meals
or other
before I got money money for more
transportation to
to buy more
the store to buy
food†
Post-Completion
(n=43)
Clients’ worries about food were high across the first three questions in the enrollment survey.
However, fewer reported these concerns by program completion and even fewer in the postcompletion period six months or more after they left the program. Nonetheless, 20-30% of
clients were still concerned about having enough money for food.
Having transportation available to buy food was an item added to the Client Survey later in the
evaluation and was completed only by respondents to the Follow-up surveys.
Fifteen percent of clients at program completion said having transportation available to buy
food was sometimes or always a problem. No clients at the post-completion follow-up said this
was sometimes or always true.
Clients who had children were also asked two other related food insecurity questions.
Figure 38. Percentage of clients with children who responded to two questions about food insecurity.
Comparison of client responses in three surveys.
50%
40%
Enrollment (n=32)
30%
20%
Completion (n=22)
10%
Post-Completion
(n=15)
0%
Could not afford to feed children
balanced meals
Children were not eating enough
because I could not afford enough food
More than one-quarter (28%) of clients at enrollment said they “couldn’t feed the children a
balanced meal because I couldn’t afford that,” declining to 7% at post-completion.
Fewer clients (19%) at enrollment said “The children were not eating enough because I just
couldn’t afford enough food,” a percentage that also dropped to 7% at post-completion.
2
Question not asked at enrollment.
49
Participants were asked several questions concerning their participation in programs designed to
alleviate food insecurity: emergency food services from a church, food pantry, food bank, of soup
kitchen; food stamps (Supplemental Nutrition Assistance Program or SNAP); and the federal schoolbased Free and Reduced Price Breakfast and Lunch programs.
Figure 39. “In the past six months, did you ever get emergency food from a church, a food pantry, or a
food bank, or eat in a soup kitchen?” Comparison of Completed Clients and Non-Clients at Time 1 and
Time 2.
50%
40%
30%
Completed Clients
(n=61/46)
20%
Non-Clients (n=97/31)
10%
0%
Enrollment/T1
Completion/T2
More than a quarter (30%) of Completed Clients had obtained emergency food in the previous
12 months, prior to enrollment. After the program, 17% of Completed Clients reported using
these services in the last six months, and this percentage decreased to 9% post-completion.
At Time 1, the percentage of participants in the Non-Client comparison group (40%) reporting
using these services was higher than Completed Clients, and that percentage stayed about the
same at Time 2 (39%) while the percentage of Completed Clients decreased.
Significantly fewer* Completed Clients than Non-Clients were using emergency food services
at Completion/Time 2.
A similar result was found for a comparison with Withdrawn Clients in the use of SNAP.
Figure 40. “In the past six months, did you or anyone in your household use food stamps
(Supplemental Nutrition Assistance Program) to buy food?” Comparison of Completed and
Withdrawn Clients at Time 2.
50%
40%
30%
20%
10%
0%
At Time 2, significantly fewer** Completed Clients than Withdrawn Clients reported that they
or members of their household were using SNAP benefits to buy food.3
*
p < .05
50
Because the evaluation began just as the economy slid into a major recession, we expected that
utilization of emergency food services would increase among More Than Wheels clients, as it has for
most lower income and working class groups. However, the trends reported by Completed Clients, and
particularly the evidence of statistically significant differences with the comparison groups, suggest
otherwise.
Figure 41. Completed Clients utilization of emergency and supplemental food programs at three
points in time.
100%
80%
60%
Enrollment
40%
Completion
20%
Post-Completion
0%
Got emergency food from Household member used Children eligible to receive
church, food pantry, food
food stamps (SNAP)
free or reduced price
bank, or soup kitchen
lunch
Although Completed Clients’ use of SNAP benefits (over 50%) and their children’s eligibility for
free and reduced price lunch (under 20%) remained fairly stable, their use of emergency food
services decreased by two-thirds over time from 30% at enrollment, to 17% at program
completion, to 9% and follow-up.
The results of this series of analyses suggest that over time clients improved their management of their
limited food budgets, which is an emphasis of the More Than Wheels’ Fin Fit financial fitness course.
**
p < .01
This item was added to the survey late in data collection. Data are not available for an analysis of the same comparison at Time 1 or for
change over time within groups.
3
51
6.5. Transportation Outcomes
Because the mission of More Than Wheels is to use the “car buying process to catalyze lasting change,
financial stability and control,” it was necessary for our evaluation to better understand the importance
of the car to clients as a motivation for change. Most of the clients in this evaluation lived outside of
metropolitan Boston and had few options other than a car for transportation. The evaluation focused
on their reasons for travel, mode of travel to different types of activities and concerns about the
reliability of their car.
Clients’ dependence on cars. The Client Survey asks clients why they needed a car.
Almost two-thirds (64%) said they needed a car to travel to their job; the second most cited
reason was to transport family members (40%).
To find out the extent of their dependence on cars, clients were asked a series of questions about how
they traveled to a variety of everyday activities: a car owned by someone else, public transportation,
bicycle or walking, or a used car supplied by More Than Wheels through their Bridge program as
temporary transportation until they could obtain their new car. The activities included a job, to buy
groceries and other errands, to get to doctor and other types of appointments, to transport family
members, and to attend or participate in recreational activities. The results were very similar for all the
activities.
The following figure shows the non-significant results of cross-sectional analyses for one of the
activities: travel to a job. Public transportation, bicycle or walking, and a car owned by someone else
were combined into one category, “Other/NA,” because of the small number of responses in each,
although a car owned by someone else made up the majority of the responses.
Figure 42. “Over the past 6 months, how did you most often travel to your job?” Comparison of client
responses in three surveys.
100%
80%
60%
A car that you own
Other/NA
40%
Bridge car
20%
0%
Enrollment (n=64)
Completion(n=46)
At enrollment, 58% of Completed Clients relied more on than their own cars for transportation.
By program completion, and reflecting back on the past six months when they were still in the
program, the same percentage (35%) relied on Bridge Program cars as relied on their own cars,
suggesting that some clients opted for the more reliable car available through the Bridge
Program.
At post-completion, by which time clients had obtained a car loan and new car, 86% of clients
reported car ownership. Apparently there was some loss of cars in the time after program
52
completion since the percentage does not attain the expected 100% (clients were surveyed 6
months or more, some up to a year or more, after they had completed the program).
Two statistical analyses compared these client results to those of the comparison groups.
Figure 43. In the past 6/12 months: “How did you most often travel to your job, to buy groceries and
run errands, to recreational activities?” Comparison of responses of Completed Clients with NonClient group in Time 2 survey.
100%
90%
A car
that
you
own
80%
70%
60%
50%
Other/
NA
40%
30%
20%
Bridge
car
10%
0%
Completed
Clients (n=46)
Job
Non-Clients
(n=31)
Completed
Clients (n=46)
Non-Clients
(n=31)
Groceries and errands
Completed
Clients (n=46)
Non-Clients
(n=31)
Recreational activities
At Time 2, Non-Clients were more likely than Completed Clients to use their own cars to travel
to their job (61%),* to buy groceries and do other errands (74%),*** and for recreational
activities (58%).**
The figure below shows a similar analysis of the activities that were statistically significant for the
Completed Clients compared to Withdrawn Clients.
*
p < .05
p < .001
**
p < .01
***
53
Figure 44. In the past 6/12 months: “How often did respondents most often travel to buy groceries
and do other errands, to doctors’ appointments, to other appointments, to recreational activities in
the past six months?” Comparison of responses of Completed Clients with Withdrawn Client group in
Time 2 survey.
100%
90%
A car
that
you
own
80%
70%
60%
50%
40%
Other/
NA
30%
20%
10%
0%
Completed Withdrawn Completed Withdrawn Completed Withdrawn Completed Withdrawn
Clients
Clients
Clients
Clients
Clients
Clients
Clients
Clients
(n=46)
(n=52)
(n=46)
(n=52)
(n=46)
(n=52)
(n=46)
(n=52)
Groceries and errands Appointments (other
than doctors)
Transport family
members
Bridge
car
Recreational activities
At Time 2, Withdrawn Clients were more likely than Completed Clients to use their own cars
to travel to buy groceries and do other errands (62%),** to doctors’ appointments (58%),*** to
other appointments (60%),*** and to recreational activities (58%).**
The importance of these findings is that without the option of a bridge car available to them, it is likely
that the Non-Clients and Withdrawn Clients had to rely on their own cars for transportation, which,
given what we know about the Completed Clients’ own cars, were probably old and unreliable. With the
bridge car program, approximately one-third of the More Than Wheels clients are able to have access to
reliable, albeit used, transportation while they progress through the program, providing a critical
support to them until they can obtain their own car.
The More Than Wheels program offers clients a way to ensure continued car ownership or to achieve
car ownership for the first time. To confirm that this was also the perception of clients about the
program, we used the Client Survey to ask respondents why they applied to the More Than Wheels
program.
Almost four-fifths (79%) of clients said they applied to More Than Wheels to obtain a new or
reliable used car to replace an old car or because they did not have a car at all.
The importance of a reliable car. Clients have provided many anecdotes to More Than Wheels staff and
in the focus groups for this evaluation about the mental and financial stressors associated with owning
an unreliable car. From those focus groups we heard that the new cars obtained through More Than
Wheels resulted in
**
p < .01
p < .001
***
54
reliable transportation (“I can take it on back roads!”) that have helped clients to keep or
obtain jobs, commute to a job or school at a distance, and not constantly have to deal with car
maintenance bills.
Four questions were asked to track clients’ anxiety about the reliability of their transportation. Data for
analyses were available only for program completion and at post-completion.
Figure 45. Percentage of clients who responded “sometimes true” or “always true” to a series of
statements about transportation concerns. Comparison of client responses in two surveys.4
100%
80%
60%
Completion
(n=46)
40%
20%
0%
I was worried
My car kept breaking
I didn't have
I was worried that
whether I would
down and I didn't transportation when something would
have transportation. have the money to
I needed it.
break on my car
get it repaired.
every time I drove it.
Post-Completion
(n=43)
From the completion survey to the post-completion survey, clients’ concerns about
tranpsortation decreased substantially. The percentage of clients sometimes or always
concerned about having transportation at all dropped from 89% down to 21%.
Clients reporting sometimes or always lacking transportation when it was needed dropped to
16% at post-completion. However, about one-quarter (26%) were still anxious that something
would break every time they drove their car, even though most were presumably driving new
cars at that point.
We followed up on the earlier set of questions about mode of transportation with a question about how
much the lack of a reliable car affected how clients traveled for various activities.
Figure 46. Percentage of clients responding “Not at all” to how much the lack of a reliable car affects
how they travel. Comparison of client responses in two surveys5.
100%
80%
60%
Completion (n=45)
40%
20%
0%
Job
4
5
Groceries and
errands
Doctors'
appts.
Other appts.
Tranport
family
Recreational
activities
This item was added to the Time 1 survey late in data collection. Data for Completed Clients at Time 1 was not available.
55
Lacking a reliable car was more likely to affect clients’ mode of travel just after program
completion, when clients responding “not at all” averaged 42% across all activities.
In the post-completion survey, six months or more after program completion, over 80% said
that the lack of a reliable car did not affect their travel for five of the six types of activities.
Slightly fewer clients (74%) reported that traveling to a job was unaffected.
These results for Completed Clients were compared to the Withdrawn Client comparison group at Time
2.6 Only one of the series of activities was reported to be affected by the lack of a reliable car, clients’
travel to doctors’ appointments, a finding reported earlier in this report in the discussion on access to
medical care.
6
These items were not included in the Non-Client survey.
56
6.6. Community Involvement Outcomes
In the focus groups that we conducted with clients who had completed the program, including some
who had been out of the program for several years, clients said that:
as a result of obtaining a new and/or reliable car, they were able to visit relatives and friends in
other states, go berry picking with the family, take children and grandchildren on long-distance
outings (like a trip to Ben & Jerry’s ice cream factory) or other places impossible to visit before.
Familial, social, and community activities were described as particularly desirable longer-term outcomes
of obtaining reliable transportation and increased financial security.
The Client Survey included a set of questions asking how often respondents interacted with different
groups of people in the past year: personal contact with family member or friends; attendance at
religious services; social or community organizations; and child’s school, social or athletic activities.
In two questions clients were asked to report on how often they had personal contact with others, i.e.,
face-to-face, on the phone, by letters or email, or engaged is particular social activities during the past
six months. The following figures present the findings for each activity.
Figure 47. “How often did you have personal contact with family members who do not live with you
during the past six months?” Comparison of client responses in three surveys.
50%
40%
30%
Enrollment (n=59)
20%
Completion (n=46)
10%
0%
Never
Less than
monthly
Monthly
Weekly
Daily
A pattern emerges of more frequent contact with family members at prior to program enrollment and at
post-completion and less frequent contact in the six months prior to program completion. One possible
explanation is that in those months prior to completion the commitment required for success in More
Than Wheels takes substantial time that might otherwise be spent in social engagement.
Figure 48. "How often did you have personal contact with close personal friends during the past six
months?” Comparison of client responses in three surveys.
60%
50%
40%
Enrollment (n=63)
30%
20%
Completion (n=46)
10%
0%
Never
Less than
monthly
Monthly
Weekly
57
Daily
Overall more clients contacted close personal friends weekly except in the post-completion
period when the percentage for daily contact increased to 52%. Also at that time, percentages
reporting never or less than monthly dropped to zero, indicating a trend toward greater
frequency of contact.
The remainder of the questions concern involvement in social or community activities.
Figure 49. “How often did you attend religious services, not including weddings and funerals during
the past six months?” Comparison of client responses in three surveys.
60%
50%
40%
Enrollment (n=60)
30%
Completion (n=46)
20%
10%
0%
Never
Less than
monthly
Monthly
Weekly
Daily
Just over half (53%) of clients reported never attending religious services at enrollment.
However, by post-completion, clients reported slightly more frequent attendance, with over
one-quarter (27%) attending weekly and the percentage responding “never” declining to 44%.
Figure 50. “How often did you attend social or community organizations or groups that you are
involved in during the past six months?” Comparison of client responses in three surveys.
60%
50%
40%
Enrollment (n=63)
30%
Completion (n=46)
20%
10%
0%
Never
Less than
monthly
Monthly
Weekly
Daily
Among those who participated in social or community groups, no clear pattern emerges
although at post-completion, clients had a higher percentage for weekly (29%) attendance and
lower for no attendance (29%) than at enrollment.
58
Figure 51. “How often did you attend your child’s school, social, or athletic activities during the past
six months?” Comparison of client responses in three surveys.
60%
50%
40%
Enrollment (n=30)
30%
Completion (n=22)
20%
10%
0%
Never
Less than monthly Monthly
Weekly
Daily
Clients increased their monthly attendance at their children’s activities from 17% at enrollment
to 36% at post-completion. Daily attendance also went up from 10% to 18%. These gains were
offset by declines in less than monthly attendance and weekly attendance.
Follow-up questions were asked after each of these questions about how much effect their
transportation circumstances had on those activities: a lot, some, or none. Results are available for the
program completion survey and post-completion and are summarized in the figure below.
Figure 52. Percentage of clients reporting “a lot” or “some” effects of transportation on social and
community activities during the past six months. Comparison of client responses in two surveys.1
100%
80%
60%
40%
Completion (n=45)
20%
0%
Personal
contact with
family
Personal
contact with
friends
Attending
religious
services
Social or
community
organizations
Children's
activities
At program completion, 44% reported effects of transportation circumstances on their personal
contact friends and 36% their contact with family.
The percentage of clients reporting these effects, and effects for all five types of activities,
dropped substantially by post-completion, after clients had been out of the program for at least
six months, to below 20% (a range of 6% to 19%).
There were no significant differences across time for Completed Clients or for the comparison groups for
any of these community engagement measures except for the question concerning contact with close
friends.
1
59
Figure 53. “How often have you had personal contact with close personal friends in the past six
months?” Comparison of the average responses of Completed Clients with Withdrawn Client and
Non-Client groups over time.
4
Completed Client
Enrollment to
Completion (n=26)*
3
4=Daily
3=Weekly
2=Monthly 2
1=Less than monthly
0=Never
1
Completed Client
Enrollment to PostCompletion (n=38)
Withdrawn Client Time
1 to Time 2 (n=47)*
Non-Client Time 1 to
Time 2 (n=31)*
0
Enrollment/T1
Completion/T2
Post-Completion
Clients’ average personal contact with friends declined significantly, by 15% from enrollment
to program completion.
At enrollment, 25% of clients reported monthly or less than monthly visits to friends, 42%
visited weekly, and 33% visited daily. By program completion monthly visits had increased
(33%) as had weekly visits (44%) while daily visits declined to 24%.
Both Non-Client and Withdrawn Client comparison groups show the same statistically significant decline
from Time 1 and Time 2 as the Completed Clients so there is no apparent impact of the program.
However, the Completed Clients matched for enrollment to post-completion showed a non-significant
trend toward increasing visits over time. It is possible that by collecting more data over a longer period
of time evidence for community engagement outcomes would begin to emerge.
60
The mission of the More Than Wheels program is to promote long-lasting social change by educating
and supporting individuals struggling with poor credit and other money-related problems to find
strategies that help them regain financial stability. The program offers clients an incentive of a car loan
and new car to motivate them to stay with the program to the end. However, the goal is transformation
of people’s lives; helping needy individuals obtain new cars is simply the means to that end. Although
the program had long documented this transformation through client testimonials, in 2008 the
program’s management, with the backing of the Robert Wood Johnson Foundation, committed to a
comprehensive and rigorous evaluation of these changes both to better understand their own program
and to persuade funders, clients and other stakeholders of the value of their innovative program model.
From August 2008 to September 2011 More Than Wheels contracted the Carsey Institute of the
University of New Hampshire to collaborate with their staff on an evaluation that included process,
outcome, and impact components. In 2010 we submitted an Interim Report on the process evaluation,
focused on the implementation of More Than Wheels during the first two years of the evaluation with
some preliminary findings about client outcomes. The purpose of the present, final report is to report
on 1) the results of statistical analyses of client outcomes from three survey administrations and several
other data sources for evidence of change over time, and 2) the results of those analyses in comparison
to those of two similar groups of people for evidence that the changes found for clients are attributable
to the More Than Wheels program.
The framework guiding the outcome evaluation was based on the More Than Wheels program theory of
change developed by staff from anecdotes collected from clients, findings from an earlier Carsey
evaluation, and responses from participants in focus groups conducted during the first year of the
evaluation. This model describes clients coming into the program coping with financial stress created by
their lack of personal money management knowledge, skills, and resources. Through an intervention
consisting of training, counseling, and positive support – with the reward of a car loan and new car at
the end of the process – these clients gain the capacity to change their attitudes and behaviors
regarding money and themselves. These changes in turn, along with the availability of transportation,
increase their mobility and reduce negative stress, improving their health and quality of life.
The evaluation found support for this model.
More Than Wheels clients reported statistically significant changes in their knowledge,
attitudes and behaviors regarding money management and themselves;
clients’ experience of stress, particularly regarding money matters, declined over time, and
their health improved; and
clients’ access to reliable transportation increased, improving their mobility and access to
health providers.
We also found as a result of comparing clients with people who either had not participated in the
program or were in the program but dropped out, that
clients’ changes appear to be the result of their participation in the program.
In the following sections, we summarize these findings and conclude with our interpretation of these
results in the context of the program’s theory of change.
61
Early in the program clients were provided some training in money matters. Results of a test given to
them prior to and after the training show that, as a group,
clients who would eventually complete the program scored 55% higher at the end of the
course than at the beginning, demonstrating that their knowledge of personal financial
management had increased significantly.
In the Client Survey participants were asked to self-report on specific behaviors that were believed to be
representative of individual money management knowledge and skills learned in the program’s Financial
Fitness training and counseling. At all levels of analysis there was a consistent pattern of money
management behaviors among clients that was lower at enrollment in the program and higher after
program exposure. The strongest findings, with statistical significance and effect sizes indicative of
program-related change, were in frequency of clients’ saving and bill-paying behaviors:
Clients increased their regular savings activity significantly over time by two-thirds (66%) and
over both comparison groups.
Clients reporting that they usually or always paid their bills on time increased, on average, by
13% between enrollment and program completion and by 31% between enrollment and postcompletion. Clients were significantly more likely to say they usually or always pay their bills
on time than one of the comparison groups; there was no significant difference with the other
comparison group.
Changing both of these behaviors is critical to controlling finances and building credit, the primary
objectives of the program. Without making these changes, it would be unlikely that clients could
succeed in improving their credit ratings, obtain a car loan or buy a new car.
We found no statistical evidence that clients’ economic status changed while in the program. However,
the data showed a non-significant upward trend in clients’ monthly net work income over time. Parallel
improvements in clients’ employment status were expected but not found, although we believe that
accurate measurement of this indicator was hindered by the quality of the data available to us and the
negative trend we found may not reflect the clients’ actual circumstances.
Other trends in the clients’ money management behaviors over time were also found, although the
findings were not statistically significant. As a group and over time clients reported:
using longer-term money-building and investment strategies with the money they saved, and
using spending plans or budgets to help them manage their finances.
Clients also reported other positive changes in their financial attitudes and behaviors:
feeling more in control of their money,
working toward financial goals, and
feeling more comfortable doing business with banks and credit unions.
These trends are meaningful, demonstrating that many clients over time learned to use new financial
management strategies, think differently about money, and change how they interact with the financial
systems that can either facilitate or become barriers to a higher quality of life. The end result – and
reward – for all of these changes in behavior was
62
a statistically significant increase over time in clients’ credit scores obtained from national
credit rating services from an average of 596 to 633, an increase of approximately 37 points.
Improved credit scores are the tangible evidence that clients made substantive improvements to their
financial circumstances while in the program.
From our focus groups and the Client Survey we learned that clients come into More Than Wheels
suffering from anxiety and stress related to living continually in debt and on the edge of financial crisis.
By the time they finish the program and into the months after, our findings indicate that much of this
stress abates with clients reporting:
significant 52% decline in the number of days of poor mental health, from 6.6 days at
enrollment to 3.2 days at post-completion, six months or more after finishing the program;
and
fewer days of poor physical health, from 4.2 days at enrollment to 1.7 days at postcompletion, a significant decline of 60%.
Also over time more clients reported feeling in better health and having fewer days when poor mental
or physical health interfered with daily activities, although neither of these findings was statistically
significant.
One of the most important findings for the evaluation is that
clients who completed the program experienced significantly less stress over time, particularly
in relation to financial problems.
This reduction of stress was measured by clients’ responses to a series of seven questions. Findings for
two of these are especially strong, with both changes from enrollment to program completion and from
enrollment to post-completion, and in comparison to a group of clients who withdrew from the
program. Over time, on average,
clients felt that things were going their way more often, increasing significantly by 17%
between enrollment and program completion, and by 19% between enrollment and postcompletion.
clients felt that difficulties were piling up too high to overcome them less often, decreasing
significantly by 21% from enrollment to program completion, and by 26% between enrollment
and post-completion.
The remaining five questions were also statistically significant for client change over time but not in
comparison to the other two groups. Over time, from enrollment to post-completion, clients who
completed the program reported, on average,
less often feeling unable to control important things in their life (declined significantly by
29%),
less often feeling helpless dealing with money problems (declined significantly by 22%),
less often feeling little control over financial things (declined significantly by 19%), and
less often feeling that there was little they could do to change many of the money issues in
their life (declined significantly by 15%).
63
More clients also reported over time
more often feeling confidence in their ability to handle personal problems (increased
significantly by 8%).
The effect sizes were medium to large for all these changes, and no significant change was found over
time for either of the comparison groups on six of these seven indicators. These findings provide some
of the strongest evidence in the evaluation for program impact: by participating in More Than Wheels,
clients who completed the program experienced a reduction in mental stress and fewer days of poor
health. The results powerfully illustrate the impact that money problems have on people’s mental
health and, by association, their ability to function day to day.
One of the characteristics of the More Than Wheels clientele is that many often live on the edge, day to
day, without a job that has benefits or without sufficient financial stability to pay the up-front costs of
medical care. Living with an old, unreliable car or without transportation at all can also limit access to
health care services.
The evaluation’s survey included several questions about barriers to health care, particularly health
insurance and access to transportation. Several of these produced findings that were not significant, but
suggest trends to watch:
the percentage of clients’ children with health insurance increased over time, and
by the end of clients’ participation in the program, fewer reported that they postponed or did
no obtain needed medical care for themselves or their children.
In the focus groups, reliable transportation was often mentioned as a serious barrier to obtaining
medical care. However, in the survey,
although most clients at enrollment worried about lacking a reliable car, by program completion
and in the months after, clients reported not worrying about this at all.
In fact, by the time of the Time 2 survey or program completion, significantly
more comparison group participants than Completed Clients said that lack of reliable
transportation was affecting their ability to get to doctor’s appointments; and
more comparison group participants than Completed Clients said they were postponing
needed medical care due to lack of transportation.
This evaluation began just before the economy began its plunge into the Great Recession in Fall 2008.
Food insecurity among the lower income groups, particularly the working poor who have lost their jobs,
has been a serious concern. Because the More Than Wheels program includes strategies that clients can
use to improve their savings and consumer behaviors to ensure they are eating enough and healthily,
the evaluation included a series of three questions in the survey concerning food insecurity outcomes.
Although not significant, results suggest a trend that indicates food insecurity was an issue for clients,
but became less so over time.
64
Clients’ worries about not being able to afford feeding their children declined by 76% and that
their children not eating enough food because they could not afford it declined by 64% from
enrollment to the third survey, after they had left the program.
Nationally utilization of emergency food programs has increased dramatically, particularly by the
working poor and lower middle class populations served by More Than Wheels. The Client Survey was
in a position to capture some of these changes and surprisingly, the pattern found for clients was the
opposite of the national trend.
Although clients’ use of Supplemental Nutrition Assistance Program (SNAP) benefits and their
children’s participation in the federal free and reduced price school lunch program remained
fairly stable, their use of emergency food services decreased over time, suggesting that clients
had learned better management of their limited food budgets.
Clients’ reports of using emergency food services from a church, food pantry, or a food bank, or
having eaten in a soup kitchen were lower at program completion than when they enrolled, and
even lower at post-completion, after leaving the program.
At enrollment more comparison group participants than Completed Clients reported using
emergency food services. However,
at Time 2, program completion, the percentage of comparison group participants did not
change while the percentage of Completed Clients, by contrast, declined and was significantly
lower than the percentage of the comparison group participants using the emergency food
services.
Also by Time 2, program completion, a significantly lower percentage of Completed Clients
than in the comparison group reported that they or members of their household were using
SNAP benefits to buy food.
Other trends found indicate a decrease in clients’ need for these services, for example, clients worries
about not being able to afford feeding their children and having access to transportation to buy food
declined over time.
All of these findings suggest that through its Financial Fitness class and ongoing counseling More Than
Wheels staff had some impact on their clients’ food security. A typical scenario for people dependent
on supplementary food programs is that their resources to buy food run out at the end of the month.
Although we cannot say which part of the More Than Wheels program helped to reduce clients’
dependency on these food programs, it is likely that having increased financial stability and better
strategies for purchasing and consuming food probably helped them become more self-sufficient and
have adequate money to purchase their own food throughout the month.
Helping clients obtain reliable transportation is at the heart of the More Than Wheels’ mission. From
our focus groups we know that for many clients it is difficult to disentangle the relationship between
their financial situation and having a no car or an old one prone to breakdowns. Having transportation
is vital to functioning in the workplace, at home, and in the community and going without it puts all of
these in jeopardy. Financial instability leaves little cash available to repair or replace a car,
compounding this risk.
Like most people, More Than Wheels clients were very dependent on their cars, relying on them for
travel to their jobs, to run errands, and to get to appointments and other activities, yet did not have
65
adequate transportation when they first enrolled in the program. The More Than Wheels Bridge
program met this need for many clients took while in the program and after finishing the program most
purchased their own cars, presumably through More Than Wheels. We found that this change in
transportation circumstances had an important, if not statistically significant result:
At enrollment most clients worried about lacking a reliable car, but by program completion and
in the months thereafter, most clients reported not worrying about this at all.
By contrast, our comparison groups during this same time period, without the Bridge cars or new cars
available to them through More Than Wheels, were significantly more likely to be using their own cars
and to report lack of a reliable car, at least in the context of traveling to doctors’ appointments. These
findings suggest that obtaining either a Bridge car or new car through the program resolves a major
problem of mobility and access to services for many clients in difficult financial circumstances.
One of the longer term outcomes we explored in the evaluation was frequency of clients’ involvement
with others including contact with friends or family, attendance at religious services, social or
community organizations, and at their children’s activities.
Looking at these results statistically for change over time, the only statistically significant change we
found was
a decline in personal contact with friends from enrollment to program completion, for most
clients, although non-significant declines in engagement in other activities were found as well.
It seems possible that while in the More Than Wheels program employed clients started employment or
began working more hours in order to improve their financial circumstances, or that the time
commitment required to participate in More Than Wheels left them experiencing a temporary shortfall
of leisure time to spend on social activities. These hypotheses are supported by a non-significant but
upward trend found when these activities were analyzed over the longer period.
For most of the activities asked about, more clients reported participating in these activities
over the long-term, after they had finished the program, than were reported either at
enrollment or at program completion.
Evidence for the Model
The More Than Wheels program is an innovative model for working with a challenging population,
people coping with often severe personal financial problems. Is this model successful in changing
clients’ behaviors over time to overcome these problems? Our evaluation found strong statistically
significant longitudinal evidence for client change over time in almost all of the key outcomes measured
and evidence through comparisons with two similar groups of people that participation in the program
was the reason for the changes.
Specifically, we found evidence that the program was successful at helping clients increase their money
management knowledge and skills, and choose better strategies for saving and handling their money,
the primary objective of the intervention. In addition, we found trends in the data indicating
improvements in other aspects of clients’ financial attitudes, behaviors, and personal debt. As expected
from the model, the end result was that clients improved their credit, gained control of their personal
66
finances and maintained stability long enough to obtain a car loan and new car. We also found that for
many of the clients these behavior changes were long term, lasting well beyond the end of their
involvement with the program.
The most striking findings of the evaluation are the changes in mental and physical health reported by
clients in the surveys that were predicted in the model. Until the evaluation, these outcomes were
known only from anecdotes in which clients were adamant that gaining financial stability, including
obtaining a Bridge car and eventually a new car, were the reasons for their improved health. In the
evaluation we found significantly more clients reporting better physical and mental health, including less
stress, after participating in More Than Wheels.
With better financial management and access to a new, more reliable car, the model also predicted
several other outcomes. Trends in client responses to the three surveys conducted show a decline in
emergency food program utilization at a time when it was increasing nationally, improvements in access
to health care through improved transportation, and more involvement with family, friend, and
community activities. We did not, however, find expected changes in employment, in part because of
limitations on the data available, but perhaps also because of the timing of the evaluation during a
major economic recession. Nonetheless, that clients reported other positive outcomes that typically
require more income, for example, increased access to medical care and decreased use of emergency
food programs, is, perhaps, evidence that they were putting their newly acquired money management
skills to work, i.e., learning to get by on less.
The strongest findings in the evaluation had mostly medium or a few large effects. These were for the
two sets of individual outcomes most directly targeted by the program: better money management
behaviors and improved physical and mental health outcomes, both addressed through the Fin Fit
education course and through the extensive one-on-one counseling and support provided by program
staff. The size of the effects probably reflects the close relationship between the program and these
particular outcomes.
The results for the first of these two outcomes are not surprising. More Than Wheels has been
successfully graduating people from the program into their new cars for years, and clients could not
have achieved this without changing the way they managed their personal finances. In this sense, the
evaluation merely documented known behavior change. However, other than anecdotes, physical and
mental outcomes of the intervention had not been systematically recorded over time and it is for this
reason that these evaluation findings are particularly valuable.
Other findings of the study included non-significant trends reported for access to health care, income,
use of emergency food assistance, and involvement in community activities. These outcomes are less
proximal, i.e., linked theoretically but only indirectly to the activities of the program and thus more
difficult to measure and evaluate. Nonetheless, had there been the resources to collect data from
clients for several more years, we believe that there might have been the data available to describe the
paths and estimate the relationships between the program, the primary financial and health outcomes,
and these less direct or longer term secondary outcomes. In this sense the failure to find much evidence
for change in these other areas does not diminish the overall findings of program impact. In fact, any
significant findings in these outcomes in this evaluation are notable because they were not targeted
specifically for change by the program.
One of the next steps that the program could do and that we recommend is to conduct further research
and evaluation on the More Than Wheels model. In particular, target the secondary outcomes for
future evaluation activity to better understand the many factors involved in and that result from the
process of gaining control over one’s financial circumstances. Such an effort could involve evaluating
67
the longer term outcomes of the program on current and past clients’ employment, income, and
financial status in more diverse settings now that More Than Wheels is expanding into urban areas. The
benefit to More Than Wheels would be a deeper understanding of the program’s impacts, but also new
insights into the needs and characteristics of the consumers of its services.
Programmatic Issues
One of the challenges that More Than Wheels has had to confront is the dropout of clients before
program completion. This problem came to our attention during the process evaluation, but it became
of particular interest during the outcome evaluation, when there were enough clients in the evaluation
who had dropped out to be used as a comparison group. The issue for the outcome evaluation was the
extent of their similarity to the other clients, which was a requirement for comparison. Since both
groups had passed the extensive initial screening required to enroll in the program, it was assumed they
would be similar and, in fact, we found no statistically significant differences between the two groups at
enrollment. Nonetheless, one group succeeded in staying in the program through program completion,
our Completed Clients group, while the other group, our Withdrawn Clients group, dropped out along
the way.
More Than Wheels staff members screen for potential clients who can afford to participate and who
have the motivation, thus increasing the likelihood that they will succeed. As shown by this evaluation
and years of program graduates, this strategy is working. However, because some of the screened
people drop out and never make it to their goal, the question is: are they fundamentally different from
the clients who succeed? That was the assumption that drove the development of the program’s
screening tool. Or is the issue something about the program that is a poor match for them, or does not
meet their needs, in some crucial way? If that is the reason, then the issue may concern the program
model itself.
For any nonprofit program with a charitable mission seeking to serve and transform a needy population,
such a finding is problematic. Do you design the program around the needs of the any client who walks
through the door or do you become more selective, developing a model that is a good fit for a particular
segment of the population? The former tends to characterize a social services agency; the latter a
consumer organization or business. More Than Wheels has evolved and is continuing to evolve in its
thinking on this issue. However, this thinking will continue to be challenged, particularly as it expands
and attempts to replicate its model in other more diverse communities in other regions of the country.
We recommend further and more detailed research on the program’s clients to better understand their
circumstances both when they enroll in the program, and as they progress through it, to understand the
contextual factors at play in their decisions to stay in or drop out. This understanding could consider, at
least in the context of a research project, factors beyond the largely financial information that is
collected about clients now. For example, mental illness, substance abuse, family dysfunction, police
involvement and other challenges to economic stability not only of the applicant but among the
applicant’s family and peer group create substantial challenges to and responsibilities for clients.
Characteristics of the community influence their decisions as well, for example, the extent of economic
opportunity in the client’s community and the resources available to support clients beyond what More
Than Wheels can or wants to provide. Case studies following individuals as they move through the
program might be particularly enlightening as to the barriers and supports like these that exist both
inside and outside the program. Some client problems that would be identified would be beyond the
influence of More Than Wheels; but identifying others might lead to new programmatic strategies to
reduce program dropouts, improve services to clients, and result in valuable refinements of the program
model.
68
Evaluation Limitations
As important and meaningful as the findings of this evaluation are for the More Than Wheels program,
they need to be interpreted and publicized with some caution. As noted several times in this report and
described in more detail in the Technical Appendix, there were some limitations to the samples we
used, in the data we were able to collect, and as a result, in the analyses we were able to perform. The
consequence of these limitations for the evaluation is that they place constraints on the conclusions that
can be legitimately drawn from the study.
The most important of these conclusions is the extent to which the findings from the sample of clients in
the evaluation are generalizable to the larger population of More Than Wheels clients. Not only was the
evaluation sample both younger and less well-educated, it represented largely white clients from small
cities and rural communities in northern New England which defined the program’s population in 2008
but, with expansion of the program into cities and other states, no longer does.
A second issue concerns the similarity between the clients in the evaluation and one of the two
comparison groups, the Non-Client group. Participants in this group had lower monthly net incomes
than the clients with whom they were compared, those who successfully completed the program.
These differences are less likely to have occurred if random assignment in selecting the samples had
been a possibility, but it was not. Their impact on the findings is difficult to know. However, knowing
that there was one characteristic on which the comparison group differed indicates that there could
have been other differences as well for which we had no data. The more the group’s characteristics
diverge from those of the clients we evaluated, the less valid it is to compare the group to the clients.
Without a similar comparison group who was not exposed to the program, the case is harder to make
that the program is responsible for the changes the clients reported.
Even with these limitations, however, the evaluation shows that with at least one group of people, the
More Than Wheels program model was successful. To learn why it worked for these clients and not for
the comparison groups, and to understand the relationship between the way the program is
implemented and the outcomes that result, would require additional data collection, further analysis
and evaluation. Although we were not able to explore these relationships as closely as we had originally
intended, we believe that the findings that we did produce should reassure the More Than Wheels
board and staff. Their model for changing financial behavior and reducing clients’ financial and
transportation-related stress is an effective strategy for improving the lives of the working poor, a
segment of the population that has too often been ignored and written off. This evaluation provides
evidence that in the right program such people can succeed and go on to change their lives.
69

Bonnie CLAC Year-End Report

Transcription

Similar documents

Groveport Madison High School And Cruiser Academy OGT

SWEC`s 2008-09 Brochure - Southwest Early College

Instructions to navigate Open Enrollment in Banweb Log into

AU23

myMiami for Families

Covered California Updates - San Francisco Health Plan

Material List - DIY Wheelchair