Youth Guarantee Scheme - Final Evaluation Report

Capital South
Youth Guarantee Scheme
2014 Pilot Programme Evaluation Report
By
Ross Feeney, Director
Make Business Happen CIC
Contents
Overview.................................................................................................................................................. 2
European Context ................................................................................................................................ 2
National Context .................................................................................................................................. 3
Croydon ............................................................................................................................................... 5
Programme Overview .............................................................................................................................. 5
Project Delivery ....................................................................................................................................... 7
Project Evaluation .................................................................................................................................... 7
European Commission ..................................................................................................................... 7
Make Business Happen .................................................................................................................... 7
Feedback from Business Mentors.................................................................................................... 8
Feedback from Schools .................................................................................................................... 8
Feedback from students .................................................................................................................. 9
Feedback from Capital South project team........................................................................................ 10
Outputs & Outcomes ............................................................................................................................. 12
Outputs .............................................................................................................................................. 13
Outcomes .......................................................................................................................................... 14
Best Practice Recommendations ....................................................................................................... 16
Conclusions ............................................................................................................................................ 17
Appendices ............................................................................................................................................ 18
Evaluation Team ................................................................................................................................ 18
Methodology ..................................................................................................................................... 18
1
Overview
CapitalSouth is an education business partnership formed in 1990 to work with schools, academies,
colleges and pupil referral units in South London and Surrey. CapitalSouth has a strong reputation
for linking schools, academies and colleges with the business community and supporting young
people in their journey to work. CapitalSouth achieves its goals through a variety of programmes and
activities that develop young people’s understanding of the world of work. That includes the
enhancement of their employability skills like self-management, teamwork and communications that
make students more attractive to potential employers.
In 2012 CapitalSouth and Croydon Council were successful in bidding for funding awarded by the
European Commission to address rising levels of youth unemployment and the growing number of
young people classified as NEET (Not in Education Employment or Training).
The Youth Guarantee Scheme (YGS) pilot project was one of 18 pilot programmes implemented in
seven European nations: Ireland, Italy, Lithuania, Poland, Romania, Spain and the United Kingdom.
Croydon, home to the largest youth population in London and a high rate of youth unemployment,
was awarded a 12 month grant by the European Commission to develop proposals that support
young people in the school to work transition.
The funding was awarded to enable the delivery of a pilot initiative in the London Borough of
Croydon which sought to:
1. Bridge the gap and strengthen co-operation between employment / career guidance services
and schools so that disadvantaged young people can make informed choices about options
available to them when leaving school.
2. Improve employer engagement between public services and the private sector to (a) ensure
that employer demands for skills are met and (b) to boost employment opportunities,
apprenticeships and other work based initiatives.
The YGS project sought to build the capacity of five schools in Croydon which had a proportion of
young people at risk of becoming NEET. This capacity-building programme (developed in partnership
with employers) aimed to raise awareness of schools of employability skills and enable teachers to
adopt teaching methods that would allow them to deliver such skills / culture in the classroom.
The project also targeted support at schools to enhance their careers advice provision to enable
improved decision-making so that young people are certain, realistic and confident about
occupational goals.
European Context
The European Commission recognised in 2012 that against the backdrop of the global economic crisis
over 7.5m young European citizens were neither in employment, education or training. The socioeconomic fall-out from such a statistic was a considerable concern for the Commission – which felt
that structural reform was needed amongst member states to improve opportunities to help young
people into employment or training.
The European Youth Forum and its members called for a youth guarantee, a policy where
governments, regional authorities and public employment services, with the involvement of youth
organisations, commit to offering young people quality jobs, training, further education, internships
or apprenticeships. The Forum argued this would address the youth unemployment crisis that had
unravelled in Europe since 2008 and that had had a dramatic impact. In some cases, the Forum
argued, it has also significantly impacted on the demographic balance and economic performance of
entire regions. Despite diverging and isolated measures at national and regional level to deal with the
issue, youth unemployment had remained high.
2
The youth guarantee began to emerge in Finland and Scandinavia in the 1980s and early 1990s.
Article 9 of the Treaty on the Functioning of the European Union already committed to ensuring a
high level of employment, a guarantee of adequate social protection and a high level of education
and training. Article 9 was further enhanced with the Youth Guarantee Scheme, rooted in the Nordic
social model with a need for early intervention to address long term unemployment, both for young
people and the broader society.
To counter the issue of youth unemployment the European Commission mobilised a number of short
term relief projects involving employers, developed to provide early action to change and address the
problem. The Youth Guarantee Scheme pilot programme was one such initiative.
The youth guarantee was aimed at reducing youth unemployment on a medium to long-term basis,
and to significantly reduce the number of NEET young people. European research conducted in
2012 calculated the costs of NEETs to be €153 billion in 2011, based on state costs for paying
unemployment insurance, other welfare benefits to NEETs, the indirect costs of the loss of income
and output for the economy as a whole. The International Labour Organisation (ILO) asserted that
implementing a youth guarantee across the Eurozone would cost around €21 billion.
The Youth Employment Initiative was introduced in March 2013 with a budget of €6 billion for the
period 2014-2016. Designed to support young NEETs in European regions where youth unemployment
was above a rate of 25% (in 2012). €3 billion would be made available from a dedicated Youth
Employment budget and a further match of at least €3 billion from European Social Funding.
The Finnish Public Employment Service estimated the Youth Guarantee success rate at more than
80% in 2011. In Sweden, 46% of participants of the programme had successful outcomes. An average
investment of €7,809 per participant was usually recouped within one year. Average profitability in
the first year was €4,200 per participant (with further profitability accumulating over time).
The CapitalSouth Youth Guarantee Scheme pilot project was one of 18 pilot programmes
implemented in seven European nations: Ireland, Italy, Lithuania, Poland, Romania, Spain and the
United Kingdom. The European Commission recognised, in awarding pilot contracts that by definition
pilot projects are challenging. Ultimately the experience gained from the pilots would be expected to
provide member states of the European Union with practical recommendations for implementing
national Youth Guarantee Schemes and programming related actions under the European Social Fund
and Youth Employment Initiative.
National Context
As a result of the recommendations of the Wolf Report, the Department for Education (DFE) under the
stewardship of Michael Gove removed the statutory requirement for work-related learning in Key Stage
4 (pre 16). This had an impact on schools with many removing work experience from their curriculums.
The rationale for this was based on the assertion that almost no young people go into full-time jobs
aged 16 and the review of vocational education focusing on Post-16 meant the legal requirement
to offer 14-16 year-olds work-related learning was abolished.
The legislation was not well received by businesses; the Co nfe de rat io n o f Brit ish In dust ry (CBI)
recommended that the government reinstate the duty on schools to provide work-related learning
at Key Stage 4 and introduce a national network of local brokers to support schools in delivering it.
The CBI argued that underachievement in UK schools has a real effect on students by the time they
reach the age of 25 - with earnings around £2 less per hour for employees from a disadvantaged
3
background. Strategic efforts were needed, it was argues, to encourage young people to pursue
e i t h e r an academic or vocational route to their career.
The CBI further argued that work experience is vital and other forms of engagement need to be
developed in an imaginative and proactive manner. A renewed UK obligation rather than an EU
commitment needed developing as a policy, detailing what schools should deliver and be accountable
within the Ofsted inspection framework.
In its report ‘Time Well Spent’ the CBI noted that each year thousands of young people went on work
experience and that enhancing the employability of school leavers through work experience is a threeway partnership between the school, student and employer. The CBI defines employability as “a set of
attributes, skills and knowledge that all labour market participants should possess to ensure they have
the capability of being effective in the workplace – to the benefit of themselves, their employer and the
wider economy”.
In strategically defining the type of interaction with employers, creating synergy between academic and
enterprise environments the CBI argues the importance of the school setting the tone for work
experience placements, briefing the student and setting objectives. Raising expectations of the
employer, ensuring the continuity of education is vital. Other forms of engagement need to be
developed to complement work experience programmes and curriculum plans for 14-18 year olds in
education.
To aid the employer the CBI identifies seven competencies and defines attributes of the ‘can-do’
approach required in the key foundations for work experience.
•
Self – Management - accept responsibility, flexibility, resilience, self-starting, appropriate
assertiveness, time management, readiness to improve own performance based on
feedback/reflective learning.
•
Team working - respecting others, co-operating, negotiating/persuading, contributing to
discussions, awareness of interdependence with others.
•
Business & customer awareness - basic understanding of key drivers for business success –
including importance of innovation, taking calculated risks and profit – and the need to provide
customer satisfaction and build customer loyalty.
•
Problem solving - analysing facts and situations and applying creative thinking to develop
appropriate solutions.
•
Communication & literacy - application of literacy, ability to produce clear, structured written
work and oral literacy, including listening and questioning.
•
Application of numeracy - manipulation of numbers, general mathematical awareness and its
application in practical contexts (e.g. measuring, weighing, estimating and applying formulae).
•
Application of IT - basic IT skills, including familiarity with word processing, spread-sheets, file
management and use of internet search engines.
It is important therefore to embed these competencies within work experience schemes through
appropriate structuring and management. Setting employability-based tasks and activities during the
placement is a layered activity and needs careful planning.
The Capital South Youth Guarantee Scheme t h e r e f o r e brings the focus firmly back to the
4
importance of work experience placements and employability skills being fundamental to the
continuous development of students – helping them to make the transition from education to
employment without significant periods of unemployment.
Croydon
The London Borough of Croydon was well placed to pilot the scheme at a time where NEET
unemployment was significant and town centre regeneration plans for a £3.4bn development would
attract commercial and retail development on a large scale. The scope and potential for employers to
locate and grow in Croydon presented a good case to identify real employment opportunity.
The borough is a credible testing ground for tangible inputs and tools for the Youth Guarantee Scheme
in the rest of the UK as well as other European Regions. The 12 month programme was due to finish
in September 2014, but due to delays in starting finished in December 2014. The project also
complemented and supported the activities of the London Borough of Croydon’s Skills and
Employment Strategy which underpinned the new Pathways to Employment Agenda.
Programme Overview
At a time when Education Business Partnerships are under increased scrutiny – with their relevance
questioned as a result of government legislation removing the need for schools to offer work experience
placements - the YGS programme sought to find an ongoing role for education partnerships and promote
the need for employability skills and work-related learning.
The aim of the YGS pilot project was to address rising levels of youth unemployment and the growing
number of young people at risk of becoming NEET. With the support of employers and other
stakeholders the pilot was designed to build capacity in 5 schools across Croydon to better prepare
young people leaving school for employment. The programme would achieve this objective through a
combination of activities that would:
•
•
•
Facilitate improved careers decision making
Ensure pupil learning and progression is linked to (and relevant to) the current and future
labour market
Improve teacher capabilities and awareness of the competencies required for work
The pilot YGS programme was designed, over a 12 month period, to work with teachers, students
and employers to deliver activities including practice (mock) interviews, mentoring, and work
placements for teachers and work experience placements for students.
The scheme would also allow participating students the opportunity to earn work credits when they
participate in any work related activity. The accumulation of a set number of credits would secure a
guaranteed job interview with a Croydon business. The outcomes of the project were forecast to
include:
• A student e-passport – an online application which contained a record of work based learning
and achieved employability skills verified by work or activity supervisors and mentors as a basis
for a work credit scheme.
• Business mentoring – younger-aged employees acting as peer mentors or role models,
providing mentoring support to young people to help them develop their employability skills
and raise their aspirations
• Enhanced employability skills for students
• Teacher CPD – providing ongoing training for teachers to help them improve how they teach
employability skills to their students
• Develop an online e-toolkit and best practice guide for project partners
5
The focus of the project would be on helping students learn and understand the concept and
significance of employability skills – i.e. what they and why they are significant. The seven key
employability skills identified for delivery during the project are those highlighted by the CBI:
•
•
•
•
•
•
•
Self-Management
Team Working
Business and customer awareness
Problem solving
Communication and literacy
Numeracy skills
IT skills
Employers would be encouraged to participate in the programme to:
•
•
•
•
•
Provide a team of ‘peer’ mentors to support students to develop their employability skills
and aspirations.
Develop a partner relationship with a school, including a hosting workplace visit / workshadowing opportunity for teachers.
Nominate one employee to develop a partner relationship role with a school.
Host 1, 5 or 10 day work placement visits for students.
Support the trialling of a student employability skills passport leading to the development
of a work credit scheme– setting credit thresholds for employment opportunities and workbased learning experiences.
Businesses participating in the programme were:
• Allianz
• BT
• Hammerson
• Mott MacDonald
• Westfield (Croydon Partnership)
Schools were actively encouraged to support the programme to:
• Support a mentoring scheme for participating students by business ‘peer’ mentors.
• Participate in employability skills training that would be delivered to both teachers and
students.
• Allow students to undertake employer workplace visits.
• Allow students to participate in 1, 5, or 10 day work experience placements.
• Participate in the trial of a student employability skills credit scheme and linked e-passport.
• Allow teachers the opportunity to participate in workplace visits / placements.
• Encourage the development of a partner relationship with a participating employer.
Schools participating in the programme were:
•
•
•
•
•
Addington High School
Edenham High School
Oasis Shirley Park Academy
St. Andrews Academy
The Quest Academy
6
Project Delivery
The original CapitalSouth tender included a timeframe for delivery over 12 months starting in April
2013 and completing by the end of March 2014. However, the European Commission did not agree to
fund the programme until September 2013 with an expectation that the project could be finished by
September 2014. CapitalSouth agreed to mobilise on the delivery of the contract on this basis but
continued to negotiate for a contract extension through to December 2014. This extension was
eventually agreed by the European Commission.
A number of contractors were engaged to support the CapitalSouth Team:
• Red Bullet – development and construction of the e-passport web portal
• HyggeUK – delivery of Train the Trainer and Mentor Training
• Make Business Happen – design and development of a programme website; and programme
best practice toolkit and an evaluation report
All contractors were retained following an open and public tender process.
Project Evaluation
The programme was designed to include both continual performance management and external
evaluation. Monitoring and evaluation were clearly linked but the CapitalSouth team were clear that
there was a distinction between performance management – which would be undertaken internally by
the CapitalSouth project team and the Senior Project Manager from Croydon Council’s Economic
Development Service – and the evaluation of the project – which would be undertaken by Make
Business Happen.
It was planned that CapitalSouth would provide real-time performance measurement of the project
throughout the twelve months utilising CapitalSouth’s existing Veryan Workplace database and Veryan
WebView web portals, and once it had been developed through the e-passport portal. Make
Business Happen consolidated and validated existing outputs, evaluated the effectiveness and impact
of the project; and developed a best practice guide and toolkit.
European Commission
The European Commission was actively engaged in the ongoing evaluation of the pilot programme.
Tony Slonecki (CapitalSouth) and Ross Feeney (Make Business Happen) attended a coordination
meeting in Brussels on 28 April 2014. The meeting, attended by representatives from all 18 projects
across 7 countries, provided an introduction to all the projects, sessions on overcoming the challenges
of the projects; monitoring and evaluation techniques and group discussions. The European evaluation
team stressed at the meeting that the pilot projects were by definition, challenging. The European
Commission acknowledged the projects had taken a long time to mobilise but were keen to understand
the challenges, solutions and recommendations for future best practice. Tony Slonecki and Topé Hunter
(CapitalSouth) also attended a further project co-ordination meeting in Brussels on 9 September 2014.
Make Business Happen
Make Business Happen was engaged early in the programme’s delivery to evaluate the programme. It
was immediately clear that the delay in securing a contract from the European Commission would have
a significant impact on the delivery of the programme. A number of challenges were identified in
delivering the project, and a number of pragmatic solutions have had to be actioned in order to
complete the project.
7
The most significant impact on the project was the uncertainty created by the contract delay itself.
The lead in time to securing the funding meant that project delivery would be out of synchronisation
with the school year timetable. Nearly all the students would undertake their work experience
placements before undertaking any of the employability skills training and teacher training would not
happen early enough in the programme for the results of the training to be seen in the classroom
environment – meaning the training would have no impact on the target beneficiaries identified to take
part in the programme. This issue was a constant concern throughout the delivery of the programme and
was a point highlighted by all delivery partners and stakeholders involved in the programme.
The Make Business Happen team, as part of their delivery, undertook an evaluation, including
communication with stakeholders and project beneficiaries, to assess their feedback and views on the
effectiveness of the programme:
Feedback from Business Mentors
The scheme was positively perceived by the business mentors and generally they thought that the
project should be commended for giving young people the opportunity to get a better understanding
of the working environment and what career opportunities there are for young people when they leave
school. One mentor stated that “The opportunity to assist young people in their journey post-secondary
education was amazing”.
100% of the business mentors felt that they now had improved links with schools in the Croydon
area, helping them feel that they are more embedded within the local community. Mentors
commented that one of the benefits of the scheme was the development of the relationships with
schools which they hoped to continue long term.
The business mentors recognised the benefits that young people took away from the scheme with
100% agreeing that the young people had a better understanding of skills needed to become employed
and that the young people who participated left with a greater understanding and focus on their
future career aspirations.
One of the mentors commented: “…..By the end they both had plans following the end of their school
education. It was very fulfilling to see them develop from not having any interest in their futures to
thinking positively about it and making plans”.
In addition to the benefits for the students all of the businesses felt that the scheme bought
opportunities for staff development and the chance to improve knowledge. It is also positive to note
that 80% of the business mentors thought that they had increased their potential pool of candidates
for future recruitment by providing young people with the necessary skills required for employment. The
mentors felt that improvements could be made to the scheme in terms of identifying those students
who are in the greatest need of assistance to make the scheme better targeted. Further clarity could
also be given to the role of the careers advisors o p e r a t i n g i n t h e s c h o o l s o u t s i d e o f t h e
p r o j e c t as one of the mentors felt that there was some overlap with their roles. Communication
between CapitalSouth and these careers advisors would have been beneficial but this would have been
dependent on school facilitation.
Feedback from Schools
The schools provided positive feedback on the scheme, in particular the event hosted at Hammerson
which enabled students to develop and demonstrate multiple skills. One school thought that more
opportunities such as this would be really beneficial to future schemes.
The mentoring scheme was commended by the schools. This was felt to be a crucial part to the
success of the scheme and schools were appreciative of the matching of mentors to students and the
8
amount of time the mentors gave to work with the young people; one noted that this had taken place
during a mentor’s holiday time.
Schools that already offered students similar opportunities p r o v i d e d b y the scheme still found
benefits in participating and stated that they had further developed their network of employers to
engage with. Whilst the schools recognised the positive outcomes that students achieved through
the scheme generally, it was felt that much could be done to improve the organisation of
some events, particularly in terms of scheduling during the academic year and the notice period
some schools require in order to plan trips and outings. However, CapitalSouth was praised for the
way that the coordination of the scheme was organised and the schools enjoyed this close working
relationship.
Improvements to the events could be made to ensure that young people were fully engaged at all
times and that the right environment is created for students to participate in the activities. One
school commented that due to poor behaviour from another school in attendance their students did
not feel comfortable in delivering their presentations. Consideration also needed to be given to the
venues and their suitability for accommodating young people of this age group.
Feedback from Students
The scheme was positively perceived by the participating students and they were thankful to all
participating mentors. Many thought that the sessions should be extended and explained learning
outcomes and personal gains. 12 student sample surveys from St Andrews and Oasis Academy schools
reviewed 9 mentors. St Andrews had 24 meetings and typically each student participated in 3 or 4
mentoring sessions facilitated by 6 business mentors. Oasis A c a d e m y students took 18 or 19
meetings with around 6 sessions per student and 3 business mentors.
Some students commented that more sessions would help. Interestingly some of the students who
had 6 sessions also commented that they would like more sessions; one commented that they
“would like more but this means coming out of more lessons” indicating that the activity would be
useful if it was part of the mainstream timetable, or that extra time is given to the activity. Students
on the whole found mentors to e s t a b l i s h r e l a t i o n s h i p s w i t h and commended mentors with
comments clearly supporting t h e i r increased understanding of employability skills. This was evident
in all student comments which show positive and improved attitudes towards learning and selfdevelopment.
Around 35% of students found it ‘fairly easy’ to get to know their mentor and the survey has
supported the view that business mentor sessions were helpful in a variety of ways. All students
demonstrated their capacity building potential had led to some increased self-esteem as well as general
knowledge. This was demonstrated by an increased confidence to meet new people and evidenced by
students thanking mentors for their support.
A greater focus on future career direction and occupation options is supported with 100% of students
considering future goals and career plans, with a wide diversity of student comments in terms of
career pathways and long term goals. Various comments from students demonstrate knowledge of
specific careers and job roles and career choices and a greater understanding of the requirements of
'working life'.
Students commented that the opportunity to get tips on presentation and clarity about career pathway
was a significant element of the feedback and showed that students have an improved attitude
towards learning and self-development. Other specific information cited eg. the l o c a l BRIT School
for the Performing Arts and Technology (sponsored by the British Record Industry Trust) gave a clearer
understanding of the working environment and what career opportunities there are for young people
when they leave school.
9
Pupils were also appreciative of the matching of mentors who were varied in background and found
benefits in participating, stating that m e n t o r s g o t t o k n o w their strengths and weaknesses,
giving an opportunity to focus on education and career choice.
The students recognised the benefits that mentors brought to the scheme with additional reference
that further communication would be beneficial. Whilst students recognised positive outcomes that
they had achieved, it was generally felt much of this activity could improve if there was more time
and activities.
Feedback from CapitalSouth Project Team
Face to face feedback interviews took place with all key members of the CapitalSouth project team –
Tony Slonecki, Topé Hunter and Claudia Granato. A telephone interview was also undertaken with
Gillian Morris from HyggeUK who delivered the Train the Trainer programme for teachers and
developed the student mentor packs.
The feedback provided by all respondents focused on the contract delay being the mitigating factor
for delays in mobilisation and delivery. The feedback from the delivery team was both frank and
realistic about the achievements of the project over 12 months. The feedback has been aggregated
to highlight both the positives of the programme and the challenges the team faced.
Positives from the programme:
• All project team members felt it was enjoyable working with the students who showed an
interest in participating in the programme. A particular reward was achieved from helping
students ‘open their eyes’ to new career possibilities and opportunities they had previously
dismissed as unachievable or uninteresting.
• All the project team members felt it was refreshing to work with the business mentors – who
brought both a private sector work ethic and commitment to the programme. All the mentors
were noted to be engaged and interested in the programme and keen to support the
students. Their engagement was regarded as hugely beneficial and genuine.
• Those team members who had worked directly with schools, employers and teachers felt
that the relationships they had built would continue beyond the lifetime of the programme.
• The project team enjoyed working with teachers that were engaged with the programme.
There was a clear understanding that t h e s e teachers would benefit from the training
undertaken, enabling them to teach employability skills to young people and to continue
beyond the lifetime of the project. The teachers understood the importance of students
developing and using their employability skills for their future prospects.
Challenges in delivering the programme:
• The project team felt they had collectively under estimated the amount of administrative
time that would be taken to deliver the project. Initially insufficient time was allocated to
project management and administration which meant the team struggled in the early months
of the programme to keep on top of the tasks that needed undertaking.
• Securing feedback from students took a long time to achieve; many students did not want to
give feedback because they h a d left school, and those that did provide feedback provided
it through their teachers – who took some time to pass the feedback on.
• Securing criminal record checks through the national Disclosure and Barring Service (DBS) for
mentors took longer to complete than expected – which meant that by the time mentors had
completed their training and waited for the DBS checks some had lost their enthusiasm for
continuing.
• Working with schools’ senior management teams proved more challenging than had been
expected. In some schools where there had been a change in management personnel during
the lengthy period from initial formal commitment to the project and actual mobilisation there
was a reluctance to get involved, making staff and class time available to actively participate
10
in the programme and take up the opportunities made available to them. For example
o
o
o
•
•
It was noted that one school sent teachers for training who were unsure of why
they were participating.
Some teachers took part in training after their teaching day had been completed and
they were tired and disengaged.
By the nature of their role (i.e. not desk-based) teachers were difficult to contact –
not always available by phone or email.
In some cases, lack of importance or indifference towards the programme from teachers
meant that co-ordination of mentoring visits / meetings between students and mentors took
much longer than expected. The combination of mentors having busy work schedules,
schools with inflexible timetables, and teachers taking a long time to respond to the
CapitalSouth project team meant the mentoring took much longer, administratively, than had
been forecast during the project development.
A number of project team members also highlighted the variations amongst schools in the
selection of students to participate in the programme. The variation in selection criteria was
highlighted by the contrast between two schools in particular. One school encouraged its
students to compete for a place on the programme and consequently its students were
engaged and receptive to the training being offered. By comparison another school selected
the students it wanted to participate on the basis that they might become NEET. However
the students were not aware of why they had been selected and consequently did not view
their own participation as positive and were not as engaged in the whole process.
Delivery of an e-Passport.
Failure to implement the e-passport was seen by the project team as a significant frustration during the
delivery phase of the project and impacted on their overall achievements – with a ‘partial completion’
noted in the following table of outputs.
The e-passport was designed to record the work credits achieved by students taking part in the
programme – recording and acknowledging the employability skills they had demonstrated during their
journey through the YGS programme. The e-passport was designed to allow students to demonstrate
their skills through a CV builder software package which would help them to create a CV they could print
off or email to potential employers. Whilst the CV-builder element of the project was completed, the epassport and associated work credits was not ready in time to be operational whilst the project was live.
Factors that impacted the delivery of the e-passport include:
•
Technical Specification – the project team noted that it quickly became apparent that there was
communication issues between themselves and Red Bullet – the IT company contracted to
deliver the e-passport. The breadth, complexity and volume of development options available to
the project team was not fully explained by Red Bullet and it is clear, with the benefit of hindsight,
that the specification in the original tender published by the project team did not adequately
cover the detail and scope of the project. It must be emphasised that errors with the tender
specification cannot be attributed to negligence by the CapitalSouth team nor Red Bullet.
However, the IT contractor was used to the language and jargon of IT development and the
project team were used to the language and jargon of the public sector and education. In some
instances these two different uses of language and terminology caused confusion and it took
some time for both parties to establish a foundation for their working relationship and a clear
way forward for the delivery of the project.
•
Data Protection, Safeguarding & Protocols – At the start of the project the project team had been
informed by Council officers that the e-passport could be hosted as an additional portal on an
11
existing website under the ownership of Croydon Council and its Young People’s Services.
However following meetings between Red Bullet, the Young People’s Service and the project
team to discuss practicalities it was resolved not to pursue the original specification proposals
due to following concerns:
o
o
Managing and monitoring the proposed ‘Ask an Expert’ section, developing FAQs on
careers, and providing advice and guidance would be problematic and possibly open to
abuse in an open web environment. There was no guarantee the correct protocols for use
and access to the data could be monitored or secured.
Staffing and maintenance. – it was not clear who would be responsible post-project for
the maintenance of the site and its constituent data.
A new methodology for ensuring the e-passport was viable had to be sought and the IT contractor
recommended a stand-alone site be developed that could be managed by CapitalSouth and not
a third party. This solution was both pragmatic and well thought through and provided
CapitalSouth with additional post-project legacy. However, the timescale involved in this
discussion and subsequent change of specification contributed to the delay in development. The
e-passport was ready for use as this report was being drafted and active trialling by more schools
will take place later this year using student’s existing pin numbers.
•
Staff resources – it is quite clear from talking to the project team that this delay in the delivery of
the website and the time taken to establish the causes of the delays highlighted above had an
impact on the project team’s available resources. A considerable amount of time was invested in
ensuring all the obstacles were worked through and overcome. However, with such a small team
working on the project it is clear that this time investment was to the detriment of the
programme and its delivery.
Phase one of the e-passport delivery had been achieved by the end of the project timescale, but the
concept and application of the software behind the e-passport had not been completed satisfactorily and
a phase 2 of development was continuing. This second phase of development, continuing after the
project-end, was designed to develop an effective interface between the data held on students and the
data held by work placement coordinators through software that did not require considerable manual
intervention. The CapitalSouth team should be commended for working outside the scope and funding
of the project to ensure it is completed as part of the project’s legacy. It must be noted that the failure
to implement the e-passport before the end of the project also had an impact on the completion of the
business accord. The accord was drafted but referred businesses to the e-passport – and support and
recognition for the hard work that it represented. As the e-passport was not ready it was not possible to
record student’s work credits - as one was designed to feed into the other. Consequently the project
team took the decision not to pursue signatures from businesses on the accord.
The delivery team were also candid in their pragmatic rationale for not achieving two other outputs:
• the required number of business internships - the project team felt that as the time taken to
secure, train and DBS-check business mentors was considerably longer than planned it was a
better use of staff resources to focus on achieving that output and therefore it took priority over
the internships; or
• establishing an advisory board - the CapitalSouth team were already scrutinised by their own
board of trustees during the development stage of the project it was determined that there was
no need for additional scrutiny.
Outputs & Outcomes
The CapitalSouth programme was designed to include both continual performance management and
external evaluation. In evaluating the success of the programme it was necessary to draw a distinction
between hard/tangible outputs and less tangible outcomes.
12
Outputs
An overview of the project outputs is highlighted below. Whilst there are a number of areas where activity has not met the agreed target, areas where there has
been significant overachievement has ensured the project team has successfully achieved the outputs of the project – with a 2% positive variance on overall
deliverability.
Outputs
Target
Actual
Achieved Y/N
% Variance
Business Internships (No. of Teachers)
20
10
Partial
-50
Education Internships (No. of Business volunteers)
5
0
N
-100
Young Intern Prog. - No. of Students
50
75
Y
50
No. of participating schools
3
5
Y
67
No. of participating employers
2
6
Y
200
Mentor training Programme (Business volunteers)
1
1
Y
0
Recruitment of Mentors (no. of Business volunteers)
25
40
Y
60
Mentor training toolkit
1
1
Y
0
Employability Skills Train the Trainer Programme (teachers)
1
1
Y
0
Employability Skills Training Programme (students)
1
1
Y
0
Employability Skills Training toolkit
1
1
Y
0
Employer Accord Signed
1
0
Partial
-100
E-Passport Developed
1
1
Partial
0
Best Practice Guide & Toolkit
1
1
Y
0
YGS Co-ordination Meetings- Brussels
2
2
Y
0
Young Intern Award Ceremony and Masterclass
1
1
Y
0
Advisory Board meetings
3
0
N
Total Project Variance
%
Project Achievement
%
-100
13
2
102
RAG
Status
Outcomes
The outcomes from the programme are less tangible and can only be determined as being achieved or not achieved based on the responses from individual
teachers, students and business mentors.
The responses from business mentors is clear-cut and broadly very positive. Mentors understood the project before they signed up to take part, and therefore
their comments and feedback were largely impartial. Feedback from teachers and students varied and it is apparent that those participants fully engaged in the
programme and understood the aims were largely more positive about the programme than those that weren’t.
Whilst there was both extremely positive and negative feedback from respondents, in all cases an aggregate assessment of the responses was used to make
determine the following outcomes.
Outcomes for individual Students
Greater understanding of the requirements of 'working life'
Increased self-esteem
Increased confidence to meet new people
Increased confidence to approach new challenged / activities
Defined attitude to education and personal achievement
Defined confidence and attitude to future employment
Developed ability to work autonomously
Understanding of personal time-management / appearance
14
Achieved
Y/N
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Outcomes for Students
Increased understanding of employability skills
Improved attitude to learning and self-development
Increased employment aspiration amongst beneficiaries
Greater focus on future career direction and occupation options
Completed e-passport
Positive perception of their work experience journey
Outcomes for Schools
Improved links with Croydon-based employers
Improved work experience programme planning and delivery
Staff development - understanding the workplace environment
Increased perception of the value of work experience
Outcomes for Businesses
Improved links with Croydon-based schools
Revised approach to offering work experience - with defined opportunities/roles
Employee development (skills and knowledge)
Increased recruitment 'pool' of future employees
Increased community engagement
Positive perception of their work experience journey
Capital South Outcomes
Strengthened links with local employers
Increased perception of the value of work experience in Croydon
Increased perception of the added value provided by Capital South
Increase in host employers for work experience
Additional case studies of successful work experience participants
15
Achieved
Y/N
Yes
Partially
Partially
Yes
Yes
Yes
Achieved
Y/N
Yes
Partial
Partial
Yes
Achieved
Y/N
Yes
Yes
Yes
Partial
Yes
Yes
Achieved
Y/N
Yes
Yes
Yes
Yes
Yes
Best Practice Recommendations
What is clear from our evaluation is the tremendous workload and dedication of the CapitalSouth
team. It is equally clear from our evaluation that the length of time from submission of the project bid
(October 2012) to the European Commission’s confirmation of the bid being successful (July 2013)
followed by agreement verification and signing (December 2013), led to significant delay in the
deliverability of the programme, which impacted on the support that could be secured from original
bid partners.
The programme has highlighted the differences between employability skills, as defined by the
private sector and articulated by the Confederation for British Industry (CBI), and employability as
defined and understood by those in education. The programme has also defined the challenges faced
in improving school leaver’s employability skills. Importantly the programme has helped student
beneficiaries to think about their future careers and the pathway / opportunities open to them.
However, pilot programmes by their very nature identify problems and challenges not initially foreseen.
Our evaluation of the pilot identifies recommendations that should be considered for the development
a more significant regional or national programme:
Recommendations:
• The programme would have been more easily delivered if it had run over 18 months instead
of 12 months. The first 6 months of the programme should be allocated to preparation and
project mobilisation – allowing time for mentor training, DBS checks and school participation
to be established.
• Working with year 10 students may be more appropriate than year 11 students. The project
team noted that in some instances students were being taken out of classes during their
final year but not being supported to catch up with missed lesson material. Year 10 students
may also have more time to think about their career choices prior to their final year in
education.
• The programme needs to be synchronised more effectively with the academic calendar to
ensure that students and teachers do not take part in training or employability training
immediately prior to a holiday period. Programmes should not include sessions that take
students away from the classroom in the run-up to important year 11 exams.
• Delivery of activities needs to be carefully sequenced to ensure teachers have undergone
training and students have learnt about (and developed an appreciation of) employability
skills before work experience and work-shadowing placements take place.
• Future programmes need to allow for a significant administrative resource to be allocated
to the programme throughout its duration to mitigate the issues highlighted in the
evaluation feedback above. Two full-time administrative resources should be a minimum
requirement for a future project to support the role of the project manager / director.
• The programme should not be mobilised until the project team have a sound and solid
understanding of the IT and software infrastructure already in place within their existing
organisation and how this may link (or not) to any proposed new software or cloud-based
solutions. Project teams should also investigate all data-protection and individual background /
security checks that might be required on individuals involved in the delivery of the frame work
(volunteers and project staff). The project team should also have a solid understanding of the
timeframe involved in securing the relevant data and personnel clearances.
• The project team should assess their own knowledge of IT and software before commissioning
any new soft or hard infrastructure and should seek the advice of independent consultants of
they are unsure of whether they ‘don’t know what they don’t know’.
As part of the project an online best practice toolkit and document library was commissioned by the
project team and created on a bespoke project website. The legacy of the project has been established
through the website and provides an opportunity for other education business partnerships and support
16
organisations in the UK to learn from the YGS project. The website – www.capitalsouth.eu – also provides
the opportunity for ongoing best practice learning by the other 17 pilot projects particularly as the best
practice toolkit has been translated into Spanish and Italian (the languages of the two countries with the
most pilot projects).
Conclusions
In reaching our conclusions on the success of the YGS pilot we have considered the feedback from
students, schools, businesses and the project team, and analysed the statistical output data from
the project. Having reviewed the data the evaluation team have concluded that, on balance:
1. Schools and their participating teachers did feel that as a result of the programme they had
a better understanding of what skills employers are looking for in new recruits.
2. Teachers did feel the programme had bridged the gap between the supply and demand of
skills provision – which over time would ultimately lead to more work-ready school leavers.
3. Participating employers did feel the project had helped them understand how to engage
with school-leaver recruits.
4. Participating employers did have a better understanding of how to engage with schools
and colleges to help their recruiting programmes.
5. Participating employers felt school-leavers who had participated in the programme were
better prepared for work.
6. Participating school-leavers did feel they had the right skills necessary for the workplace and
had a better understanding of what would be expected of them in the workplace.
Both the quantitative and qualitative results s u p p o r t o u r a s s e r t i o n s a b o v e a n d that the
evaluation concludes that the programme has been successfully delivered. The outcomes and outputs
highlighted in the original tender submission have been achieved or exceeded, and the European
Commission can take comfort from the work undertaken and delivered by the CapitalSouth team.
The conclusion of the evaluation of the programme is however mixed, with the success of the
programme overwhelmingly impacted, negatively, by the delayed contract negotiations with the
European Commission. Yes, the project did achieve the outputs specified in the tender - thanks to
the near-herculean efforts of the CapitalSouth team. Yes, the outcomes for beneficiaries were
achieved, and yes, the pilot has identified that there is a clear need to improve employability skills,
the teaching of employability skills and a greater level of understanding between schools and the
private sector. However, buy-in from all three key stakeholder groups (schools, students and
employers) is fundamental to securing the success of such programmes. When stakeholders understood
the aims of the project (and the benefits to them) they responded positively and the right outcomes
were achieved.
For CapitalSouth, the legacy of the programme can be evidenced through the best practice toolkit
that has been developed and is available for education business partnerships across the UK – ensuring
a legacy of through-leadership and creative delivery. CapitalSouth can also use the experience of
delivering the Youth Guarantee pilot to consolidate its links with local schools and businesses, and
use the documents and mentoring collateral to support future work experience / employability
initiatives as the documents and training guides prepared for the programme will remain relevant
(student activity e-pack, training of schools staff, cascade training e-pack, Mentor training pack).
17
Appendices
Evaluation Team
The evaluation was undertaken by Make Business Happen CIC (MBH). MBH is a not for profit social
enterprise, established in 2012, to support business growth and enterprise in London and the South
East. MBH was awarded the contract to evaluate the pilot programme, and create a best practice
toolkit, following a competitive tender process in early 2013.
The evaluation was undertaken by:
• Ross Feeney – Ross is an accomplished project director with over 12 years commercial
experience. For 10 years Ross worked for South London Business, a sub-regional business
support organisation funded by the LDA, Business Link and the LSC/SFA. For seven of those
years Ross was part of the senior management as director of business support. Ross has
delivered contracts for most London boroughs south of the Thames, as well as ESF, ERDF
and Leonardo Fund European programmes. In addition to his role, as managing director of
Make Business Happen, Ross is also part of the White Label senior management team and part
time chief executive of Successful Sutton Business Improvement District.
•
Andrew Dennis – Andrew began his business advice career in recruitment and increasingly
supports a diverse and influential portfolio of London based businesses with various
disability, skills brokerage and business support contracts of which employability is a
recurring theme.
Methodology
The Make Business Happen (MBH) evaluation team applied a mixture of desk and field-based research
techniques to assess the success of the project – to determine the overall effectiveness of the project,
the economic and social impact of the programme, and an assessment of the value for money impact
on beneficiaries.
Working with the CapitalSouth team MBH agreed an indicator framework for assessment based on
the government’s FABRIC performance management tool (Focused, Appropriate, Balanced, Robust,
Integrated, & Cost effective). The assessment used the internationally accepted Red, Amber, Green
approach to evaluating the success of the programme – based on the original tender specification.
The MBH team delivered both the project evaluation and toolkit preparation through a staged
delivery process:
1. Met with the project team to agree delivery timeframe, key personnel, reporting structure
and delivery contract
2. Made contact with all project partners, introducing the core team and outlining our
proposed delivery timetable
3. Desk research review of all the project materials - including original tender specification and
final European Youth Guarantee Scheme contract, activities undertaken to date and the
overall project plan and delivery timetable
4. Reviewed all communications and agreements with project partners
5. Prepared and agree an assessment framework
6. Engaged with (education) stakeholders to obtain their feedback on the programme and its
methodology – focusing on their understanding of the employment market and what they
learnt, or need further assistance with, to help them prepare students for employment at the
end of their education
7. Engaged with (employer) stakeholders to obtain their insight to the effectiveness of the
programme and the suitability of students and their ‘ready for employment’ status
8. Engaged with (student) stakeholders – i.e. those who have participated in the programme –
to ascertain their thoughts on the programme, its delivery and the medium to long-term
outcomes for their career
18
9. Collated and prepared the evidence and data from stakeholders and cross reference against
aims and objectives of the project
10. Outputs and data collected through the evaluation activities were collated and prepared into
a comprehensive project best practice guide and delivery toolkit
11. Draft documents will be circulated to partners for approval
12. Draft documents will be prepared into web-ready PDFs. The documents will also be
translated into 2 European languages
13. The final project evaluation report will be completed and all economic assessments will be
prepared and printed
14. The project website will be developed and published with information about the project and
a password protected ‘partner’s area’ within which the best practice guide and toolkit will be
available to download
Evaluation tools:
1. SWOT (Strengths, Weaknesses, Opportunities and Threats) analysis of the project and its
delivery and achieved outcomes.
2. We also applied the principles of the Social Return on Investment (SROI) evaluation tool.
SROI is an outcomes-based measurement tool that helps organisations to understand and
quantify the social, environmental and economic value they are creating. Developed from
traditional cost-benefit analysis and social accounting, SROI is a participative approach that is
able to capture in monetised form the value of a wide range of outcomes, whether these
already have a financial value or not.
SROI is informed by a set of principles that are designed to ensure that the process is
robust, transparent, and informed by stakeholders. The principles inform a six-step
methodology:
• Establishing scope and identifying key stakeholders. Clear boundaries about what the
SROI will cover, and who will be involved are determined in this first step.
• Mapping outcomes. Through engaging with stakeholders, an impact map, or theory
of change, which shows the relationship between inputs, outputs and outcomes is
developed.
• Evidencing outcomes and giving them a value. This step first involves finding data to
show whether outcomes have happened. Then outcomes are monetised.
• Establishing impact. Having collected evidence on outcomes and monetised them,
those aspects of change that would not have happened anyway (deadweight) or are
not as a result of other factors (attribution) are isolated.
• Calculating the SROI. This step involves adding up all the benefits, subtracting any
negatives and comparing them to the investment.
• Reporting, using and embedding. Easily forgotten, this vital last step involves sharing
findings and recommendations with stakeholders, and embedding good outcomes
processes within your organisation.
19