Musical transitions from informal to formal: effective

Transcription

Musical transitions from informal to formal: effective
Musical transitions from informal to
formal: effective teaching resources
Kevin Grist (May 2012)
Introduction
In May 2012, the Arts Council announced funding for 122 music education hubs
across England, the result of a vision laid out in A National Plan for Music
Education (Department for Education, 2012). The hubs are intended to
fundamentally change the way music education in England is delivered to 5-18
year-olds both in and out of school, by creating a more joined up approach on a
local level. This means schools, music organisations, the voluntary sector and
higher and further education providers will come together to deliver shared aims
for young people; an opportunity to sing and learn a musical instrument, to
progress to the next level, and play with other children as part of an ensemble or
choir (Arts Council England, 2012).
I write this study as a music teacher, with eight years of experience working with
young people in and out of schools in Brighton & Hove. This has ranged from
delivering music education inside primary and secondary schools, with FE
colleges and in youth clubs, at performance venues and even in the street
(outreach projects). Drawing from my experiences, the work that music education
hubs are attempting to deliver will cover a diverse and often complex range of
young people’s needs – those who are gifted and talented, in challenging
circumstances, approaching school transitions and identifying progression
pathways to name a few. A challenge for hubs will be how formal and informal
teachers come together from differing ideologies and approaches to music
education. The notes from a networking event held prior to the formation of the
Berkshire Music Education Hub, for example, cited informal versus formal and
accredited versus non-accredited as key future discussion points (MusicLeader,
2012). This study of my own practice, therefore, intends to explore the very
notion of what informal and formal music education is and what effective teaching
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
1
resources could be deployed to help young people’s transition between the two1.
In light of A National Plan for Music Education (Department for Education, 2012),
it is likely that a learners’ journey between informal and formal music education
will occur more fluidly and instantaneously than ever before.
1 – 35 music practitioners surveyed in April 2012 were asked to rank the importance of the following question on a scale
of 1-5 (5 being most important): “How important is it to have teaching resources that help the transition from informal to
formal music education?” 65.7% ranked the importance at 4 or above.
Formal, non-formal and informal?
The definition of formal and informal learning varies in parts of the world but there
have been attempts to clarify the terms at an international level so that
academics can at least speak the same language (Werquin, 2010). For the
purpose of this study, I will use Patrick Werquin’s definitions as laid out in his
recent work for the Organisation for Economic and Co-operation Development
titled Recognising non-formal and informal learning: Outcomes, policies and
practices. Here, three domains of learning have been identified; formal, nonformal and informal:
•
Formal learning is identified as intentional and occurs in an organised and
structured environment with clear learning objectives, typically leading to
accreditation. In music education, this would typically be the state-run system
including schools and universities working towards theory-based GCSE/ALevel/BA in Music, ABRSM Grades and some practical-based qualifications
such as the Rockschool Award.
•
Non-formal learning is also identified as being intentional, but is embedded in
activities not necessarily designated as learning or accreditation. In music
education, this could be learning something about you such as teamwork or
management skills in a recording studio or music ensemble. This is typically
seen in vocational and adult learning working towards continued professional
development, apprenticeships and engagement programmes.
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
2
•
Informal learning is identified as unintentional learning taken from daily
experiences in work, home or leisure, with no structured objectives. In music
education, this would likely be seen in home-based ‘bedroom producers,’
community choirs, youth centres, recording studios and music venues.
In April 2012, I conducted a survey amongst music practitioners. One set of
questions asked them to submit words and phrases they would associate with
the terms ‘formal music education’ and ‘informal music education.’ Non-formal
learning was left out of the survey, as I wanted maximum clarity on the two
opposing concepts. The results, (shown in figures 1 and 2) provide consistency
in the way each concept is perceived by music practitioners, and the effects it
has on their learners.
Figure 1: Most commonly used word and phrases used to describe "Formal Music Education" taken
from a poll of 35 music practitioners in April-May 2012 (SurveyMonkey/Wordle)
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
3
Figure 2: Most commonly used word and phrases used to describe "Informal Music Education"
taken from a poll of 35 music practitioners in April-May 2012 (SurveyMonkey/Wordle)
For formal music education, the survey provided words such as “structured,
teacher-led, qualifications, curriculum, school and exam” as common ways to
describe the learning experience. There were some negative perceptions of
formal music education provided – “boring, discriminatory, exclusive, closedaccess and less creativity.” Taking into account that 54% of those surveyed
considered themselves as working in both formal and informal education, it would
suggest that there are some strong ideologies against elements of formal music
education (particularly around issues of access and equality).
For informal music education, the survey provided words such as “fun, studentled, out of school, experimental, self-learning and engaging” as common ways to
describe the learning experience. Likewise, there were some negative
perceptions of informal music education provided - “anarchic group, lessstructure, frustration and no accreditation” Again, taking into account that around
half of those surveyed considered themselves as working in both formal and
informal education, it would suggest that there are some strong ideologies
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
4
against elements of informal music education (particularly around issues of
professionalism and validation).
Learning continuum
The difficulty with the model of domains is that many organisations offer music
education that cuts across all of them. For example, a university that offers a
formal BA in Music may also deliver non-formal day programmes in Music History
or provide informal Music Technology e-learning resources online. Because of
this, there are a number of models that position formal and informal learning on a
learning continuum, with the suggestion that people move between their polar
extremes (Reinhard, 2010). The most basic is the simple line:
Formal learning
Non-formal learning
Informal learning
Figure 3: Continuum model – (Reinhard, 2010)
Using this simple linear continuum model, I have mapped to it some common
ways in which young people learn music and cited properties of formal and
informal learning as set out by Werquin, 2010 (figure 4):
Formal learning
Non-formal learning
Informal learning
Intentional
Structured
Controlled
Intentional
Structured
Non-intentional
Learning is
deliberate
Learning is
strongly
structured
by discipline
or field
Learning is
non-intentional
Awarding Body
Learning is not
structured by
discipline of
field
University
College
Secondary School
Learning is
strongly
structured in
curricular
terms
Primary School
Music Service/Strategic Organisation
Learning as
such is not
planned and
has no formal
designation
Private Training Provider
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
5
Open market
State
regulation
Vocational / CPD
Music Venue
Providers
are public or
stateaccredited
Youth Service/YOS
Community Arts Project
Band/Ensemble
Quality
assurance
mechanisms
At Home/Studio
Audience
Private
providers &
volunteer
organisations
with no
restricted
market entry
No quality
assurance
mechanisms
Figure 4: Common ways in which young people learn music (mapped to continuum model)
The diagram, for example demonstrates how music (in the informal domain) is
learnt through listening (audience), making music in the home and developing in
groups. The transition into non-formal music making via a community project or
youth centre is shown as the next transition before more formalised school and
college. But like the three domains themselves, this diagram is limited and does
not show the full breadth of how each may span wider across the continuum
(Reinhard, 2010). Rather, it implies what domain/s most likely cover each of the
ways a young person can learn music. The width of each on the continuum is an
estimate based on my own understanding of music education, and there is room
here to develop a more reliable way of measuring it. However, this diagram is
helpful in identifying common transition points across music education and to
scope areas of similarity when developing teaching resources. Transition points
might include primary to secondary school, year 9 choosing formal GCSE options
and post-16 accessing formal music education for the first time. Also, there are a
number of other models including the Learning Mixer (Cross, 2007), which could
be useful alternatives to this standard continuum, as discussed by Reinhard,
2010 at www.infed.org.
Using the same model as figure 4, I created a second diagram (figure 5) mapping
common ways in which music learning is assessed and added an assessment
continuum based on Hills, 1992:
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
6
Formal learning
Learning is
deliberate
Learning is
strongly
structured
by discipline
or field
Summative
assessment
Instrument or Exam
tests with a grade. Pass
or fail, little or no
feedback.
Non-formal learning
Informal learning
Formative assessment
Self
assessment
Planned observations
with set criteria from
self and teacher
Observations of other
music as an audience
member/consumer –
listening, watching
Not normally written
Learning is
strongly
structured in
curricular
terms
State
regulation
Academic style writing
for essays and
rationale.
Some contextual and
theory research
Providers
are public or
stateaccredited
Quality
assurance
mechanisms
Modular work samples
of music in response to
set projects/goals
Internet searches on
inspiring artists and
techniques
Self-evaluation of own
music
Open market
Discussion, audiovisual
record, reviewing/diary,
record or tickets sales
Sense of learning
outcomes
Written feedback given
against set learning
criteria
Learning is not
structured by
discipline of
field
Learning as
such is not
planned and
has no formal
designation
Clear presentation of
information
Deeper analysis
drawing on existing
body of knowledge
Learning is
non-intentional
Structured feedback
from peers and tutor.
Feedback of own music
from others
Opportunity to refine
and resubmit work
Discussion, peer
mentoring
Formal focus on
developing language,
literacy and numeracy
Setting personal
projects/goals that are
not necessarily timely
Private
providers &
volunteer
organisations
with no
restricted
market entry
No quality
assurance
mechanisms
Public comments via
social media
Figure 5: Common ways in which music learning is assessed (mapped to continuum model)
The idea of this diagram is to understand ways in which music learners gain a
sense of achievement through various types of assessment. It is useful for this
study as it provides a basis for creating teaching resources that not only address
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
7
transition but also recognise achievement.
Rogers, 2003 believed that learners were more conscious of learning in a formal
academic environment, and less so in an informal ‘authentic’ environment such
as the workplace (situational learning). He argued that learning lost its context
the more formal it was (Rogers 2003 and Smith 2009). As figure 5 shows, music
achievement is completely ‘real-world’ in the informal domain with assessment in
the form of, for example, attending gigs, listening to music, getting feedback from
band mates and social media. As you move to the formal domain, the sense of
achievement in music is largely related to the academic process itself (not realworld). Wenger, 2007 describes situational learning within a social situation as
“communities of practice” and is linked with the ideas of vocational learning;
apprenticeships, internships, professional networking and on-the-job training. It is
perhaps surprising, then, that mainstream education in the UK still relies heavily
on formal learning (out of context to the real-world), although there is an
increasing value and importance being put on vocational learning (Wolf 2011).
For future study, what would the continuum show us if we mapped other areas of
music education such as musical genres, socio-economic groups and gender?
Existing music education models that help transition
One of a music education hub’s key aims – to progress to the next level – implies
that the music education sector sets out clear progression pathways for young
people. There are some existing music education models that help such
transition. In secondary schools, some institutions have adopted Musical Futures,
a Paul Hamlyn funded initiative that brings “non-formal teaching and informal
learning approaches in the more formal context of the school (Price, 2006).” It
draws on the idea that young musicians learn initially from aural skills (listening)
and use these to learn develop and copy instrumental and vocal parts of a song.
It doesn’t discourage using notation, but isn’t explicit in its use either. Musical
Futures’ success in schools appears to be because of its simple learner-centred
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
8
model, that encourages fun, engagement and acknowledgement of informal
learning. Price, 2006 highlights the different modes of learning in music (and
different use of language):
Formal
Non-formal & Informal
Lesson
Projects
Music teacher/conductor
Music leader/coach
Learning through notation/instruction
Learning by aural means (playing by
ear, copying and trial-and-error)
Learning is sequential
Learning is often haphazard
Donor-recipient model of teaching
Teacher is co-learner
Learning activities are separately
Teaching is distributed
specialised
Activities are integrated
Technique is pre-requisite to repertoire
Technique s acquired through
repertoire
There are a number of other music education models and projects, which aim to
help transition into formal education, including In Harmony Sistema, Sing Up,
Music and Dance Scheme and qualifications Rock & Pop, Rockschool and Arts
Award.
There is no reason to assume that transition cannot be also from formal to
informal education, and this is another area for future study.
Potential barriers to transition from informal to formal
From the same survey conducted in April 2012, I asked participants to identify
barriers to formal music learning for young people, based on their own practice.
There was a set list of barriers based on my own experience and the option to
add others. Because of this, the identification of all potential barriers is limited
and needs further work. The results are presented in a bar chart (figure 6):
Musical transitions from informal to formal: effective teaching resources | Kevin Grist |
9
Figure 6: Most common barriers for young people accessing formal music education taken from a
poll of 35 music practitioners in April-May 2012 (SurveyMonkey/Wordle)
There is clear indication that particular barriers are considered more common
than others; incorrect teaching style, low confidence of the learner,
intergenerational low aspirations of the learner, to name the top three. It suggests
that there is commonality in issues around access to formal music education –
something that music education hubs could potentially address. This list informed
another phase of my research; to observe non-formal music education sessions
at two different transition points on the learning continuum (figure 4). I chose to
observe a music session at Steyning Grammar School, East Sussex (year 9
choosing their GCSE options) and one at the Crew Club youth centre, Brighton
(includes 16-19 year-olds working to progress to formal music education). The
criteria for the observation was constructed using:
1. The top 10 barriers to learning (as identified in the survey and figure 6) – I
would comment on how the session addressed the barriers, if at all
2. The two most important non-musical learning outcomes for an informal
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 10
session (as identified in the survey)2 – I would comment on how the
session met these outcomes, if at all
2 – 35 music practitioners surveyed in April 2012 were asked to identify: “Which one of the following outcomes is most
important to achieve during an informal music session?” The choices were: to develop literacy and numeracy, to improve
social and emotional wellbeing, to encourage creativity, to introduce diversity and to make use of inclusive technology. 2/3
chose ‘to encourage creativity’ and 1/3 chose ‘to improve social and emotional wellbeing.’ The other outcomes were not
chosen. The outcome of ‘making music’ was deliberately omitted as an option, so to draw focus on the potentially wider
benefits music education.
Figure 7 shows how the observation sheet was constructed. I wanted to find out
how existing practice has been working to aid transition from informal to formal
by tackling common barriers and meeting common outcomes. Using results from
this, and the earlier research into how music learners achieve (mapped on the
learning continuum), I hoped to be informed enough to identify effective teaching
resources later in the study.
Observation Sheet: Musical transitions from
informal to formal: effective teaching resources
Aside from making music, how did the session meet the following outcomes?
To improve social and emotional
wellbeing
Session date:
Time:
Venue:
To encourage creativity
Description of session:
Any other comments:
Top 10 barriers to formal music
education
How did the session address these
barriers, if at all?
The teaching style is not right
Having low confidence
Having intergenerational low
aspirations
The course fees are too high
Having no desire to gain accreditation
Having no previous qualifications or
experience
The equipment is not adequate or
inclusive
Not knowing anyone else at the
venue
The class size is too big
Having a learning disability
Figure 7: Observation sheet criteria based on survey results
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 11
The observation at Steyning Grammar School was with year 9 students working
towards a Rockschool Practitioner qualification, delivered using the informal,
learner-centred ‘Musical Futures’ model. This was at a time when many were
considering their GCSE Music options and making the transition from non-formal
to more formal approaches to music education. The one-hour session involved a
brief group recap with one teacher, before students (about 20 of them) went into
pre-formed groups and made music in separate spaces. This ranged from live
rock/pop/soul to computer-based electronic and classical. The students chose
the repertoire of music and were encouraged to listen to the pieces and replicate
using aural skills. The music teacher supported this as he visited each group to
provide feedback. The session addressed many of the barriers to formal learning
as identified in the survey. The teacher had adopted a dual teaching style
between formal and non-formal depending on the learner and their needs. So, for
example, he would use formal teaching methods at the group recap to check
learning and ask students to explain what the session’s aims were. Once in their
groups, he would take a more one-to-one mentor role, providing an informal and
often ‘chatty’ approach to questioning and giving feedback to individuals. Low
confidence in some students was addressed indirectly by working in smaller
groups and having peer support. These friendship circles are similar to
communities of practice mentioned earlier (Wenger, 2007) and provide a real
opportunity to develop wider outcomes for improving social and emotional
wellbeing (one of the key outcomes identified in the survey). Students kept
recordings of their work and a weekly rehearsal diary, which allowed them to
informally reflect and to recognise and track their progress and achievements.
Some of the work was uploaded to NUMU, a national website for schools and
youth centres to share young people’s music and create evidence portfolios.
Again, this reinforced a sense of achievement as students can get peer
comments from across the UK and is inclusive in the way that young people have
access to it. In terms of attitudes towards accreditation, 3 out of the 5 students I
spoke to thought it was important to have music qualifications and grades to
progress further. It is difficult to establish where those attitudes of aspiration
come from, but all understood that GCSE Music potentially means less practical
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 12
activity than the Rockschool Award. This acceptance of formal education may
come from external factors such as high expectations from parents and that the
classroom had lots of posters and literature promoting higher level theory and
contextual ideas.
The observation at the Crew Club youth centre was with 13-19 year olds
attending a Youth Music-funded project called ‘Brighton Music Hub,’ which I am
familiar with as I helped to set it up. The programme doesn’t use a prescribed
model of learning, but would be considered as using an informal to non-formal
approach to music education. This was at a time when many of the participants
aged 16-19 were considering progressions into level 1 and 2 college courses in
Music Performance and Music Production. For many of them, the project was
their first access to and experience of music education. The group size averages
20 per week and there is a core group of 13 who account for over 50% of the
attendance, of which 77% have a special educational need (Youth Music, 2012).
40% come from the socially deprived Whitehawk Estate, where there are a high
number of young people in challenging circumstances. The 3.5-hour evening
session involved an initial 20-minute slot for socialising, then a group recap and
icebreaker/warmer to welcome newcomers. Despite a core group who come
every week, the other young people vary. To overcome this, the grouping of
learners is decided on the night based on the need and motivation of the young
people. Like Steyning Grammar School, young people are put into smaller peer
groups to make music in separate rooms/spaces. There are 5 music tutors
supporting the session and each is assigned a group to deliver mini workshops in
singing, rapping, lyric-writing, studio recording and live band performances. The
groups break, reconvene, and then work towards a final group showcase of their
performance/recording in the main hall. The session addressed many barriers to
formal learning as identified in the survey. The teachers consist of 3 who are
qualified and 2 who are undertaking training to be community workshop leaders –
they are lead by a project manager. The teachers responded to the learners
needs – their choice of song, their special educational needs and wheelchair
adaption. Teachers discussed any issues of concern and attempted to resolve
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 13
them regularly throughout the session. Low confidence in the young people was
addressed by peer-supported groups, no pressure to perform and teachers
doubling up their roles as youth workers/mentors to support those who are
vulnerable. Intergenerational low aspirations were addressed by making small,
but regular steps of achievement – in the session feedback and end of evening
performance. There are also 2 young people who are trainee music leaders – this
means they have added responsibility to help assist in the delivery of sessions.
One supports the studio recording, whilst the other helps set up and run a rock
band group - the impact of this is to have peer role models. Likewise the teachers
are informal and share stories of their music industry/professional background
and serve as other authentic role models to aspire to. On occasions, external
colleges are invited to the project to showcase their students and courses in a
Q&A session - another strategy to change attitudes towards formal education.
Young people can access the Rockschool Award at a subsidised cost of £10, but
this is not compulsory. Like Steyning Grammar School, the Crew Club session
helped improve social and emotional wellbeing through dedicated social time,
social learning and peer support. Creativity was encouraged by the teachers
through merging genres/styles in the workshops and creating new compositions
including the use of mobile phones to record with.
Identifying effective resources for transition - conclusions
Taking into account the results of the survey conducted, lesson observations and
the research into mapping music education over a continuum, I am better
equipped to identify effective resources and strategies that could aid the
transition into formal learning. Based on the idea that reducing barriers and
achieving helps transition, I can summarise my findings:
1. Feedback – instant and regular through the form of small performances or
playbacks of work. Delivered by peers and teachers. Feedback is considered
to have the highest effect on achievement according to research by John
Hattie on effect sizes (Hattie, 2008).
2. High expectations of learners – setting goals and aims that are achievable
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 14
but challenging including performance deadlines and recording release dates.
This may encourage the idea of more formal deadlines and aspirations to
achieve higher.
3. Fun and engaging music making – aural-based music making being the
core of all sessions. The latest Ofsted report Music in schools: wider still, and
wider (Ofsted, 2012) cited that not enough actual music making was
happening in schools.
4. Diary keeping – keeping a diary or blog supports learners in keeping track of
their work and to visualise progress, boosting confidence. It is also an
effective way to compile evidence towards a practical-based music
qualification.
5. Using digital technology – using computers, phones or other devices to
record music and videos provides another instant way of analysing and
feeding back on a performance. Musical Futures, NUMU website is an
effective and safe space to upload and share such work. Garageband on the
iPad and Jampod devices for recording live music are amongst new effective
music technology.
6. Well-resourced spaces – the learning environment should be spacious with
multiple spaces with the inclusion of ample instruments, computers and
adapted technology.
7. Training – many teachers in formal education may have become
institutionalised and could benefit from some CPD to recognise or refresh
techniques in non-formal to formal teaching styles.
8. Installing inspiring role models - from both teachers with industry
experience and young trainee music leaders, to raise aspirations. Also to host
Q&A sessions from external colleges and students to offer progression routes.
9. Social areas – musical learning is a largely social experience, but having
extra time to socialise within the session (if it is long enough) is beneficial in
developing social and emotional wellbeing, particularly of the most vulnerable
learners.
10. Visual posters and literature – posters and leaflets that promote deeper
and more formal thinking around music theory/contextualisation, are useful to
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 15
introduce more formal ideas to the learning space.
11. High number of teachers – I believe a higher ratio of teachers to students
improves the quality of teaching to larger groups and helps assist with
differentiation and identifying issues with vulnerable learners.
12. Singular use of language – Having a unified and singular use of language to
describe informal and formal music education might be helpful in dispelling
myths around quality and recognition of informal education. By referring to a
community workshop leader as simply a ‘music teacher’ would also help
learners associate more formal terminology with an informal environment, and
help to ease the transition.
Bibliography
Arts Council England, (2012). ‘We announce successful music education
hub applications’ [http://www.artscouncil.org.uk/news/arts-council-news/weannounce-successful-music-education-hub-applica/. Accessed 19th May 2012]
Cross, J (2007) ‘All or nothing.’ Informal Learning Blog, 9th February
[http://www.informal.com/2007/02/09/all-or-nothing/. Accessed 10th May 2012]
Department for Education, (2012). The Importance of Music: A National Plan for
Music Education. London: Department for Education, Crown Copyright
Hattie, J. (2008). Visible Learning: A Synthesis of Over 800 Meta-Analyses
Relating to Achievement. Routledge
Hills, T. W. (1992). Reaching potentials through appropriate assessment (Vol. 1,
pp. 43-63). Washington, DC: National Association for the Education of Young
Children.
MusicLeader South East, (2012). Music Education Hubs: How can the formal and
non-formal music education sectors work together for the best outcomes for
children and young people? Berkshire: MusicLeader/Rhythmix
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 16
Ofsted, (2012). Music in schools: wider still, and wider. London: Ofsted, Crown
Copyright
Price, D. (2006). Musical futures: Teachers' resource pack. [S.l.]: DfES
Price, D. (2006). Musical futures: Personalising music learning. [S.l.]: DfES
Reinhard (2010). 'Teaching-learning processes between informality and
formalization', the encyclopaedia of informal education.
[www.infed.org/informal_education/informality_and_formalization.htm. Accessed:
13th April 2012]
Rogers, A. (2003) What is the difference? A new critique of adult learning and
teaching: Leicester: NIACE
Smith, M. K. (1999) 'Learning theory', the encyclopedia of informal education,
[www.infed.org/biblio/b-learn.htm, Accessed: 4th April 2012]
Wenger, E (2007) Communities of practice. A brief introduction,
[http://www.ewenger.com/theory/. Accessed 4th April 2012].
Werquin, P. (2010). Recognising non-formal and informal learning: Outcomes,
policies and practices. Paris: OECD.
Wolf, A (2011) Review of Vocational Education – The Wolf Report, Department
for Education UK: HM Government (2011)
Youth Music (2012) Open Programme Interim Report: Brighton Music Hub,
Brighton: Crew Club
Musical transitions from informal to formal: effective teaching resources | Kevin Grist | 17