The two year funded EMQT project

Comments

Transcription

The two year funded EMQT project
EMQT project (2009-2011)
Erasmus Mobility Quality Tools
CONFERENCE “STRENGTHENING THE IMPACT OF
LEARNING MOBILITY”
CMEPIUS, 8TH DECEMBER 2011, LJUBLJANA
Lupo Donà dalle Rose
University of Padova
This project has been funded with support from the European Commission.
Agreement Number: 2009 – 3668/001-001
When?
ORIGIN & STARTING POINTS
The Bologna Process impact and its progression
towards the European Higher Education Area
“mobility is at the heart of the Bologna Process”
Difficulties in recognition, Nancy 2008; preliminary discussion took place
between Deusto and Padova…
… the idea subsequently became a concrete project, with the main
support provided and role played by the “Education Training and Mobility”
Task Force of the Coimbra Group.
Important documents are:
1. European Quality Charter on Mobility
2. Erasmus University Charter
3. Erasmus Student Charter
4. Green Paper on Mobility
Who?
The EMQT Consortium….
Project coordination:
University of Padova
14 Universities :
Padova, Bologna, Deusto, Aarhus, Åbo,
Charitè Berlin, Granada, Graz, K. U. Leuven,
Leipzig, Iaşi, Jena, Paris-Sud, Thessaloniki
3 Associations:
ESN-Erasmus Student Network, Brussels
EuroPACE ivzw (BE)
Coimbra Group Office, Brussels
1 National Agency:
Agencia Nacional de Evaluación de la Calidad y
Acreditación ANECA, Madrid
1 Private counselling partner:
CHE Consult (DE)
2 Associated Partner Universities: Trinity College Dublin, Turku University
What?
The EMQT project is an “Erasmus structural network”
Why?
AIMS and PURPOSES:
• To identify organisational models,
good practices, benchmarking
procedures and related indicators,
which should characterise the
quality of the students’ exchange
mobility.
• To define a QTB – Quality Tools’ Box
and develop guidelines for different
“Quality patterns or profiles”, where a
given institution can position itself,
and for mechanisms of accountability
and of external validation.
How?
Physical (5+1x2) and virtual meetings (8)*
– through online and telephone conferences of the Managing Committee and the Plenary Assembly
* Skype and Flashmeetings
5+1 General Meetings
Padova (kickoff meeting)
Brussels
Leuven
Bologna (1st Open Seminar)
Brussels (Final Validation Conference)
workshops
and Where?
Task Forces (6) and respective chairs
General organisational models
within HEIs
Language preparation
Universidad de Granada
Information and orientation
Aristotle University of Thessaloniki
Students’ performances and
recognition
Alma Mater Studiorum Università
di Bologna
Reception of host students
Karl-Franzens-Universität Graz
e-Coaching or ICT mobility tools
Katholieke Universiteit Leuven
CHE Consult
How…
The Methodological framework
• By acting through 6 Task Forces (TFs)
each one having its own deliveries &
each TF following the
GAIN sequence
G : identification of Goals
A : Actions
IN : INdicators
• By producing Questionnaires, both at internal and at external level of HEIs
• By drafting Mapping Reports
• By identifying Good Practices
• By listing key Indicators
And what about the academics’ point of view?
• EMQT statement on the meaning of academic quality
during a study period abroad
• 85 interviews with academics in partner institutions
What are the main mobility goals?
I1. Opening doors to other kinds of mobility
INSTITUTION
I2. Boosting reputation and increasing visibility of the HEI through ERASMUS
and cooperation
I3. Enriching the institution teaching offer and services by international
mobility
I4. Achieving institutional awareness of intercultural diversity
STUDENTS
S1. Allowing every student an ERASMUS mobility according to his/her
academic needs
S2. Achieving transversal competencies and awareness of intercultural
diversity (links also to the society)
S3. Ensuring the most successful stay with emphasis on academic
achievement
SOCIETY
So1. Building awareness of European citizenship
So2. Fostering interaction between HEIs and non-HEI
organisations as well as the civil society
Priority in the goals according to EMQT survey
order found in the
questionnaire lay-out
GOALS
MERGED SAMPLE
INTERNAL
EXTERNAL
Ensuring the most successful stay with
emphasis on academic achievement
1°
1°
1°
S1.
Allowing every student an ERASMUS
mobility according to his/her academic needs
2°
3°
2°
3°
4°
3°
4°
2°
6°
5°
7°
4°
6°
6°
5°
7°
5°
7°
8°
9°
8°
9°
8°
9°
S3.
7
5
2
1
4
3
6
I2. Boosting reputation and increasing visibility of the
HEI through ERASMUS
I1. Opening doors to other kinds of mobility and
cooperation
I4.
Achieving institutional awareness of
intercultural diversity
I3.
Enriching the institution^s teaching offer and
services by international mobility
S2.
Achieving transversal competencies and
awareness of intercultural diversity (links also to the
society)
So2.
9
8
Fostering interaction between HEIs and nonHEI organisations as well as the civil society
So1. Building awareness of European citizenship
What are the EMQT outputs?
EMQT Tools’ Box: Main components
1. EMQT Questionnaire and Glossary
2. List of key/main Indicators
3. Suggestions for Good Practices
4. Report on Academic Quality in
exchange mobility
What are the EMQT outputs?
EMQT reference documents
1. Quality in mobility: how to measure and
assess it, a paper by Maria Sticchi Damiani
2. EMQT Mapping Report
3. Paper on the Recognition Process
Facts and figures: the mobility rate
Erasmus area:
7.7
students out of 1,000 (SMS, EC data
for 2008-09)
• Internal sample
14.9
(max 31.8 ÷ min 4.8)
21.4
(max 77.7 ÷ min 3.4)
18.8
(max 77.7 ÷ min 3.4)
13 universities
• External sample
65 HEIs
• merged sample
The EMQT questionnaire
1. early version – tested internally
≈ 200 questions, 340 fields
2. revised version (95% overlap, very similar)
– tested externally
3. Toolbox version
≈ 200 questions, 340 fields
PAGE 1
…
PAGE 3
EMQT Questionnaire
Section 1: Organisational Models
For those terms which seem unclear, we advise consulting the EMQT glossary. Initials and acronyms
are also included in the glossary under the term “Initials”.
The numbers you fill in should refer to the last academic year for which these are available, keeping in mind to
fix that academic year for all the numbers. So, if for certain questions the last available data refer to the
academic year 2008/2009 and for others these refer to 2009/2010, please keep 2008/2009 as the academic
year of reference for all the questions, unless otherwise explicitly specified.
Part I - General Information
I.I. CONTACT PERSON / ADDRESS
Contact Person / Address
Please indicate the details of a contact person in charge of this questionnaire and the address of the
structure/service of your Higher Education Institution (HEI) to which the contact person belongs.
Name of HEI: original name & (official) translation into English__________________________________
ERASMUS ID code__________________________________________________________________________
Country_______________________________________________________________________________
Contact Person
Name of contact person_________________________________________________________________
Department/Unit_________________________________________________________________ ____
E-mail-address_________________________________________________________________________
I.II. CENTRAL-LEVEL DATA
I.II.I Total number of staff in your HEI (in FTE - full time equivalents)
a) Academic staff (teachers and researchers)
b) Administrative and technical staff
I.II.II Number of offered Degree Courses
1st cycle
a.
2nd cycle
b.
1
c.
3rd cycle
d.
Pre-Bologna organisation of studies
e.
One-tier degrees
I.II.III Total number of enrolled students
1st cycle
a.
2nd cycle
b.
One-tier degrees
c.
3rd cycle
d.
Pre-Bologna organisation of studies
e.
_________
_________
1.1 Institutional backbone for ERASMUS mobility
1.1.1. Is mobility made explicit in the mission statement of your institution?
 yes
 no
1.1.2. Does your institution have a strategy on ERASMUS agreed upon at institutional level?
1.1.2.1. If yes, is the ERASMUS strategy encompassing the following cycles?
a) 1st cycle (BA)
b) 2nd cycle (MA)
c) 3rd cycle (doctoral)
 yes
 no
 yes
 yes
 yes
 no
 no
 no
1.1.3. Does your institution have an office for the ERASMUS programme?
a) As an independent unit
 yes  no
b) As part of an IRO
 yes  no
c) Either independent or part of an IRO but in strong synergy with other services (registrar office,
career guidance service, computing centre, etc.)
 yes  no
d) No, but there is multi-task administrative staff taking care of ERASMUS
 yes  no
1.1.4. What is the number of staff at your University for the administration of the ERASMUS programme?
_________
1.1.5. What is the total number of personnel; academics, administrative and technical staff involved at the
ERASMUS programme management?
a) Academic staff (i.e. ERASMUS Coordinators/Advisors1)
________
b) Administrative and technical staff (ERASMUS Offices)
________
1.1.6. Does your institution provide incentives for staff to get involved?
a) Academic staff
 yes  yes, in most cases  yes, in few cases  no
b) Administrative and technical staff  yes  yes, in most cases  yes, in few cases  no
1.1.6.1. If yes, could you briefly describe them? _______________________________________________
1.1.7. Management system on quality for the ERASMUS programme; Does your institution use a quality
management strategy for the ERASMUS programme?
 yes  no
1.1.7.1. If yes, could you briefly describe it?___________________________________________________
1.1.8. Does your institution monitor the mobility of students according to numerical indicators?
 yes  yes, in most cases  yes, in few cases  no
1.1.9. Does your institution use the results of the monitoring process for strategic decisions?
 yes  yes, in most cases  yes, in few cases
1.1.11. Does your institution draw on results from monitoring and surveys for quality improvement activities?
 yes  yes, in most cases  yes, in few cases  no
1.1.12. Does your institution set corridors on (some) key numerical indicators for improvement in respective
value?
 yes  no
1.1.12.1. If yes, on which indicators? ______________________________________________________
I.II.IV Mobility
1.1.13. Does your institution adjust corridors according to actual indicator development?
1
1
See glossary
 no
1.1.10. Does your institution run regular satisfaction surveys related to ERASMUS with:
a) Students?
 yes  no
b) Academics?
 yes  no
c) Others: ___________________________________________________________________
Ibidem
 yes
 no
list of EMQT main indicators - 1
• Indicators as a hint to the quality of istitutional actions
aimed at a certain mobility goal (a priori group reflection)
• indicators were then translated into questions, to be
asked for in the questionnaire
most answered / seldom answered
• Internal reflection/debate based on agreed methodology:
after several stages, from a questionnaire with 340
questions, the TFs selected 4 general indicators plus 31
indicators
list of EMQT main indicators - 2
General Part
•
•
•
•
1. number of outgoing study mobility
2. study mobility rate
3. number of incoming study mobility
4. average duration of study mobility
list of EMQT main indicators - 2
General Part
•
•
•
•
1. number of outgoing study mobility
2. study mobility rate
3. number of incoming study mobility
4. average duration of study mobility
LIST OF INDICATORS – TFa
organisational models
1. Existence of a strategy on ERASMUS agreed upon at institutional level
2. ERASMUS Office – Function




independent unit;
part of an IRO;
in strong synergy with other services (registrar office, career guidance service);
multi-task administrative staff taking care of ERASMUS)
3. Implementation of a quality management system for ERASMUS
4. Provision of incentives for staff to get involved
5. Monitoring the mobility of students according to numerical indicators
…..
7. total number of academic, administrative and technical staff (FTE)
involved in counselling and in managing ERASMUS student mobility
…
Suggestions for Good Practices
1. The point of view of a QA National Agency: extraction
of good practices from questionnaire returns (care of
ANECA)
2. cases of good practices identified by the 6 TFs
3. Document and example of implementation strategies
• Task Force b: Language preparation and related issues
For this dimension, both the provision of language courses by the institution
and the actions the institution takes to standardize the different language
levels are analysed.
•
•
•
•
•
•
•
•
•
Good Practices:
GP2.1. The institution offers pre-departure language units for outgoing
students and incoming students.
GP2.2. The institution offers semester/year-long language course units for
outgoing students and incoming students.
GP2.3. The institution offers language course units for specific purposes (i.e.:
Law, Medicine, Engineering, etc.) for outgoing and incoming students.
GP2.4. The institution offers language course units at different levels.
GP2.5. The institution follows strictly the Common European Framework of
Reference, CERF.
Interviews with academics
• 20 QUESTIONS
 Institutional aspects
6 questions
 Exchange coordinator aspects 7 questions
 Recognition issues
4 questions
 Problems and future
2 questions
Suggestions
1 question
THE INTERVIEWS
- 14 H.E.I.’S PARTICIPATING IN THE EMQT PROJECT
- AVERAGELY 6 ACADEMICS INTERVIEWED
OVERALL: 83 INTERVIEWED ACADEMICS
(AND 2 ADMINISTRATIVE STAFF)
METHODOLOGY OF ANALYSIS (1)
EXTRACTING RECURRING ELEMENTS FROM THE
ANSWERS AND COUNTING THE NUMBER OF
ANSWERS IN WHICH THESE ELEMENTS WERE
MENTIONED.
WE USED MATRIXES FOR THIS (EXCEL FILES) IN
WHICH WE COULD “TICK” THE RELEVANT COLUMN
ANY TIME THESE ELEMENTS TURNED UP IN AN
ANSWER.
METHODOLOGY OF ANALYSIS (2)
EXTRACTING INTERESTING AND/OR SURPRISING
ANSWERS WHICH RECURRED IN DIFFERENT
INTERVIEWS
SOME INTERESTING OUTCOMES (2/3)
Academics from 4 different universities mentioned the
differences in courses as interesting aspects that could
make them decide to set up agreements (different types
of learning or assessment, or different approaches to
contents), instead of the much more expected argument
of compatibility.
Interesting as well that in 5 out of the 6 quotes, the
academic was from the subject area of Science, who,
notably, tend to be more rigid on the recognition issue.
EMQT Mapping Report
General Index
Page
General Part.........................................................................................2
Section 1: Organisational Structures (Tfa)
Section 2: Language issues (TFb)
Section 3: Information and Orientation (TFc)
Section 4: Students’ performance and recognition (TFd)
Section 5: Reception of incoming students (TFe)
Section 6: e-coaching or ICT tools (TFf)
13
42
49
59
80
90
Appendix
FACTS AND FIGURES: an accompanying paper to the EMQT survey
106
The different sections offer an accurate description of the Erasmus
landscape, its physiology, established practices, etc.
Distribution of the external sample institutions
over geographical areas and mobility size
GEOGRAPHICAL AREA
DISTRIBUTION OF RETURNS FROM
EXTERNAL TESTING
A – NORTHERN
EUROPE
B – EASTERN
EUROPE
C – SOUTHERN
EUROPE
D – CENTRAL
EUROPE
GROUP 1: from 1 to 100 outgoings ;
MOBILITY SIZE
GROUP 1
GROUP 2
GROUP 3
11
3
1
6
7
10
2
9
4
3
4
5
GROUP 2: from 101 to 300;
GROUP 3: above 300
building the landscape of the European
mobility consisted in
• identifying and extracting important aspects and elements of
mobility as witnessed by the EMQT returns,
• giving a smart/realistic interpretation to the several –
sometimes conflicting – data collected, on the basis of
personal experience based competences,
• extracting ranges of variability for meaningful indicators,
the whole being offered in an coherent and
usable manner.
Important ingredients of the landscape are the values of several
“derived” indicators (e.g. the mobility; the reciprocity ratio; etc.).
examples of derived indicators - 1
derived indicator
Overall mobility rate
First cycle mobility rate
Second cycle mobility rate
One-tier degree courses mobility rate
Average
value
Minimum
value
Maximum
value
Number of
usable returns
18.8
3.4
77.7
40
17.0
20.7
20.1
2.0
2.2
1.4
78.4
50.0
23.9
30
25
14
(115.0)
Third cycle mobility rate
15.3
0.3
33.3
15
(138.9)
Number of outgoing students per
degree-course (overall)
Number of outgoing students per degree-course
(first cycle)
Number of outgoing students per degree-course
(second cycle)
Number of outgoing students per degree-course
(one-tier degree-courses)
Number of outgoing students per degree-course
(third cycle)
5.5
0.7
26.8
39
5.9
0.6
21.0
27
1.6
0.1
5.9
23
15.3
2.0
65.0
12
0.5
0.1
3.3
10
examples of derived indicators - 2
derived indicator
Average
value
2.05
Minimum
value
0.48
Maximum
value
7.8
Number of
usable returns
46
Ratio “academics” over “administrative
& technical staff”
Reciprocity ratio
average number of outgoing students
per Erasmus Office staff member
average number of outgoing students
per counselor/manager
AVERAGE VALUE OF EU MONTHLY
GRANT per STUDENT (in euro)
number of incoming students per
buddy/peer tutor
total number of outgoing students over
available places (as from BAs)
percentage of degree-courses offered in
English language
1.01
0.11
2.9
48
63.5
3
185
38
11.5
1
52
38
246
74
460
28
20
4.5
0.5
26
0.44
0.11
0.98
29
10.5%
0.2%
60%
23
The concept of profile
• for a group of respondents
• for a single institution
external versus internal testing
question
number
QUESTION
internal
sample
external
sample
57
13
83
5,4
NOTES
Section 1
1.1.4.1
1.1.5 a&b2
average number of outgoing students per Erasmus Office staff member
1.1.15.1 3
are students included in Erasmus QA?
support schemes for students with special needs outgoing
67%
44%
92%
45%
ratio STA over SMS
ratio STT over SMS
0,073
0,027
0,172
0,051
pre-departure / pre-arrival language course units, outgoing
75%
32%
pre-departure / pre-arrival language course units, incoming
75%
61%
50%
75%
61%
87%
67%
39%
1.2.4.0
1.3.1.3
average number of outgoing students per counselor/manager
Erasmus area average = 0,170
Erasmus area average = 0,039
Section 2
2.1.1 .1
2.1.2 .3
2.1.3 .5
semester/year-long language course units, outgoing
semester/year-long language course units, incoming
language course units for specific purposes (subject related…),
incoming
1. long term preparation is quite
stronger in ET;
2. the students of the ET seems to
be more equipped with language
skills
Section 3
3.1.2.1e
33%
77%
42%
69%
3.2.1.1e
Social network tools - addressed to incoming
comunication of satisfaction monitoring results with
partners
buddy system in place: a) for outgoing
8%
50%
67%
91%
3.2.4.1e
buddy system in place: b) for incoming
motivate local students to get engaged as buddies
67%
80%
BAs used both ways
60%
65%
12%
7%
16%
5%
13%
16%
37%
47%
50%
73%
17%
27%
8%
36%
58%
88%
33%
79%
0,5%
12,5%
17%
46%
3.1.8.1e
ET more open to new technologies
Section 4
4.1.2.1e
4.1.2.1e
4.1.2.1e
BAs used only for outgoing
BAs used only for incoming
4.1.2.1e
sleeping BAs
4.1.3 - derived average ratio "SMS number/available places"
indicator
4.3.1.1e
pre-departure ToR sent to host institution
4.3.2.1e
offer special preparatory activities (discipline
outgoings
offer special preparatory activities (discipline
incomings
specific) to
specific) to
IT is more open to asymmetric flows
mutual trust vs modern QA?
Section 5
5.2.7.10
5.2.8.11
course units taught in English
degree programmes entirely taught in English
derived indicator percentage of degree-courses offered in English language
Section 6
6.3.2.13
monitoring feedback from mobile students online
ET more open to new technologies
…. Thus any institution can find out how to
position itself in the landscape (institutional
profile)…
the EMQT open questions
QUESTIONNAIRE
SECTION
general part
Sect. 1
Sect. 2
Sect. 3
Sect. 4
Sect. 5
Sect. 6
total
number of
question
columns in
EXTERNAL
testing
number of
open
questions
number of
numerical
questions
number of
closed
questions
50
79
27
41
45
60
38
340
100%
8
11
2
7
8
5
3
44
12,9%
33
19
6
2
16
4
8
88
25,9%
9
49
19
32
21
51
27
208
61,2%
An example of open question returns
Open question 4.3.9.1. – If your institution does not send out
students to go abroad to do exclusively research – why not? (12
valid returns)
1) it is forbidden by the Erasmus rules (e.g. as stated by
National Agency), 3 returns as a whole!
2) research can be carried out only together with other
activities (only 1 return!)
3) research project is considered as a study activity
4) thesis work in first cycle is not considered as research
(some testing institutions have only or prioritize the first
cycle)
5) almost no mobility in the 2nd cycle, where research can
be carried out
6) not required by students (in a medical faculty)
7) other European or National programs are best suited for
research activities abroad (3 returns)
What are the EMQT outputs?
EMQT Tools’ Box: Main components
1. EMQT Questionnaire and Glossary
2. List of key/main Indicators
3. Suggestions for Good Practices
4. Report on Academic Quality in
exchange mobility
What are the EMQT outputs?
EMQT reference documents
1. Quality in mobility: how to measure and
assess it, a paper by Maria Sticchi Damiani
2. EMQT Mapping Report
3. Paper on the Recognition Process
For any further information please visit the EMQT website
www.emqt.org
or contact: [email protected]
Thank you very much for your kind
attention!

Similar documents