Research Evaluation NHTV Breda University of Applied

 Research Evaluation NHTV Breda University of Applied Sciences Academy for Tourism Independent External Evaluation January 2013 Thierry & Schuiling Graaf Aelbrechtlaan 2 1181 SW Amstelveen www.tslearning.nl
2 RESEARCH EVALUATION Academy for Tourism NHTV Breda University of Applied Sciences January 2013 4 Content 1. INTRODUCTION 6 COMMITTEE ASSIGNMENT DOCUMENTS AND INTERVIEWS 6 6 7 2. EVALUATION 8 THE ACADEMY AND ITS RESEARCH UNIT GENERAL FINDINGS FINDINGS WITH RESPECT TO THE RESEARCH FINDINGS WITH RESPECT TO PROFESSIONAL PRACTICE (‘THE INDUSTRY’) FINDINGS WITH RESPECT TO EDUCATION FINDINGS WITH RESPECT TO THE ORGANIZATION OF THE UNIT FINDINGS WITH RESPECT TO THE SELF-­‐EVALUATION REPORT APPENDIX 1: PROGRAM VISITATION DAY JANUARY 28H, 2013 APPENDIX 2: DECLARATIONS OF INDEPENDENCE 8 11 13 15 16 17 18 19 21 5 1. Introduction Committee In the context of quality assurance NHTV organizes an external evaluation of its research units every six years. The independent external committee (IEC) that visited the research unit of the Academy for Tourism in the beginning of 2013 consisted of the following members: Prof. dr. M.S.H. (Mia) Duijnstee University of Applied Sciences of Utrecht, emerita (Chair IEC) Prof. dr. A. (Andrew) Holden Professor in Environment and Tourism, University of Bedfordshire Drs. W. (Wouter) van der Meer Owner/manager Blini Reizen Dr. G.J. (Gertjan) Schuiling Consultant Thierry & Schuiling and lecturer Vrije Universiteit Amsterdam (Secretary IEC) Assignment The Board of Governors acted as the principal of the evaluation assignment. The assignment is to: 1. evaluate the research unit of the academy with help of the evaluation questions of the BKO (Brancheprotocol Kwaliteitszorg Onderzoek)1; 2. provide recommendations for further development of the research unit in relation to knowledge development, professional education and professional practice. The five evaluating questions of the BKO are: 1. Is there sufficient relevant productivity, impact, appreciation and recognition in terms of: • knowledge development within the research domain? • valorization to professional practice and society? • significance for education and training? 2. Is work undertaken on the basis of a relevant and challenging mission and a clear research profile? 1
This is the Quality Assurance Protocol for Research. The Netherlands Association of Universities of Applied Sciences adopted this protocol for the period 2009-­‐2015 during the general meeting in December 2008.
6 3. Are the mission and the research profile secured by the portfolio and by the way in which the unit has been organized? 4. Is the deployment of people and resources sufficient from a qualitative and quantitative point of view? 5. Are the internal and external collaborative arrangements, networks and relations sufficiently relevant, intensive and long-­‐term? Documents and interviews The committee has first studied the documentation and then interviewed people involved in the research unit. The documentation we received: •
•
•
•
Self-­‐evaluation report of the Academy for Tourism, December 2012 Appendix, December 2012 Several publications of researchers of the academy Several theses of students. The committee had interviews with three associate professors2, six core researchers, six students who participate in research, six people who are responsible for linking research to education and industry, five external stakeholders and four directors of the academy and NHTV. See appendix 1 for the program of the visitation day and for the names of the people that were interviewed. On request of the committee Dr. Johan Bouwer, research director of NHTV, was present with all meetings, including the internal deliberations of the committee. The purpose was to ensure the continuity of the evaluation and the thinking about the developmental process. He did not participate in the discussion that led to the findings of the committee. During all six interviews the committee experienced the participants to be open and constructive and willing to have a critical dialogue concerning developmental challenges of the research unit. 2
NHTV knows three types of associate professors: (a) (key) associate professor; (b) personal associate professor; (c) work field associate professor. The committee interviewed one associate professor and two work field professors. 7 2. Evaluation The academy and its research unit The combination of professional education and research has been at the origin of NHTV and its Academy for Tourism (AfT). AfT is the original unit of NHTV and at the present the largest academy of the five academies of NHTV. Henceforth referred to as NHTV, the present University of Applied Sciences was the result of a merger in 1987 from two specialized institutions, the Dutch Scientific Institute for Tourism and Recreation in Breda (NWIT; 1966) and the National Traffic Academy situated in Tilburg (VAT; 1972). NHTV was founded with the intention of offering relevant education on behalf of professional practice, but also of applying scientific knowledge in these fields. Since then, NHTV has developed from an educational institute into an international knowledge institute. The Academy of Tourism aspires to be – and to remain – a centre of expertise in the field of tourism and recreation, doing both education and research. The education is organised in four tracks: •
•
•
•
The three year academic bachelor of science course in Tourism (a joint degree with Wageningen University); The four year professional bachelor’s course HTRO (Higher Education in Tourism and Recreation); The four year professional bachelor’ s course FBTR (Profession-­‐oriented bachelor in Tourism and Recreation); The professional master’s course TDM (Tourism Destination Management). The first and the fourth program are exclusively in English, while the second and the third are both in Dutch and in English. There is also an academic premaster Strategic Business Management & Marketing (SBM) that offers direct access to MSc programs in the Netherlands and Belgium. The research mission is phrased as follows: ‘AfT’s research unit exists to maximize the learning of future international tourism professionals by positioning lecturers as experts, connecting lecturers to international industry, inspiring lecturers to intellectual growth, and inviting students to participate in every stage of research themselves, thereby teaching them to think in a more responsible, critical, and evidence-­‐based manner. AfT research converts internal and external private and public sector international funding sources into a wide variety of research products that examine tourism in its varied economic, social, and political contexts to increase the value of tourism to global society, ultimately contributing to quality of life.’
This mission is founded in the following vision: 8 ‘AfT aims to become an internationally recognized expertise center that produces highly qualified and responsible tourism professionals who challenge and upgrade the industry with evidence-­‐based, research-­‐backed thinking. Supporting this educational core are projects funded by government subsidies and industry contracts executed by qualified, continually educated staff within a supportive, vibrant, informal research community, resulting in a wealth of national and international popular, professional, and academic products. Students participate in the creation and use of these products, while associate professors create unique mentorship, authorship, and funding opportunities for lecturers, all of whom have a role in research.’
The academy has a research program with three program lines, and each program line is subdivided in themes with associated knowledge groups: 1. The tourist a. Travel industry b. Visitor studies and heritage c. Storytelling; Consumer behaviour; Events3 2. The destination a. Destination branding b. Destination management c. Productivity 3. The context a. Sustainable tourism and transport b. Transnational healthcare4 c. Tourism education in an international context d. Coastal tourism5. The academy wishes to reduce the number of groups in which it specializes to make the research program more focused. A similar change is occurring in the educational offerings. The intention is that these offerings match the knowledge groups. There are seven associate professors working at a total of 3.5 FTE. They lead knowledge groups and are alternatively involved in lecturing or connections with industry in addition to researching. The academy comprises 180 staff members at 120 FTE, of which 86 FTE are lecturers and associate professors who deliver education in tourism content and develop knowledge in the form of teaching materials and research. Of the educational staff 77% has a master’s degree or higher. Over the past four years, this research unit has produced a total of 615 publications and other knowledge products, including 63 peer-­‐reviewed scientific articles as well as 166 professional publications. NHTV has defined four roles in which educational staff contributes to knowledge development: 3
These are knowledge groups of the Academy for Leisure in which staff of AfT participates. This is a knowledge group together with HZ University of Applied Sciences. 5
Idem. 4
9 •
•
•
•
Role 1: Educational implementation Role 2: Applied professional research, consultancy and design Role 3: Applied academic research Role 4: Fundamental academic research. The total capacity invested in knowledge development is 18.42 FTE out of 86 FTE (or 21%). This is the time staff spends in the four roles. The highest effort is concentrated in lecturers in role 2 (39%), and a quarter is invested in role 4, directly tied to PhD trajectories. External funding has been stable at around half of the € 1,000,000 per year investment in research capacity. The academy for Tourism collaborates with all other units at NHTV, with partner higher education and research institutes such as Tilburg University, Wageningen University, and Penn State University. Furthermore, AfT is deeply networked in the professional field of tourism, partnering with organisations such as NBTC, ANVR, KLM, Amadeus and many others on specialized cooperative education and research projects. The academy describes its educational outcomes as follows. All AfT students are involved in research at some point in the process of fulfilling program obligations, such as research methods courses, placements, or theses. Others learn about research within content courses that utilize research processes. Still others earn optional study points by assisting lecturers with research projects. Research findings feed back into education through the upgrading of teaching materials, personal development of lecturers, curriculum upgrades, and integration of external experts into the educational process. 10 General findings This is an academy with a long tradition in doing both education and research. It is one of the few academies in the Netherlands that has both a professional and an academic bachelor and with a history of doing research long before research was introduced in higher professional education in the Netherlands. Having studied aforementioned documentation and interviews, the committee has come to the conclusion that the research function is working well in this academy. It shows impressive results and strong internal and external collaboration. The academy has developed its own journals and annuals, creating a platform for the dissemination of the work of its researchers as well as dialogues and discussions with others. Research and education are very well integrated. There is a strong group of seven associate professors. The report already gave us the impression that researchers work well together, and with the educational staff, and we see this confirmed during the interviews. The academy has a high ambition to have 70% of lecturers in the roles 2 and 3. The academy is performing well on all BO-­‐questions, except question two, where we evaluate the mission and research profile as sufficient. The academy is in the process of making the step from good to excellent. The approval of the Centre of Expertise by the Ministry of Education (together with HZ University of Applied Sciences and Stenden University) as well as the recommendations of this report can help to make this step from good to excellent in a few years. BKO-­‐question (1a) Is there sufficient relevant productivity, impact, appreciation and recognition in terms of knowledge development within the research domain? (1b) Is there sufficient relevant productivity, impact, appreciation and recognition in terms of valorisation to professional practice and society? (1c) Is there sufficient relevant productivity, impact, appreciation and recognition in terms of significance for education and training? (2) Is work undertaken on the basis of a relevant and challenging mission and a clear research profile? (3) Are the mission and the research profile secured by the portfolio and by the way in which the unit has been organized? (4) Is the deployment of people and resources sufficient from a qualitative and quantitative point of view? (5) Are the internal and external collaborative arrangements, networks and relations sufficiently relevant, intensive and long-­‐term? Table 1 The five BKO-­‐questions 11 There is clear evidence of knowledge development through research, evidence that is described in the appendix of the report and is also explicitly demonstrated through the discussions with the various groups (1a). This knowledge development has significance in the academic community, its rigor being evidenced by publications in refereed journals. The interface between the development of theoretical knowledge through research and applied knowledge through research appears to be well managed, integrated and with minimum tensions. There is also clear evidence of valorization (1b). During the committee’s discussions with associate professors, lecturers and stakeholders we learned that there certainly is valorization to the professional practice and society although the self-­‐evaluation report does not sufficiently reflect this. The strong ties with the professional practice are reflected in long-­‐standing relations with partners such as ANVR and KLM. The relation between AfT and the touristic practice is a two-­‐way street: AfT bases research on questions from practice, but also the tourist industry reacts on research output from the AfT. The research function has clear significance for education and training (1c). There appears to be no tension between education and research and both are well integrated with the industry. Although the differences between the educational goals of the different programs necessitates different levels of research methods education, lecturers do show to have a sharp eye for the meaning of research for the future professional practice of the students and its significance is also well recognized by students in their education program. Bringing results of research to the curriculum is well taken care of. The integration of education and research is well described in the report. The mission of the academy (2) has been relevant for the development phase of the research function of the academy. It introduces the significance of research in the education of future professionals and to the industry. Now a new mission is needed that defines the content in which the academy wants to specialize and the value it wants to add to education, professional practice and to the research domain. The research profile is too broad as the three program lines cover the whole field (tourist, destination and context). The knowledge groups have a clear focus but there should be more vision underlying and integrating the selection of the themes on which the knowledge groups are predicated. The next step is creating a sharper definition of the research profile, the external collaboration and the desired funding to make the step from good to excellent. The mission and the research profile are well secured (3). First by an organic and inclusive portfolio, second by the way the unit is organized in terms of the four roles and knowledge groups and third also by the smooth handling of the double identity of researcher and lecturer by the staff. The deployment of people and resources is good (4). The knowledge groups bring lecturers and associate professors together and they are a useful form of internal knowledge exchange and intellectual growth. Staff is enthusiastic and committed and has strong informal ties. Evidence suggests that staff feel supported in their research, for example through resources for conference attendance, time allocation for research in their workloads and a good library. The development of a research helpdesk as a reference point for help with research alongside more informal arrangements with more senior research staff is a positive step forward. It is also noted that staff are supported in their PhD endeavors and that this is important for the academic credibility as a university. 12 The strong internal collaborative arrangements are evidenced by the frequent contacts between lecturers in roles 1 and 2 and those in roles 3 and 4 and also by the contacts between students and the lecturers from roles 3 and 4. The collaborative arrangements between the AfT and other academies of the NHVT are extensively described in the self-­‐evaluation report. There are extensive external contacts, both nationally and internationally, both with universities and other partners. However, the panel misses a vision or strategy for developing and maintaining these contacts. AfT aspires to maintain its international recognition and positioning. This requires direction and strategy. What are the goals? In which dimensions does the academy want to be recognized in five years time? A strategy is advisable considering the coming challenges concerning research focusing, finding 3th stream funding and the new center of expertise. One other reason for this is mentioned in the self-­‐evaluation report: some external contacts of the AfT depend too much on individual researchers. Findings with respect to the research An important question for the committee regarded the strategy that will enable AfT to maintain its leading position in the Netherlands and internationally. It produces a high scientific output with a small amount of time for research. How will it improve on this output when there are pressures to raise the external funds? How will this impact the development of AfT? What kind of research does AfT want to do and what is the long term program for acquisition? The strategy was explained to be twofold: in 2007 the aim was set to gain access to the academic community and to simultaneously develop strategic partnerships with global players in the industry. This implied the task of changing the industry’s perception of the academy: the academy is not looking for student placements only, but has a big bank of knowledge and offers research capacity to investigate the questions with which the industry is concerned. As such the academy offers an attractive alternative to the hiring of management consultants. This strategy is spearheaded by seven associate professors that fulfil two different roles. The professors with an academic orientation are responsible for connecting lecturers and students to academic discourses and skills and for producing and supporting academic output. Associate professors with an industry orientation are responsible for connecting lecturers and students to industry issues, and for producing and supporting professional output. These work-­‐field associate professors have high-­‐level contacts in the industry as well in policy, for instance in the EU. All associate professors are responsible for coordinating efforts to secure research funding via subsidies and contracts. Both types of associate professors combine research and consultancy, but in different proportions. Research staff is happy with the research environment offered by the academy and with the available resources: knowledge groups, helpdesk, PhD support, access to scientific journals. They would welcome a lab facility with advanced software and technological equipment. This would enable them to do interesting research, for instance into the question of emotional reactions to tourism websites and other commercial material. Others suggested having more equipment and assistance to do qualitative research. 13 Researchers do not complain about a high workload. They reserve days or weeks in their agenda to do research and these arrangements are respected and facilitated by managers. Researchers feel no tension between theoretical and applied research. They feel they have the freedom to choose between these fields and establish links between them as they see fit. They experience the cycle between practice and theory, in the form of industry workshops, student theses and journal papers, as satisfying and inspiring. Individuals in the knowledge groups aim to do meta-­‐analysis of fieldwork. The academy has for instance done fieldwork in Bali for 15 years. The aim is to work with the material gathered during this substantial period from a research perspective. This is good example of role 3 type of research. The next step is to create focus, to determine where the academy wants to be in the lead and where it will tap in to others’ expertise. The Centre of Excellence will help to do this. Researchers expect this to result in a specialization of staff in one of the four roles. The variety in roles played by the researchers, the smooth transitions they have learnt to make between them and the ensuing productive relation between research and education may be compromised. The committee also sees this potential development as a possible risk for the present integration of research and education. Finally, the research is spread out over the knowledge groups, in which 72% of the lecturers participate (in FTE), spending 21% of their working time on knowledge development and research. Research is not a department separate from the educational department. Researchers meet students and lecturers on a daily basis, which they feel is fruitful for the integration of education and research. Recommendations to be considered: 1. Be more articulate in the link between NHTV’s KD&R profile and the research program of AfT. The visualization of NHTV’s profile in appendix, page 22, is on a very high level of abstraction, while the research program of AfT is described in terms of projects. Fill the space in-­‐between with intermediate concepts. 2. Define goals also in terms of content, not only in terms of the number of publications. 3. Specify research competences available and required, with a sharp eye for new methodological developments. Who is competent in what research methodology and who is leading the development of new methodological competences? For an academy of this level it does not suffice anymore to define qualifications of staff only in academic titles. A more refined type of knowledge management is possible and necessary. 4. Develop a model of the interaction between the different types of associate professors. One could imagine a circle, with practice or industry at the right side and science at the left side of the circle. Work-­‐field associate professors work from right to left, academic associate professors work form left to right. How do they meet? Where do they meet? What kind of research do they want to collaborate in? And who leads what area? 5. Define role 4 more broadly than only PhD trajectories. A research program for role 4 needs to be more than a program for PhD trajectories. There are fundamental questions in the field of tourism, pick these up when no one else does and continue to facilitate capacity for doing fundamental research into these questions when needed as a basis for further development in the field. Role 4 research is also necessary to keep track with methodological developments, to 14 create a leadership position internationally, to earn recognition from the university, in addition to produce PhD’s. 6. Be more assertive on the leadership role the academy can play for the industry. The question in the report ‘is it our prerogative to think beyond industry’s current needs and instead try to move industry forward, directly and through the knowledge of our graduates’ (p. 10) should be answered with a full and resounding ‘yes!’. During the discussion the associate professors gave good examples of industry following the academy’s research. 7. Describe the research program as a complete bundle of research projects and the criteria for selecting and accepting projects. The report shows nice examples of projects based on a good format. Apply this format to all projects and add a section on methodology. Findings with respect to professional practice (‘the industry’) The self-­‐evaluation report shows that AfT has strong and varied links with the industry. External stakeholders agree that AfT has high standing in the academic field as well in the business field. In the academic community this position is based on the growing international credibility of its research and on its collaboration with the University of Wageningen. Its impact in industry and policy can be seen in the ‘Industry Vision 2025’ document that was made here in the academy. This report analyses tourism as part of broader developments and is adopted by many entrepreneurs and associations in the travel industry, who are currently working on addressing the topics the report mentions. The academy is also a partner in publishing the yearly Trend Report Tourism, Recreation and Leisure of the Tourist Board. AfT is also seen as the natural partner by the Province of Zeeland and SME’s in Zeeland. Another example of the impact of research is the productivity ladder that is adopted by the Sociaal-­‐Economische Raad (SER) as an official recommendation to the Netherlands region. Stakeholders recommend AfT to strengthen the research leadership by elaborating a research culture and to add some more organization to the organic growth of the past. However beware that professionalization can be distorting and that there is some danger in emulating the current university professionalization. Keep building on curiosity based individual research and on respecting the human measure. They also recommend fine-­‐tuning the portfolio in dialogue with the industry and then streamlining the research agenda. Use the historical moment of the one million euro the government invests each year in a period of four years in the expertise centre to develop a research agenda with the field and with the University of Wageningen. This will lead to a higher visibility of the academy for the industry. Develop an international reputation in knowledge transfer between university and industry. This transfer needs the research agenda of the coming Centre of Expertise to be implemented. Management of the academy recognize that research is also about money. It wants to build up a stronger relationship with Wageningen and sees the Centre of Expertise (CoE) as an opportunity to ask the industry to work with the academy. The Board of Governors will finish NHTV’s new strategic plan 15 before the summer and considers defining four themes for the CoE. These match with the knowledge groups of AfT. The program for knowledge transfer involves role 1, 2 and 3. Lecturers in role 4 are involved in academic research and education in the context of the academic bachelor program in cooperation with Wageningen. Recommendations to be considered: 1. Consolidate current acquisition efforts into a comprehensive acquisition plan in which organizations (airlines, cities, tour operators, provinces?) are identified that give out assignments that help AfT to achieve the position it wants to be in within five years, and develop a strategy on how to acquire these assignments. 2. Give the work field professors the leading role at the beginning and the end of each research project starting with translating field questions into research questions and ending with valorisation projects to implement research outcomes in the field. 3. Make a project for developing the parameters for assessment of impact on the industry. The outcome is well described for instance on page 24 and 25 of the report, but the impact is vaguely indicated as ‘often used in industry’ (page 49). The question is how to substantiate ‘often used’. Also look at implementation outside own region. Make a valorization agenda and follow implementation trajectories actively, to detail benefits. 4. Prevent role 3 from sliding back into role 2 by relating role 3 to the academic research agenda. The danger of the need to raise funds is that role 3 will also be doing consultancy. Findings with respect to education The self-­‐evaluation report shows a strong integration between education and research. Students hear about research from the first or second year of their study. Lecturers tell students how they apply in practice what they teach students in the course. In the second and third year students have a course in quantitative and qualitative research methods, though not in one of the professional bachelor’s courses (FBTR). The academic bachelor’s course prepares students to become a researcher. Already in their first year they already write a research paper. Staff aims at one publishable paper together with a first year student each year. This bachelor is a joint degree with Wageningen University. Students do their minor and thesis in their third year in Wageningen. Students of the professional bachelor’s programs say they learn from the students of the academic bachelor’s course. The goal of the professional master’s course is to produce new knowledge and advice for stakeholder companies. The master’s program starts with a course on research methods in which students prepare a research plan. Then field trips are made to countries around the world during which research is done and 16 on-­‐the-­‐spot lectures are given on doing analysis. The third and last phase is analysis and advice and writing a thesis. Students and lecturers mention three reasons for learning how to do research: 1. to know the field well. This is a fast moving industry, the only way to keep up with this pace is knowing how to do research; 2. to solve problems as a practitioner later on, for instance as a consultant; 3. to learn to see from different perspectives, to be flexible in your mind, to be open to different opinions. Research is an international language, it is a way to communicate and deal with different opinions instead of arguing. An impressive moment was when a student from Bhutan explained how research helped her to make the transition from a culture in which it is suicide to challenge your teacher to a culture in which challenging your teacher is the norm. An attitude of inquiry helps one to cope with cultural differences. This is a good example of the ‘international classroom’ of AfT. Lecturers feel that 10% of research time to be allocated to role 1 and 2 works well. There is no clear boundary between collecting and then lecturing knowledge: it is an on-­‐going process. The 10% is a formalization of a process that has been going on for several years now. Recommendations to be considered: 1. Maintain the high level of integration of education and research. 2. Facilitate that students in all bachelor programs can prepare for the option to do research in their thesis. 3. Explain students the relevance of doing research also in the perspective of becoming a reflective practitioner, in addition to being competent in collecting information and viewing things from different perspectives. 4. Link student research to the research agenda of the academy by aggregating and analysing research that students have done on a certain topic during many years. Findings with respect to the organization of the unit The concept of the four roles is likely to be the foundation of the academy’s strong integration of research and education. It represents a significant innovation in the field of higher education, as lecturers are recognized as contributors to knowledge development in their educational work. People in role 1 and role 2 are stimulated to define research questions, go to conferences and transfer research outcomes to the curriculum. Innovation involves the creation of a new language. After the introduction phase of this concept NHTV should now work – among other things -­‐ on the streamlining of the wording of this concept. To stimulate this we have the following recommendations: 17 1. Use the term ‘knowledge development’ for the whole process. This fits with the vision of ‘NHTV as a knowledge institution’. 2. Speak of knowledge development roles, as is done on page 35, not of ‘KD&R roles’ as is done on page 4. 3. See research as a part in the process of knowledge development. 4. One could imagine the process of knowledge development to have four phases: a. Articulating research questions from practice b. Studying the literature and providing answers based on the literature c. Doing research d. Transferring findings to others 5. NHTV wants to do proper research, so delete the word research from the description of roles 1 and 2. Role 1 and 2 are knowledge development roles, but not research roles. We suggest to name role 1 ‘supervising student-­‐driven knowledge development’ and to call role 2 ‘practice development’. Both roles base their specific contribution to knowledge development on research methods and outcomes. 6. Role 3 involves practice-­‐oriented research. However, relevant questions not only come from the field, as the table on page 4 of the self-­‐evaluation report states, but also from the scientific literature. Similarly, the literature is relevant to build on for answering questions from the field. 7. Correct the title of the table on page 29. This table does not represent ‘% of total research unit FTE’, but academy staff active in knowledge development (in the four roles). Such a streamlining of the definitions of the roles will avoid a devaluation of research. At the same time it will give full recognition to the important function of knowledge development in education and consultancy, when both are done with an inquiring and reflective attitude and linked to the methods and outcomes of scientific research. Findings with respect to the self-­‐evaluation report The report gave an excellent basis for discussion. It is quite informative, clear, concise and well structured. It shows consistent reasoning and demonstrates the enthusiasm of the academy. It has a good summary that immediately draws the reader into the rest of the report. A flaw is that the report does not contain an explicit external stakeholder evaluation of the research program. Recommendations to be considered: 1.
2.
3.
4.
Make a special report of the research program. Do not make this a chapter of the appendix. Also write a separate valorization program. Ensure to have stakeholder information and evaluation in the next self-­‐evaluation report. Describe the process of making the self-­‐evaluation report in the next self-­‐evaluation report. 18 Appendix 1: Program visitation day January 28h, 2013 19 20 Appendix 2: Declarations of Independence 21 22 23 24