The 7th ORPHEUS Conference 2012: Establishing Evaluation of PhD Training
Mission of the Conference
7th ORPHEUS 2012 Conference: “Establishing Evaluation of PhD Training”
Throughout Europe there is a growing emphasis on PhD education as the “third cycle” in the Bologna process. For the health sciences, this provides new opportunities for the advancement of clinical research as well as for strengthening basic research in the area. This is, however, dependent on the content and quality of the PhD degree, and it was for this reason that ORPHEUS was established.
ORPHEUS (Organisation for PhD Education in Biomedicine and Health Sciences in the European System) held its first conferences in 2004 and 2005, where standards for the health science PhD were discussed and defined. In particular, the “Zagreb declaration” was formulated, which defined the expected content of a PhD thesis in the biomedical context. The third conference in 2007, discussed the role of clinical PhD programmes in graduate schools in health science. The fourth meeting held in 2008 in Aarhus set the basic standards for the PhD degree in biomedicine and health sciences. The fifth conference was held in Vienna 2007, with the motto “The Advancement of European Biomedical and Health Science PhD Education by Cooperative Networking”. The most recent conference 2011 was held in Izmir, Turkey, with the theme: “Quality Indicators of PhD training”. The position papers from each of these conferences can be found on the ORPHEUS homepage (http://orpheus-med.org). ORPHEUS now has a membership of around 100 biomedical and health science faculties and institutions from almost all European countries.
Recently, ORPHEUS in collaboration with Association of Medical School of Europe (AMSE) and World Federation for Medical Education (WFME) issued a joint document: Standards for PhD Education in Biomedicine and Health Sciences in Europe. The document is the product of discussion at numerous conferences with hundreds of participants. The aim now is to discuss how these standards can best be promoted and applied. In particular, whether some form of ORPHEUS label could be offered to institutions meeting these standards. Such actions could have an important effect on improving the quality of PhD training in European biomedicine and health sciences, and preserving the reputation of the PhD degree.
The forthcoming seventh conference, to be held 19-21 April 2012, in Bergen is on the theme “Establishing Evaluation of PhD Training”. On this topic, keynote speakers will provide impulses that we will further discuss in workshops to elaborate guidelines and strategies on ORPEHUS organized international evaluation of PhD training in Europe.
Local Organizing Committee
Local organizing committee:
Roland Jonsson, Conference Chair of the 7th ORPHEUS Conference in Bergen
Berit Rokne, Deputy Rector UiB
Robert Bjerknes, Vice Dean
Gry Kibsgaard, Advisor UiB
Jan Petter Myklebust, Assistant Director, UiB
Rune Nilsen, professor
Ernst Omenaas, Chief of Research, Haukeland Univ. Hospital
Reidar Thorstensen, Advisor, Haukeland Univ. Hospital
Kate Frøland, Senior Executive Officer
Torunn Olsnes, Senior Executive Officer
Marianne Stien, Higher Executive Officer
Inger Hjeldnes Senneseth, Assistant Director
Merete Allertsen, Senior Executive Officer
Ane Brorstad Mengshoel, Higher Executive Officer
Tone Friis Hordvik, Head of Administration
Anlaug Lid, Senior Executive Officer
Executive committee ORPHEUS:
Professor Dr. Zdravko Lackovic, Zagreb, President
Professor Dr. Michael Mulvany, Aarhus, Vice President
Professor Dr. Seppo Meri, Helsinki, General Secretary
Professor Dr. Andrea Olschewski, Graz, Treasurer
Professor Dr. Miroslav Cervinka, Hradec Kralowe
Professor Dr. Gul Guner-Akdogan, Izmir
Professor Dr. Konstantin Gurevich, Moscow
Professor Dr. Petr Hach, Prague
Professor Dr. Jadwiga Mirecka, Krakow
Professor Dr. Chris Van Schavendijk, Brussels
Professor Dr. Andre Nieoullon, Marseilles
Professor Dr. Roland Jonsson, Bergen
Co-opted Members of Executive Committee:
Professor Dr. Luis Martinez Millan, Vizcay
Dr. (PhD) Debora Grosskopf-Kroiher, Cologne
Declaration
|
UNIVERSITY OF ZAGREB PhD Programme:
|
EUROPEAN CONFERENCE ON University of Zagreb – Medical School
|
The Declaration of European Conference on
Harmonisation of PhD Programmes in Biomedicine and Health Sciences
Convened in Zagreb on April 24 – 25, 2004
(hereafter referred to as the «Zagreb Declaration»)
After extensive discussion and exchange of ideas and experiences among participants coming from 25 universities and from 16 European countries having different schemes for obtaining PhD degree in medicine and health sciences regarding both form and the way of evaluation, ranging from monograph and evaluation within the same university to high standards of PhD thesis containing four or more papers published in internationally recognized peer reviewed journals, often with high impact factor and the inclusion of evaluators from abroad, the participants of the European Conference on Harmonisation of PhD Programmes in Biomedicine and Health Sciences (hereafter referred to as the «Zagreb Conference» or the «Conference») have agreed on the following:
Article 1
PhD programme is intended to enable individuals, after completing and defending their PhD thesis, to carry out independent, original and scientifically significant research and critically evaluate work done by others. To assure the above, the participants of the Conference reached consensus on the following:
Article 2
As in any kind of scientific peer review process, the reviewers of PhD thesis should be competent and independent from the PhD thesis, candidates and supervisor. In this sense, the participants of the Conference would like to encourage the inclusion of supervisors from other universities and countries.
Article 3
The Conference agreed that a suitable benchmark to describe the necessary achievement is a PhD thesis based on original in extenso publications in internationally recognized scientific-medical journals. The independent contribution of the candidate should be clearly demonstrated (for example the candidate being the first author). The Conference recommends that the minimal requirement for the PhD thesis in medicine and health sciences should be the equivalent of at least three in extenso papers published in internationally recognized journals. In addition to the papers presented the candidate should provide a full review of the literature relevant to the themes in the papers, and, where necessary, a fuller account of the research methods and results. Where the PhD research is presented in other formats, such as the single monograph, reviewers should demonstrate that the contribution is at least equivalent to this benchmark, and should encourage inclusion of publication from the research.
Article 4
While the main demonstration of the achievement should be the thesis and published papers, PhD programmes should include a theoretical basis as well as the development of technical research skills in taught courses where appropriate.
Article 5
The Conference recommends to all universities to make their PhD programmes publicly available to students, lecturers and tutors from other universities and countries. All medical schools are recommended to create their web pages and written material about PhD programmes in English and to make their programs open to candidates from other universities and countries. The Conference encourages the development of joint PhD programmes in order to enhance the link between the European Higher Education Area and the European Research Area with a view to ensure higher quality and enable joint degree recognition.
Article 6
The development of well-designed and high-quality PhD programmes requires substantial support by medical faculties, universities, national governments, the European Commission or private sponsors and other institutions in order to engage the best medical students into scientific research so as not to lose our future in medicine and public health.
The Zagreb Declaration was adopted unanimously on April 25, 2004 at 2:00 P.M. by:
Conference participants
Representatives of international and Croatian professional / academic associations and governmental institutions (in alphabetical order)
Association of Medical Education in Europe (AMEE)
Prof. Jadwiga Mirecka, MD, PhD, Executive Committee member
Association of Medical Schools in Europe (AMSE)
Prof. Petr Hach, MD, PhD, President
Association of Schools of Public Health in the European Region (ASPHER)
Prof. Charles Normand, BA, DPhil, FFPHM, President
Croatian Medical Association
Prof. Ivan Bakran, MD, PhD, Vice-President
European Medical Association (EMA)
Vincenzo Costigliola, MD, President
German Academic Exchange Service (DAAD), South-Eastern European Cooperation, Curriculum Reform in Medicine
Prof. Hans Joachim Seitz, MD,
Ministry of Health and Social Welfare of the Republic of Croatia
Prof. Velimir Božikov, MD, PhD, State Secretary for Health
Ministry of Science, Education and Sports of the Republic of Croatia
Prof. Pavo Barišić, PhD, Assistant Minister
Prof. Aleksa Bjeliš, PhD, Vice-Rector
Prof. Helena Jasna Mencer, PhD, Rector
Representatives of medical schools and schools of public health (in alphabetical order by country name)
University of Mostar, Medical School, Mostar, Bosnia and Herzegovina
Prof. Filip Čulo, MD, PhD, Dean
Prof. Mirna Saraga-Babić, MD, PhD, Vice-Dean for Science
University of Sarajevo, Medical School, Sarajevo, Bosnia and Herzegovina
Prof. Jadranka Dizdarević, MD, PhD, Vice-Dean for Undergraduate Studies
Prof. Benjamin Vojniković, MD, PhD, Secretary General of the Medical School
University of Tuzla, Medical School, Tuzla, Bosnia and Herzegovina
Prof. Lejla Begić, MD, PhD, Vice-Dean for Science
Prof. Osman Sinanović, MD, PhD, PhD ProgrammeDirector
Prof. Husref Tahirović, MD, PhD, Dean
Higher Medical Institute of Pleven, Pleven, Bulgaria
Prof. Maria Simeonova, MD, PhD, Head of Medical Genetics Department
- J. Strossmayer University, Medical School, Osijek, Croatia
Asst. Prof. Gordan Lauc, MD, PhD, Vice-Dean for Education
Asst. Professor Ante Tvrdeić, MD, PhD, Vice-Dean for Postgraduate Studies
University of Rijeka, Medical School, Rijeka, Croatia
Prof. Anđelka Radojčić Badovinac, MD, PhD, Vice-Dean for Postgraduate Studies
Prof. Dragica Bobinac, MD, PhD, Vice-Dean for Graduated Studies
Asst. Prof. Zlatko Trobonjača, MD, PhD
Prof. Luka Zaputović, MD, PhD, Vice-Dean for Science
University of Split, Medical School, Split, Croatia
Prof. Mladen Boban, MD, PhD, Dean
Prof. Željko Dujić, MD, PhD, Coordinator of Postgraduate Studies
Prof. Stjepan Gamulin, MD, PhD, Head of Postgraduate Studies Committee
Prof. Marijan Saraga, MD, PhD, Vice-Dean for Science
University of Zagreb, Medical School, Zagreb, Croatia
Prof. Nada Čikeš, MD, PhD, ECTS Coordinator
Prof. Marija Dominis, MD, PhD, Vice-Dean for Postgraduate Studies
Prof. Boris Labar, MD, PhD, Dean
Prof. Zdravko Lacković, MD, PhD, PhD Programme Director, Deputy Dean for Postgraduate Studies
University of Zagreb, Medical School, Andrija Štampar School of Public Health, Zagreb, Croatia
Prof. Jadranka Božikov, PhD, PhD Programme Deputy Director
Prof. Luka Kovačić, MD, PhD Deputy Director
Prof. Stjepan Orešković, PhD, Director
Charles University in Prague, First Faculty of Medicine, Prague, Czech Republic
Prof. MUDr. Stanislav Štípek, DrSc., Vice-Dean for Pedagogical Affairs
University of Helsinki, Faculty of Medicine, Finland
Prof. Seppo Meri, MD, PhD, Head, Committee for Postgraduate Scientific Studies in Medicine
University of Hamburg-Eppendorf, Germany
Prof. Dr. Hans Joachim Seitz, MD, Director of the Institute for Biochemistry and Molecular Biology III – Biochemical Endocrinology
University of Szeged, Albert Szent-Gyorgyi Medical and Pharmaceutical Centre, Faculty of General Medicine, Szeged, Hungary
Prof. László Vécsei, MD, PhD, DSc, Director of the Experimental and Clinical Neuroscience PhD Programme
University of Dublin, Trinity College, Dublin, Ireland
Prof. Charles Normand, BA, DPhil, FFPHM, Edward Kennedy Professor of Health Policy and Management
University of Pavia, Faculty of Medicine and Surgery, Pavia, Italy
Prof. Alberto Calligaro, Deputy Dean
University “St. Cyril and Methodius”, Medical School, Skopje, R. Macedonia
Prof. Magdalena Žanteva-Naumoska, MD, PhD, Vice-Dean for Postgraduate Studies
Prof. Ljubica Georgijevski-Ismail, MD, PhD, FESC, Member of the Postgraduate Studies Committee
Norwegian University of Science and Technology (NTNU), Faculty of Medicine, Trondheim, Norway
Anne Britt Storeng, Senior Executive Officer, Research Administration
Prof. Alf O. Brubakk, Professor of Environmental Physiology
University of Oslo, Faculty of Medicine, Oslo, Norway
Sigrid Bergseng, Senior Executive Officer and Head of PhD Programme University Administration
Medical Centre of Postgraduate Education, Warsaw, Poland
Zbigniew Wegrzyn, MD, Department of Education and Quality Assessment
Jagellonian University, University Medical College, Kraków, Poland
Prof. Jadwiga Mirecka, MD, PhD, Head of the Department of Medical Education
Poznan University of Medical Sciences, Poznan, Poland
Prof. Maciej Zabel, PhD, Head of PhD Program
Iuliu Hatieganu University of Medicine and Pharmacy, Cluj-Napoca, Romania
Prof. Petru Adrian Mircea, Vice-President of the University
University of Niš, School of Medicine, Niš, Serbia and Montenegro
Prof. Goran Nikolić, MD, Vice-Dean
University of Novi Sad, Faculty of Medicine, Novi Sad, Serbia and Montenegro
Prof. Nevena Sečen, MD, PhD, Vice-Dean for Foreign Communication and Foreign Students
Comenius University, Jessenius Faculty of Medicine, Slovak Republic
Prof. Kamil Javorka, MD, DSc, Vice-Dean for PhD Study
University of Navarra, Medical School, Navarra, Spain
Prof. Alfonso Sánchez Ibarrola, MD, PhD, member of University PhD Committee
List of other invited lecturers not listed above (in alphabetical order):
Tina Dušek, MD, PhD student, University of Zagreb Medical School, Croatia
Dr. Guy Haug, Expert on the European Higher Education Area (Bologna Process), Bruxelles
Alena Kavalírová, graduated pharmacist, PhD student, Faculty of Pharmacy in Hradec Králové, Charles University in Prague
Dr. Cees C. Leibbrandt, MD, Former Secretary General (1999–2002) of the European Union of Medical Specialists (UEMS)
List of observers (in alphabetical order)
Sandra Belko, BA (English), PhD Programme Secretary, Medical School, University of Zagreb; Kristina Fišter, MD, Research Fellow, Andrija Štampar School of Public Health, Medical School, University of Zagreb; Asst. Prof. Ileana Linčir, MD, PhD, Vice-Dean for Postgraduate Education, University of Zagreb School of Stomatology; Prof. Josip Madić, DVM, PhD, Vice-Dean of Science and International Cooperation, Faculty of Veterinary Medicine, University of Zagreb; Prof. Albert Marinculić, DVM, PhD, Vice-Dean of Education, Faculty of Veterinary Medicine, University of Zagreb; Anita Putrić, BA (Political Science), Senior Executive Officer of PhD Programme Administration, Medical School, University of Zagreb; Marita Mimica, BA (psychologist), Head of Postgraduate Studies Department, Medical School, University of Split, Miroslav Sabolek, BA (economy), Head of PhD Programme Administration, Medical School, University of Zagreb; Assoc. Prof. Velimir Sušić DVM, PhD, ECTS Coordinator, Faculty of Veterinary Medicine, University of Zagreb; Tea Vukušić Rukavina, MD, Research Fellow, Andrija Štampar School of Public Health, Medical School, University of Zagreb.
Programme
|
13:00 – 15:00 |
Registration |
|
15:00 – 15:30
|
|
| 15:30 – 17:15 |
Key Note Lectures – Need for Quality PhD Training(Chairs: Zdravko Lackovic, Michael Mulvany) |
 Strategic challenges in doctoral training and the Innovation Union agenda of the EU – Stefaan Hermans, DG Research & Innovation, European Commission EC |
|
 Strategic Challenges in Doctural Education: Raising Our Game Across Europe – David Gani, EUA Council for Doctoral Education EUA-CDE |
|
 Marie Curie actions: Innovation and excellence in PhD training – Frederico Miranda, Marie Curie Actions, European Commission EC |
|
 ESF-EMRC overview on Medical Research Education in Europe – Pacini Giovanni,, European Science Foundation ESF |
|
| 18:00 – 19:00 |
Reception by invitation of the City of Bergen, Haakonshall |
|
|
|
| 19:30 |
|
April 20, 2012
![orpheus719veb[1].jpg](http://orpheus-med.org/wp-content/uploads/2012/06/orpheus719veb1.jpg "orpheus719veb[1].jpg")
Balancing the Ying/Yang of doctoral education – evaluating and examining Intended Learning Outcomes –
How to improve quality of PhD Training
April 21, 2012
| 09:00 – 10:30 |
|
| Doctoral Education as a Global Phenomenon – Thomas Jørgensen, EUA Council for Doctoral Education (EUA-CDE) | |
| Global doctoral education: Critical Conversations – Karen P. DePauw, Council of Graduate Schools (CGS), USA | |
 Short national reports |
|
| PhD education in Kazakhstan – Raushan S. Dosmagambetova, Karaganda Medical State University, Kazakhstan | |
| Doctoral candidates’ view on standards of education at Polish medical schools – Agata Skrzypek, Jagiellonian University, Faculty of Medicine, Krakow | |
| Evaluation of PhD work in Russian Federation – Konstantin G. Gurevich, Moscow State University of Medicine and Dentistry, Moscow | |
| Evaluating structured PhD programmes – Helen P. McVeigh, Royal College of Surgeons in Ireland, School of Postgraduate Studies, Dublin | |
| 10:30 – 11:00 |
|
| 11:00 – 12:30 |
Presentation of Workshop Papers and draft CONSENSUS DOCUMENTSChairs: Jadwiga Mirecka, Peter Hach) |
| Workshop 1: Miroslav Cervinka Workshop 2: Michael Mulvany Workshop 3: Zdravko Lackovic Workshop 4: Tove Ragna Reksten – Kirstine Kjær Kirkegaard Draft Consensus document: Roland Jonsson |
|
| 12:30 – 13:30 |
Lunch |
| 13:30 – 14:00 |
|
| 14:00 – 15:00 |
ORPHEUS General AssemblyChair: Zdravko Lackovic; Co-chairs Helga Ögmundsdottir, Jadwiga Mirecka; Minutes: Debora Grosskopf-Kroiher |
Key Note Lectures – PhD Training around the Globe
Break
Consensus Document
Participants at the 7th ORPHEUS conference in Bergen agreed the following.
Conference affirmed that the ORPHEUS/AMSE/WFME PhD standards document was a practical tool for quality assurance of PhD programmes.
ORPHEUS warmly endorses the new PhD principles document: “Best practice based principles for innovative doctoral training”, approved by the EU Council of Ministers November 2011. The PhD standards document is fully compatible with the PhD principles document, and ORPHEUS has expressed a wish to work closely with the European Commission on implementing this in the field of biomedicine and health sciences. ORPHEUS also looks forward to supporting the work of the EUA-CDE and the ESF in this field.
Conference recognized that the PhD standards document could provide a basis for global conversations concerning the quality and content of PhD programmes and ORPHEUS will work closely with WFME on this.
To increase awareness of the PhD standards document, national workshops should be held, and all ORPHEUS members are encouraged to initiate these. For practical quality assurance, ORPHEUS members should consider establishing benchmarking exercises nationally and internationally.
ORPHEUS encourages that awards of PhD financing by funding bodies should be based on in-depth assessment of the quality of PhD training in addition to assessment of scientific quality.
ORPHEUS will introduce a labelling project based on initial self-evaluation that can be verified through information given via the institution website. This may be followed up by some form of site visit from academics with experience in PhD education.
Conference agreed that internal evaluation must be an essential part of an institution’s quality assurance system for PhD education. The evaluation criteria have to be transparent and should be defined at the institutional level.
Conference agreed that the PhD students should be represented in all committees and decision-making organs. To promote active involvement of students, their efforts must be acknowledged, the infrastructure provided and their influence reflected in decision-making. PhD students should have the opportunity to evaluate supervision and supervisors; this point should be added in any revision of the PhD standards document. Results of such evaluations should be communicated to the students.
Concensus Workshop
ORPHEUS2012, Bergen, 19-21 April 2012
Workshop reports
Participants were divided into four workshop groups
- Workshop 1: Internal evaluation of PhD programmes
- Workshop 2: External evaluation of PhD programmes according to ORPHEUS standards
- Workshop 3: International evaluation and ORPHEUS labelling of PhD programmes
Workshop 4: The role of students in PhD training evaluation
Workshop 1: Internal Evaluation of PhD Programmes
In the ORPHEUS/AMSE/WFME PhD standards document, Chapter 8, Structure, it is stated:
“There should be approved procedures for regular review and updating of the structure,
function and quality of PhD programmes”. It was agreed that this should include all types of
feedback. Internal evaluation must be an essential part of the whole quality assurance system
and regular analysis of internal evaluations is the optimal way to achieve quality improvement
in PhD training. The following recommendations were made.
- Academic institutions should activate mechanisms to collect, analyse and respond to
evaluations, as a part of quality assurance of PhD training. - Each university should develop a formal framework for internal (intramural) evaluation
by collecting and acting upon feedback from all involved in PhD education (students,
supervisors, examiners, administrative staff, and others). - Internal evaluation should include evaluation of the institution
(university/faculty/programme faculty) and evaluation of performance indicators. The
evaluation has to be carried out at different levels (PhD programme / administration /
supervision / training). The evaluation criteria have to be transparent. - The workshop strongly recommends the evaluation of the supervisors and other
academics responsible for each student’s PhD programme. - Evaluation has to be carried out repetitively. The workshop recommends evaluations at
the time of the student’s admission to the PhD programme, midway, and at the end of
the programme. - Evaluation of the administration by the supervisors as well as by the PhD students
(doctoral candidates) should also be performed - Evaluation of all aspects of the PhD training programme (courses, time at other
laboratory, etc.) has to be included in quality assurance system.
Workshop 2: External evaluation of PhD programmes according to ORPHEUS standards
The workshop considered different ways in which the ORPHEUS PhD standards could be used
for performing external evaluation of PhD programmes. There were several views.
- External evaluation is necessary and should be provided by an external body of
auditors. It is, however, difficult to come up with a good practical means of
implementing this. Neither the Commission nor the EUA can act as accrediting bodies.
Standards could serve as guidelines, not regulations. There could initially be internal
evaluation by the institutions themselves, supplemented by institutional accreditation.2. Peer-review evaluation. Evaluation of applications by funding agencies for PhD stipends
could be made not only in terms of scientific excellence (publications, received funding,
etc.), but also on in-depth peer-review evaluation of PhD programmes. This approach
might be of interest in assessing Marie Curie applications.3. Benchmarking was seen as a practical means of external evaluation that could encourage
compliance with the ORPHEUS standards at grass-roots level.4. No formal external evaluation needed. This was a view of industry which is concerned
with outcomes. The outcomes of interest include academic excellence of the PhD thesis
as well as “ability to work in a team”. Thus industry places great emphasis on
collaboration with universities to ensure development of competences in transferable
skills. Academically trained employees who do not have practical skills are unlikely to
be attractive for industry. It was recognized that the institutions following ORPHEUS
PhD standards would provide the outcomes that industry needs.Representatives for all views underlined that the standards that are established by ORPHEUS
can form a basis for quality assurance. However, they should be dynamic and the organisation
should always aim for further quality improvements of the PhD programmes. University
autonomy must be respected. Any evaluation of PhD programmes should be performed by
academics with experience of PhD training. PhD programmes should be transparent. There
should also be transparency concerning the employability of the institution’s PhD graduates.
Students are likely to form a driver to ensure the quality of PhD programmes.These views were supplemented by comparison with the US system of accreditation of PhD
training. This is university-driven and there is reluctance to link evaluation of PhD programmes
with funding. The workshop representative from the US, Karen de Pauw, recommended that
data should be collected to determine the extent to which graduate schools complied with
ORPHEUS standards.Workshop 3: International evaluation and ORPHEUS labelling of PhD programmes
International evaluation is usually part of the evaluation of research. Accordingly international
evaluation of PhD programmes is especially important for smaller scientific communities to
ensure independent and competent evaluation of PhD programs (and PhD Theses). Even for
large and competitive scientific communities, international evaluation can be significant added
value. Beside it can be expected that international evaluation will facilitate international
collaboration of PhD programme providers.ORPHEUS-AMSE-WFME standards, have been developed over 7 years by ORPHEUS, with
participation of hundreds of health science schools, international associations such as AMSE
and WFME, and positive assessment of the European Commission EU , and EUA-CDE. This
PhD standards document is the only existing comprehensive document on PhD in biomedicine
and health sciences and is suitable for use in international evaluation.Participants of the workshop welcomed the idea of ORPHEUS labelling as a reward to those
ORPHEUS members organizing PhD program in line with the PhD standards document.
ORPHEUS labelling should be organised in a simple and not expensive way. Evaluation of
compliance of PhD programme with the PhD standards document should be the basis for
evaluation and labelling. PhD programmes should be described on the web site in English in
sufficient detail to allow “virtual evaluation”of compliance of PhD programmes with the PhD
standards document as the main requirements for ORPHEUS labelling. Participants had
different views about the necessity of site visits. Most participants would recommend site visits
only if it can be organised as short and not expensive procedure. Details of evaluation and
labelling should be developed by ORPHEUS Executive Committee.Workshop 4: The role of students in PhD training evaluation
Participants were grouped according to nationality and student status ensuring that as many
aspects as possible were discussed. Main focus of the discussions was student involvement in
the topics PhD programme organisation, supervision and instruments for quality improvement.PhD programme organisation. The following conclusions were drawn.
• Active and formal involvement in the PhD programme organisation: The PhD students
should be represented in all committees and decision-making organs. Student
involvement should be hierarchic constructed with different levels of influence, ensuring
that all students have a student representative for reporting issues. Student
representatives bridge the students and the PhD organisation, and facilitate the
communication between students and organisation. Student representatives identify
and communicate challenges and ideas to the programme organisation, and
communicate the outcomes to the students. To promote active involvement of students,
the efforts must be acknowledged, the infrastructure provided and the influence
reflected in decision-making. - Passive involvement: Mandatory evaluation of all courses taken, own project progress,
supervision and the PhD programme should be completed every term (semester) by all
PhD students. No approval of the term should be possible or accreditation of courses
given before this is done. - Supervisors should be evaluated both by the PhD programme and the PhD student. The
standards for supervision are stated by ORPHEUS in Chapter 5. These standards should
form a basis for objective, non-personal criteria for use by students in anonymous
evaluation of supervision. With basis in evaluation by the programme and the students,
quality of supervision should be monitored continuously with regard to both individual
supervisors and the general level of supervision in the programme. The results should
be communicated back to the students, and the latter should be publically available. It
was noted that several of the PhD students had no opportunity to evaluate supervision
and supervisors. This aspect is not part of the ORPHEUS standards. - At the beginning of the project, students and supervisors shall make a detailed progress
plan. The responsibility for supervision is shared between students asking for
supervision and providing information on progress, and supervisors for monitoring
progress and offering supervision where they see the necessity in following up the plan.
The student is responsible for informing the faculty in case of insufficient supervision. - PhD students shall have the opportunity for formal networking in settings outside the
research programme. Association with a broad network increases student involvement
and sense of belonging.
Conference Participants
Introduction
Conference affirmed that the ORPHEUS/AMSE/WFME PhD standards document was a practical tool for quality assurance of PhD programmes.
ORPHEUS warmly endorses the new PhD principles document: “Best practice based principles for innovative doctoral training”, approved by the EU Council of Ministers November 2011. The PhD standards document is fully compatible with the PhD principles document, and ORPHEUS has expressed a wish to work closely with the European Commission on implementing this in the field of biomedicine and health sciences. ORPHEUS also looks forward to supporting the work of the EUA-CDE and the ESF in this field.
It was recognized that the PhD standards document could provide a basis for global conversations concerning the quality and content of PhD programmes and ORPHEUS will work closely with WFME on this.
To increase awareness of the PhD standards document, national workshops should be held, and all ORPHEUS members are encouraged to initiate these. For practical quality assurance, ORPHEUS members should consider establishing benchmarking exercises nationally and internationally.
ORPHEUS encourages that awards of PhD financing by funding bodies should be based on in-depth assessment of the quality of PhD training in addition to assessment of scientific quality.
ORPHEUS will introduce a labelling project based on initial self-evaluation that can be verified through information given via the institution website. This may be followed up by some form of site visit from academics with experience in PhD education.
Internal evaluation must be an essential part of an institution’s quality assurance system for PhD education. The evaluation criteria have to be transparent and should be defined at the institutional level.
The PhD students should be represented in all committees and decision-making organs. To promote active involvement of students, their efforts must be acknowledged, the infrastructure provided and their influence reflected in decision-making. PhD students should have the opportunity to evaluate supervision and supervisors; this point should be added in any revision of the PhD standards document. Results of such evaluations should be communicated to the students.
Workshop reports
Participants were divided into four workshop groups
- Workshop 1: Internal evaluation of PhD programmes
- Workshop 2: External evaluation of PhD programmes according to ORPHEUS standards
- Workshop 3: International evaluation and ORPHEUS labelling of PhD programmes
- Workshop 4: The role of students in PhD training evaluation
The term ‘PhD programme’ in the following text encompasses both the concept of an organized programme involving several individuals, as well as an individual PhD programme for one person.
Workshop 1: Internal evaluation of PhD programmes
In the ORPHEUS/AMSE/WFME PhD standards document, Chapter 8, Structure, it is stated: “There should be approved procedures for regular review and updating of the structure, function and quality of PhD programmes”. It was agreed that this should include both structural and ad-hoc feedback. Internal evaluation must be an essential part of the whole quality assurance system and regular analysis of internal evaluations is the optimal way to achieve quality improvement in PhD training. The following recommendations were made.
- Academic institutions should activate mechanisms to collect, analyse and respond to evaluations, as a part of quality assurance of PhD training.
- Each university should develop a formal framework for internal (intramural) evaluation by collecting and acting upon feedback from all involved in PhD education (students, supervisors, examiners, administrative staff, and others).
- Internal evaluation should include evaluation of the institution (university/faculty/programme faculty) and evaluation of performance indicators. The evaluation has to be carried out in different contexts (PhD programme / administration / supervision / training). The evaluation criteria have to be transparent.
- The workshop strongly recommends the evaluation of the supervisors and other academics responsible for each student’s PhD programme.
- Evaluation of the administration by the supervisors as well as by the PhD students (doctoral candidates) should also be performed
- Evaluation of all aspects of the PhD training programme (courses, time at other laboratory, etc.) has to be included in quality assurance system.
- Evaluation has to be carried out repeatedly over time. The workshop recommends evaluations at the time of the student’s admission to the PhD programme, midway, and at the end of the programme.
Workshop 2: External evaluation of PhD programmes according to ORPHEUS standards
The workshop considered different ways in which the ORPHEUS PhD standards could be used for performing external evaluation of PhD programmes. There were several views.
- External evaluation is necessary and should be provided by an external body of auditors. It is, however, difficult to come up with a good practical means of implementing this. Neither the Commission nor the EUA can act as accrediting bodies. Standards could serve as guidelines, not regulations. There could initially be internal evaluation by the institutions themselves, supplemented by institutional accreditation.
- Peer-review evaluation. Evaluation of applications by funding agencies for PhD stipends could be made not only in terms of scientific excellence (publications, received funding, etc.), but also on in-depth peer-review evaluation of PhD programmes. This approach might be of interest in assessing Marie Curie applications.
- Benchmarking was seen as a practical means of external evaluation that could encourage compliance with the ORPHEUS standards at grass-roots level.
- No formal external evaluation needed. This was a view of industry which is concerned with outcomes. The outcomes of interest include academic excellence of the PhD thesis as well as “ability to work in a team”. Thus industry places great emphasis on collaboration with universities to ensure development of competences in transferable skills. Academically trained employees who do not have practical skills are unlikely to be attractive for industry. It was recognized that the institutions following ORPHEUS PhD standards would provide the outcomes that industry needs.
Representatives for all views underlined that the standards that are established by ORPHEUS can form a basis for quality assurance. However, they should be dynamic and the organisation should always aim for further quality improvements of the PhD programmes. University autonomy must be respected. Any evaluation of PhD programmes should be performed by academics with experience of PhD training. PhD programmes should be transparent. There should also be transparency concerning the employability of the institution’s PhD graduates. Students are likely to form a driver to ensure the quality of PhD programmes.
These views were supplemented by comparison with the US system of accreditation of PhD training. This is university-driven and there is reluctance to link evaluation of PhD programmes with funding. The workshop representative from the US, Karen de Pauw, recommended that data should be collected to determine the extent to which graduate schools complied with ORPHEUS standards.
Workshop 3: International evaluation and ORPHEUS labelling of PhD programmes
International evaluation is usually part of the evaluation of research. Accordingly international evaluation of PhD programmes is especially important for smaller scientific communities to ensure independent and competent evaluation of PhD programmes (and PhD Theses). Even for large and competitive scientific communities, international evaluation can be significant added value. Besides it can be expected that international evaluation will facilitate international collaboration of PhD programme providers.
ORPHEUS-AMSE-WFME standards, have been developed over 7 years by ORPHEUS, with participation of hundreds of health science schools, international associations such as AMSE and WFME, and positive assessment of the European Commission EU , and EUA-CDE. This PhD standards document is the only existing comprehensive document on PhD in biomedicine and health sciences and is suitable for use in international evaluation.
Participants of the workshop welcomed the idea of ORPHEUS labelling as a reward to those ORPHEUS members organizing PhD programmes in line with the PhD standards document. ORPHEUS labelling should be organised in a simple and not expensive way. Evaluation of compliance of PhD programmes with the PhD standards document should be the basis for evaluation and labelling. PhD programmes should be described on the website in English in sufficient detail to allow “virtual evaluation”of compliance of PhD programmes with the PhD standards document as the main requirements for ORPHEUS labelling. Participants had different views about the necessity of site visits. Most participants would only recommend site visits if they can be organised as short and not expensive procedures. Details of evaluation and labelling should be developed by the ORPHEUS Executive Committee.
Workshop 4: The role of students in PhD training evaluation
Participants were grouped according to nationality and student status ensuring that as many aspects as possible were discussed. Main focus of the discussions was student involvement in the topics PhD programme organisation, supervision and instruments for quality improvement.
PhD programme organisation. The following conclusions were drawn:
- Active and formal involvement in the PhD programme organisation: The PhD students should be represented in all committees and decision-making organs. Student involvement should be hierarchically constructed with different levels of influence, ensuring that all students have a student representative for reporting issues. Student representatives bridge the students and the PhD organisation, and facilitate the communication between students and organisations. Student representatives identify and communicate challenges and ideas to the programme organisation, and communicate the outcomes to the students. To promote active involvement of students, their efforts must be acknowledged, the infrastructure provided and their influence reflected in decision-making.
- Passive involvement: Mandatory evaluation of all courses taken, own project progress, supervision and the PhD programme should be completed every term (semester) by all PhD students. No approval of the term should be possible or accreditation of courses given before this is done.
- Supervisors should be evaluated both by the PhD programme and the PhD student. The standards for supervision are stated by ORPHEUS in Chapter 5. These standards should form a basis for objective, non-personal criteria for use by students in anonymous evaluation of supervision. With basis in evaluation by the programme and the students, quality of supervision should be monitored continuously with regard to both individual supervisors and the general level of supervision in the programme. The results should be communicated back to the students, and should be publically available. It was noted that several of the PhD students had no opportunity to evaluate supervision and supervisors. This aspect is not part of the ORPHEUS standards.
- At the beginning of the PhD, students and supervisors shall make a detailed expected progress plan. The responsibility for supervision is shared between students asking for supervision and providing information on progress, and supervisors for monitoring progress and offering supervision where they see the necessity in following up the plan. The student is responsible for informing the faculty in case of insufficient supervision.
- PhD students shall have the opportunity for formal networking in settings outside the research programme. Association with a broad network increases student involvement and sense of belonging.
Conference Book
Orpheus Med Group 1 Main St, Brussels, Belgium
Monday-Friday: 8am - 5pm (CET)