ORPHEUS 2011: PhD Quality Indicators, Izmir

Izmir Consensus Document

The aim of ORPHEUS since its inception in 2004 has been to work towards improving the quality of PhD programmes and their outcomes. In particular, a position paper was prepared in 2009 which synthesized consensus papers of previous conferences: “Towards standards for PhD education in biomedicine and health sciences”. An essential basis for such standards is identification of the quality indicators by which PhD programmes should be assessed. For this reason, at the 6th ORPHEUS conference held in Izmir, 27-30 April 2011, 311 participants from 102 universities from 40 countries worked on quality indicators of Research Environment, PhD Thesis, PhD Supervision, combined PhD-clinical specialisation programmes and Student Perspectives. The quality indicators shown below were agreed to be of particular importance. Other important outcomes of the ORPHEUS 2011, Izmir Conference were the reports on PhD Education in Nursing and PhD Education in Physiotherapy and Rehabilitation which resulted in descriptions of PhD education in these areas.  Here it was reemphasised that PhD education is research-based, and that the standard for a PhD is the same for all biomedical and health science specialities. ORPHEUS aims to conduct similar work on other areas of PhD education at future conferences. These reports together with full reports from the other conference workshops will be placed on the ORPHEUS homepage (www.orpheus-med.org).


Note, in this document the term “PhD student” is used synonymously with “doctoral candidate”.

Research environment

The research environment is the sine qua non regarding the quality of a PhD programme. The research environment can be evaluated at various levels, but the following two seem to be of primary importance: the level of the organization, and the level of the supervisor’s group.

Quality indicators for the research environment of the organization include:

  • Quantitative: number of scientific staff, number of supervisors and PhD students, number of technical staff, number of PhD degrees awarded, publications (PubMed, SCI, etc) and their impact factor.
  • Descriptive: laboratories, computer facilities, libraries and online journal access, financial resources, relationship with industry, technology transfer office, focus on scientific ethics, clinical and laboratory practices, student support facilities, editorial and statistical assistance, collective activities, and level of internationalization.

Quality indicators for the research environment of the supervisor’s group include:

  • Quantitative: dedicated time for supervision, publication record, publications with national or international co-authors, number of PhD students and PhD theses, size of the group, number of international and national guest members
  • Descriptive: research activities of group and techniques used, financial support, description of international and national networks and infrastructures (e.g. ESFRI, NENS, EMTRAIN), industry relationship, journal clubs, weekly group meetings, “career development” measures, social activities of the group. 

PhD programme
 The keywords for a successful PhD programme are ‘transparent organisation’, where students understand what to do, and ‘follow-up culture’, where they receive clear feedback on their progress. Around this concept, the following quality indicators stand out:

  • The graduate school should have website providing full details about the programmes offered, including details of admission policy, submission criteria for the PhD thesis, and assessment of the thesis. A version of the website in English should be available.
  • Information about average duration of funding, average duration of enrolment and drop-out rate
  • A brochure for newly enrolled PhD-students about the programme and the standards for the PhD-thesis
  • PhD programme activities developed by an educational council and communicated to both students and their supervisors
  • An influence, via the educational council, of PhD students and supervisors on what is offered by the PhD programme
  • A programme in which course quality is adequately monitored with a transparent methodology
  • PhD programme that supports doctoral student’s research in which published papers and international congress contributions are also registered as part of the programme
  • Feedback to students and teachers on the outcome of the quality surveys on teaching in the programme
  • Clear quality standards for faculty staff to become a supervisor
  • Annual follow-up by the PhD programme with clear criteria of study progress, with involvement of the supervisor and with clear feedback to the students
  • An independent Ombuds-person that can help resolve conflict
  • Guarantee of a truly independent jury to evaluate the PhD thesis
  • Involvement in career development and follow-up of employment will contribute to the sustainability and the civil effect of the PhD programme
  • Finally, a good PhD programme should not only have clear rules and regulations, but also the flexibility to make evidence-based exceptions for helping PhD students in exceptional situations. These can be study interruptions due to medical or familial reasons or finding extra funding to finalize the studies, or other situations.


PhD supervision
Effective supervision and a two-way process between supervisor and student are important factors in a successful PhD programme. The PhD supervision can be described at various levels, where the following quality indicators seem to be of primary importance:

  • Clearly defined qualifications for faculty staff to be supervisor and the extent to which supervisors are formally authorized according to local rules or regulations.
  • The formal requirement(s) for becoming a primary or main PhD supervisorinclude formal qualifications (PhD or equivalent) plus expertise in the specific field, affiliation to an eligible institution, research funding.
  • The responsibilities of a supervisor, as indicated by the extent to which the supervisor gives advice on the project, monitoring and documenting of the progress, taking care of ethics and proper scientific conduct, support for further career development of the PhD candidate.
  • The relationship between the supervisor and the PhD candidate as indicated by regular meetings, availability for consulting, provision of constructive feedback, respect for academic autonomy of the PhD candidate, and promotion of personal development.
  • The efficiency, effectiveness and sustainability of the supervision process as indicated by the number of PhD students enrolled, and number of theses finalized successfully within the assigned period. The number of papers (and impact factor) published by supervised PhD students
  • Availability of formal courses for supervisors and the extent to which supervisors take these courses.
  • The use of a “thesis committee” as advisors to the PhD student and the supervisors regarding the development of the PhD project.



PhD thesis


The PhD thesis is the major outcome and an important indicator of the quality of the PhD program. The quality of thesis can be assessed on the basis of the following characteristics:

  • Contribution to formation of new and significant knowledge or to a new, original and significant application of existing knowledge, with usage of appropriate research methodology
  • The extent to which the thesis demonstrates that the PhD student has acquired the skills to carry out independent, original and scientifically significant research, consistent with a 3-year research project, and to critically evaluate work done by others
  • Compliance with internationally accepted ethical standards (Helsinki, Oviedo and other important international conventions) and approval by appropriate ethical committees
  • Compliance with standards of “responsible conduct of research” and acknowledgement of conflicts of interest
  • The extent to which the candidate has demonstrated ability to critically evaluate and solve difficult and previously unsolved questions in different areas of human activity
  • The extent to which the thesis contains publication of original findings in internationally recognized journals (e.g. PubMed, SCI), the impact factor of these, and whether the student is first author of the publications. (The ORPHEUS benchmark for the PhD thesis in biomedicine and health sciences is the equivalent of at least three in extenso peer-reviewedpapers published, or accepted for publication, in internationally recognized journals. Normally the student would be the first author on these).
  • Transparency of the intellectual and practical contribution of the PhD candidate
  • The extent to which evaluation is made by competent persons through an independent peer-review process, together with an open public defence.


Combined PhD-clinical specialisation programmes
The aim of a combined programme is to train medical doctors to become professional clinical specialists with a PhD. This allows clinicians to maintain critical academic thinking and have better tools to understand the underlying causes of diseases, their pathogenesis, treatment, clinical trials, epidemiological and public health aspects.


It is fundamental that both the clinical training and the PhD research meet the same quality standards as the separate programs. The training obviously takes a longer time than individual PhD work or clinical specialization alone. Nevertheless, time can be saved because some components (theoretical training, courses etc.) can simultaneously be parts of both competencies. High quality of the combined training needs to consider the following components:

  • The ability of the student to have time-sharing between the clinic and research without compromising either component
  • Maintenance of academic freedom also in the clinical training environment
  • Ability for the research work to be in an area related to the clinical specialization and for the theoretical training to include issues relevant for clinical research.
  • The possibility to arrange training into flexible modules that meet appropriate quality standards, tailored according to the needs of the trainee.
  • Sufficient support to the student to be able to carry out duties related to the program and to maintain scientific independence and academic freedom. This involves making an annual progress report and having a follow-up committee that convenes at least once a year. Furthermore, the student should interact with other PhD students and get support from peers.


Student perspectives
The opinions and experiences of the PhD students in the PhD programmes represent a vital resource for the development of PhD education. Students attending the conference considered that the following quality indicators were essential:

  • Transparency in all aspects of the programmes, including admission and evaluation procedures
  • The provision of information to students about the PhD programme, and clarity concerning the standards for the PhD thesis
  • The extent to which students are able to play a role in, or initiate, evaluations of PhD programmes
  • The extent to which students play an active role in the organization of the graduate school
  • The extent to which activities taken at other institutions are given credit.
  • The extent to which there is appropriate feedback from evaluations to students, supervisors and the graduate school, and the extent to which this is acted on
  • The extent to which students are able to make an active contribution to the development of the programme and to quality assessment
Marlies Stouthard,Amsterdam
Maria  Georgieva,Plaven
Abdul Haque,Faisalabad
Michal Mikolaj,Praha
Pavol Prazenica,Praha
Kistine Kirkegaard,
Tatjana  Simic,
Vladimir Bumbasirevic,Belgrade
Maria  SimeoNOIMva,Plaven
John  Kelly,Galway
Konstantin Gurevich,Moscow
Hasan Efe,
Ingrida Uloziene,Kaunas
Jurate Janusaitiene,Kaunas
Sina Henrichs,Basel
Sezgin Gunes,Kurupelit
Immanuel Taivans,Riga
Gunta Strazda,Riga
Miroslav Cervinka,Hradec Kralove
Sultan  Gulce Iz,
Hannes Stockinger,Vienna
Christa Janko,Vienna
Helga Ögmundsdottir,Reykjavik
Ayse Hilal Bati,Izmir
Ljubica Glavas-Obrovac,Osijek
Tijen Temiz,Sivas
Zrinka Kovarik,Zagreb
Karmela Barisic,Zabreb
A.Cavidan Akören,Ankara
M.Kemal Kucukersan,Ankara
Jadranka Varljen,Rijeka
Jan Stasko,Martin
Kamil Javorka,Martin
Jan Hanacek,Martin
Robert Bjerknes,Bergen
İrem Hilavin,Izmir
Andrea Olschewski,Graz
Ljerka Ostojić,Mostar
Vesna Varunek,Mostar
Monija Ivankovic,Mostar
Samra Zahiragic,Graz
Günther Gell,Graz
Johan Van De Voorde,
Kirsten Wisborg,Aarhus C
Chris Van Schravendijk,Brussels
Sven Hendrix,Diepenbeek
Jörg-Wilhelm Oestmann,Berlin
Juraj Mokry,Martrin
Marianne Stien,Bergen 
Torunn Olsnes,Bergen 
Inger Hjeldnes  Senneseth,Bergen 
Jozsef Timar,Budapest
Birute Strukcinskiene,Klaipeda
Jelena Santric,Belgrade
Cane Tulic,Belgrade
Goran Curic,Osijek
Jakob Ousager,Odense
Hatice Efsun Kolatan,
Anil Özüdoğru,Izmir
Sergo Tabagari,Tbilisi
David Tvildiani,Tbilisi
Tinatin Gagua,Tbilisi
Gunnevi Sundelin,Umea
Ann Sörlin,Umea
Francisco Gracia-Navarro,Córdoba
Sevil Uzer,IZMIR
Pavel Jezek,Nymburk
Roghaiyeh Safari,Izmır
Ingeborg Van Der Ploeg,Stockholm
Aliye Mandiracioglu,
Francesco Saverio Ambesi-Impiombato,Udine
Sonja Levanat,Zagreb
Sinem Gülat,Izmir
Michael Mulvany,
Zuzana Cervinkova,Hradec Kralove
Tomáš Zima,
Reza Salimi,Izmir
Gül Güner Akdogan,İZMİR
Gurkan Ersoy,
Meral Karaman,
Oya Sayin,
Zübeyde Erbayraktar,IZMIR
Evin  Ozen,Izmir
Yasemin Saygideger,Buldan/ Denizli
Ayşegül Sacvı,
Seniz Inanc,Izmir
Mualla Aylin Arıcı,Izmir
Mine Doluca,İzmir
Aygül Kıssal,
Firat Soyarat,
Pınar Kemanli,İZMİR
Duygu Harmancı,İZMİR
Serap  Cilaker Micili,
Ferhan Girgin Sagin,BorNOIMva 
Derya Murat Özgün,Izmir
Oğuz  Dicle,
Nihal  Gördes  Aydoğdu,
Fatma  Ersin,
Aysim Gozukizil,Inciralti/Izmir
Dilek Göktürk,
Aavo Lang,Tartu
Nessar Ahmed,Manchester
Bora Uzun,
Burak Evren İnanan,
Melis  Dinç Kant,Izmir 
Jean Chambaz,PARIS
Petr Hach,PRAHA 2
Merve Akiş,
Burcu Akpınar,
Serhat Taslica,Izmir
Zahide  Cavdar,Balcova
Meric Senduran,
Guldal Kirkali,Inciralti-Izmir
Gökhan Karakülah,Izmir
Hakan Abacioglu,Balcova
Jadwiga  Mirecka,Krakow
Mostafa Ashrafi,
Murat Ozgoren,Izmir
Nil Hocaoglu Aksay,A
Feriha Özkaya,Inciraltı
Ebru Sezer,IZMIR
Özlem  Midik,
Ozan Akçay,izmir
N.Pinar Ay,Istanbul
Gülden Zehra Omurtag,
Pınar  Erçetin,İzmir
Reidar Thorstensen,Bergen
Fazilet Vardar Sukan,Izmir
Mustafa Sabuncu,
Ibrahim Tuglu,MANİSA
Mahmud Mustafa Özkut,manisa
Nida Demircak,Inciralti
Bojan Polic,Rijeka
Ismet Deliloglu Gurhan,
Murat Sipahi,İzmir
Mike Hardman,Macclesfield
Nilgün Yener,Izmir
Eren Sahin,
Attila Matkovits,Pécs
Réka Brubel,Pécs
Ferdi Başkurt,ISPARTA 
Zeliha Başkurt,Isparta
Tomris Ozben,Antalya
Cevval Ulman,Manisa
Roland Jonsson,Bregen
Pinar Yilgor Huri,Adana
Ebru Dundar Yenilmez,Adana
Peter Galajda,Martin
Seher  Özyürek,35340
Gürsel Sönmez,Bursa
Maksut KulzhaNOIMv,Almaty
Tanya MoNOIMva,Sofia
Züleyha Şimşek  Yaban,KOCAELİ
Yalçın Yetkin,VAN
Paola ZaNOIMvello,Padova
Dolores Prieto,
Figen  Şengün,Izmir
Erdi Sözen,İstanbul
Burak Yazgan,ISTANBUL
Olivera Stojceva-Taneva,Skopje
Neriman Elibol,Kocaeli
Zeynep Sercan,Izmir
Ismail Hakki Tekiner,ISTANBUL
Seyit Can Güloğlu,
Gizem İrem  Kinikli,
Emre Meral,
Senay Sarmasoglu,
Joey Barnett,Nashville
Ozlem  Kucukhuseyin,Istanbul
Anette Saltin,Lund
Zeynep Özdemir,Eskişehir
Jitka Ulrichova,
Aygul  Akyuz,Ankara
Aylin Aktaş,
Peter  Van Der Hijden,
Bora Akat,
Serkan Karaismailoglu,
Roda Seseogullari Dirihan,
Fusun Torun,Ankara
Hakan S. Orer,Ankara
Sezgin Ataç,İzmir
R. Selma Görgülü,Ankara
Serap Gökçe,Aydın
Péter Kiss,Pécs
Rory Morty,
Johanna Ackemar,Stockholm
Firouz Ghader…,Urmia 
Sukran Darcan,Izmir
Jürgen Deckert,
Marvin Jie Rong Lee,
Jerko  Barbic,Osijek
Baykal  Seda,
Firat  Soyarat,Izmir
Nicolas Goulding,London
Osman Yılmaz,
Erika Halašová,
Emna EL Golli-BenNOIMur,
Mostafa Ashrafi Osalu,
Constantin Drains,
Emmanuel Fragkoulis,Atheny
Tuğba Yardımcı,Izmir
Ilhan Yaylım Eraltan,Istanbul
Arzu Ergen,Istanbul
Ikbal Leblebicioglu,Talas
Emel Kırac,
Barış Gürpinar,Izmir
Pınar Kemanli,
N.Nuray  Ulusu,
Afshin Samadi,
Canan Saylam,Izmir
Gozde Gokce Isbir,İzmir
Samet Alboy,Edirne
Dilek Tiyekli,Edirne
Susan  Hamilton,Brisbane
Mohamed-Anis Bennour,Monastir
Maria-Eleni-Mavra Fragkouli,Athens
Kate Frøland,Bregen
Carmen R. Ahufinger,Córdoba
Pavo Barisic,Zabreb
Claudia Siener,Graz
Ingrid Gell,Graz
Luběna Hachová,PRAHA
Jenny Goulding,London
Izmir Conference Programme
Izmir Conference Book

Orpheus Med Group 1 Main St, Brussels, Belgium

Monday-Friday: 8am - 5pm (CET)