The aim of ORPHEUS since its inception in 2004 has been to work towards improving the quality of PhD programmes and their outcomes. In particular, a position paper was prepared in 2009 which synthesized consensus papers of previous conferences: “Towards standards for PhD education in biomedicine and health sciences”. An essential basis for such standards is identification of the quality indicators by which PhD programmes should be assessed. For this reason, at the 6^th ORPHEUS conference held in Izmir, 27-30 April 2011, 311 participants from 102 universities from 40 countries worked on quality indicators of Research Environment, PhD Thesis, PhD Supervision, combined PhD-clinical specialisation programmes and Student Perspectives. The quality indicators shown below were agreed to be of particular importance. Other important outcomes of the ORPHEUS 2011, Izmir Conference were the reports on PhD Education in Nursing and PhD Education in Physiotherapy and Rehabilitation which resulted in descriptions of PhD education in these areas.  Here it was reemphasised that PhD education is research-based, and that the standard for a PhD is the same for all biomedical and health science specialities. ORPHEUS aims to conduct similar work on other areas of PhD education at future conferences. These reports together with full reports from the other conference workshops will be placed on the ORPHEUS homepage (www.orpheus-med.org).

Note, in this document the term “PhD student” is used synonymously with “doctoral candidate”.

Research environment

The research environment is the sine qua non regarding the quality of a PhD programme. The research environment can be evaluated at various levels, but the following two seem to be of primary importance: the level of the organization, and the level of the supervisor’s group.

Quality indicators for the research environment of the organization include:

• Quantitative: number of scientific staff, number of supervisors and PhD students, number of technical staff, number of PhD degrees awarded, publications (PubMed, SCI, etc) and their impact factor.
• Descriptive: laboratories, computer facilities, libraries and online journal access, financial resources, relationship with industry, technology transfer office, focus on scientific ethics, clinical and laboratory practices, student support facilities, editorial and statistical assistance, collective activities, and level of internationalization.

Quality indicators for the research environment of the supervisor’s group include:

• Quantitative: dedicated time for supervision, publication record, publications with national or international co-authors, number of PhD students and PhD theses, size of the group, number of international and national guest members
• Descriptive: research activities of group and techniques used, financial support, description of international and national networks and infrastructures (e.g. ESFRI, NENS, EMTRAIN), industry relationship, journal clubs, weekly group meetings, “career development” measures, social activities of the group. 

PhD programme
 The keywords for a successful PhD programme are ‘transparent organisation’, where students understand what to do, and ‘follow-up culture’, where they receive clear feedback on their progress. Around this concept, the following quality indicators stand out:

• The graduate school should have website providing full details about the programmes offered, including details of admission policy, submission criteria for the PhD thesis, and assessment of the thesis. A version of the website in English should be available.
• Information about average duration of funding, average duration of enrolment and drop-out rate
• A brochure for newly enrolled PhD-students about the programme and the standards for the PhD-thesis
• PhD programme activities developed by an educational council and communicated to both students and their supervisors
• An influence, via the educational council, of PhD students and supervisors on what is offered by the PhD programme
• A programme in which course quality is adequately monitored with a transparent methodology
• PhD programme that supports doctoral student’s research in which published papers and international congress contributions are also registered as part of the programme
• Feedback to students and teachers on the outcome of the quality surveys on teaching in the programme
• Clear quality standards for faculty staff to become a supervisor
• Annual follow-up by the PhD programme with clear criteria of study progress, with involvement of the supervisor and with clear feedback to the students
• An independent Ombuds-person that can help resolve conflict
• Guarantee of a truly independent jury to evaluate the PhD thesis
• Involvement in career development and follow-up of employment will contribute to the sustainability and the civil effect of the PhD programme
• Finally, a good PhD programme should not only have clear rules and regulations, but also the flexibility to make evidence-based exceptions for helping PhD students in exceptional situations. These can be study interruptions due to medical or familial reasons or finding extra funding to finalize the studies, or other situations.

PhD supervision
Effective supervision and a two-way process between supervisor and student are important factors in a successful PhD programme. The PhD supervision can be described at various levels, where the following quality indicators seem to be of primary importance:

• Clearly defined qualifications for faculty staff to be supervisor and the extent to which supervisors are formally authorized according to local rules or regulations.
• The formal requirement(s) for becoming a primary or main PhD supervisorinclude formal qualifications (PhD or equivalent) plus expertise in the specific field, affiliation to an eligible institution, research funding.
• The responsibilities of a supervisor, as indicated by the extent to which the supervisor gives advice on the project, monitoring and documenting of the progress, taking care of ethics and proper scientific conduct, support for further career development of the PhD candidate.
• The relationship between the supervisor and the PhD candidate as indicated by regular meetings, availability for consulting, provision of constructive feedback, respect for academic autonomy of the PhD candidate, and promotion of personal development.
• The efficiency, effectiveness and sustainability of the supervision process as indicated by the number of PhD students enrolled, and number of theses finalized successfully within the assigned period. The number of papers (and impact factor) published by supervised PhD students
• Availability of formal courses for supervisors and the extent to which supervisors take these courses.
• The use of a “thesis committee” as advisors to the PhD student and the supervisors regarding the development of the PhD project.

PhD thesis

The PhD thesis is the major outcome and an important indicator of the quality of the PhD program. The quality of thesis can be assessed on the basis of the following characteristics:

• Contribution to formation of new and significant knowledge or to a new, original and significant application of existing knowledge, with usage of appropriate research methodology
• The extent to which the thesis demonstrates that the PhD student has acquired the skills to carry out independent, original and scientifically significant research, consistent with a 3-year research project, and to critically evaluate work done by others
• Compliance with internationally accepted ethical standards (Helsinki, Oviedo and other important international conventions) and approval by appropriate ethical committees
• Compliance with standards of “responsible conduct of research” and acknowledgement of conflicts of interest
• The extent to which the candidate has demonstrated ability to critically evaluate and solve difficult and previously unsolved questions in different areas of human activity
• The extent to which the thesis contains publication of original findings in internationally recognized journals (e.g. PubMed, SCI), the impact factor of these, and whether the student is first author of the publications. (The ORPHEUS benchmark for the PhD thesis in biomedicine and health sciences is the equivalent of at least three in extenso peer-reviewedpapers published, or accepted for publication, in internationally recognized journals. Normally the student would be the first author on these).
• Transparency of the intellectual and practical contribution of the PhD candidate
• The extent to which evaluation is made by competent persons through an independent peer-review process, together with an open public defence.

Combined PhD-clinical specialisation programmes
The aim of a combined programme is to train medical doctors to become professional clinical specialists with a PhD. This allows clinicians to maintain critical academic thinking and have better tools to understand the underlying causes of diseases, their pathogenesis, treatment, clinical trials, epidemiological and public health aspects.

It is fundamental that both the clinical training and the PhD research meet the same quality standards as the separate programs. The training obviously takes a longer time than individual PhD work or clinical specialization alone. Nevertheless, time can be saved because some components (theoretical training, courses etc.) can simultaneously be parts of both competencies. High quality of the combined training needs to consider the following components:

• The ability of the student to have time-sharing between the clinic and research without compromising either component
• Maintenance of academic freedom also in the clinical training environment
• Ability for the research work to be in an area related to the clinical specialization and for the theoretical training to include issues relevant for clinical research.
• The possibility to arrange training into flexible modules that meet appropriate quality standards, tailored according to the needs of the trainee.
• Sufficient support to the student to be able to carry out duties related to the program and to maintain scientific independence and academic freedom. This involves making an annual progress report and having a follow-up committee that convenes at least once a year. Furthermore, the student should interact with other PhD students and get support from peers.

Student perspectives
The opinions and experiences of the PhD students in the PhD programmes represent a vital resource for the development of PhD education. Students attending the conference considered that the following quality indicators were essential:

• Transparency in all aspects of the programmes, including admission and evaluation procedures
• The provision of information to students about the PhD programme, and clarity concerning the standards for the PhD thesis
• The extent to which students are able to play a role in, or initiate, evaluations of PhD programmes
• The extent to which students play an active role in the organization of the graduate school
• The extent to which activities taken at other institutions are given credit.
• The extent to which there is appropriate feedback from evaluations to students, supervisors and the graduate school, and the extent to which this is acted on
• The extent to which students are able to make an active contribution to the development of the programme and to quality assessment