How can we distinguish good science? CEITEC applies a demanding evaluation method
23. Nov. 2018
Most of research groups operating within CEITEC have high international standard and carry out highly competitive research. This has been confirmed by the international evaluation organized by the research institute on a regular basis in four-year intervals. According to the statement of international experts, CEITEC significantly exceeds the usual standard in other European countries in the current setting of the science evaluation system.
The city of Brno has for the third time already hosted more than twenty experts from leading international research institutes who have arrived to evaluate the quality of their colleagues across all departments of the Central European Institute of Technology - CEITEC. An independent international evaluation performed by the top experts, so-called peer review, allows not only to ascertain the quality of the performance of individual research groups, but also to determine their future potential. The motivation of CEITEC for undergoing this demanding evaluation is based exclusively on its internal decision.
Jaroslav Koča, the scientific director of CEITEC, points out that: “The quality evaluation based solely on the statistical assessment of publication activities, i.e. bibliometrics, can be performed by a robot. You don’t need a human for this. I believe that CEITEC can serve as a good example of evaluating science, which is especially important at this time when we are transferring to the new national methodology of evaluating scientific outputs that will be decisive for the allocation of institutional support among academic institutions in the upcoming years.”
And what is done next with the evaluators’ recommendations? Each evaluated researcher receives a feedback and recommendations from the top experts in their respective field of specialization. Experience shows that those who have taken the evaluators’ recommendations to the heart have proven to be more successful in obtaining prestigious scientific grants and in the end improved their results in the next evaluation. On the other hand, a very negative evaluation may in extreme cases result in the dissolution of the given research group. Luckily, this rule will not be applied this year, because the international experts have emphasized in their evaluation report that CEITEC continues to develop as a top research institute in the Central Europe.
The recommendations of international evaluators are also valuable for the institute’s management where they will be further discussed and incorporated in specific measures to be adopted. As indicated by the executive director of CEITEC, Markus Dettenhofer, these recommendations will also be used in searching for future directions of our co-operation: “Based on the evaluators’ recommendations, we want to increase our support for interdisciplinary research. CEITEC has a potential to become a leader in cross-disciplinary areas, such as materials inspired by nature and nanotechnologies, technological innovations in biomedicine and sustainable agriculture and healthy nutrition.”
The International Scientific Advisory Board of CEITEC (ISAB) meets once a year and once every four years meets the evaluators for thorough scientific evaluation. The evaluators are acknowledged experts in the given field of specialization and are selected so as to ensure their absolute impartiality, diversity and equanimity. Some of the evaluators this year were Gustaaf Borghs of Leuven University in Belgium, a respected expert in the field of nanotechnologies, Lars Berglund of KTH Stockholm, who specializes in bio-composites, Wilhelm Gruissem of ETH Zürich, an expert in plant genomics, Witold Filipowicz of Friedrich Miescher Institute v Basel, an expert in molecular medicine, or structural biologists Michael Sattler of the Technical University in Munich and Eva Pebay-Peyroula of IBS, France. The continuity of evaluation is ensured by means of the fact that about one half of the evaluation panel members has been evaluating CEITEC repeatedly, while the other half consists of completely new individuals. These bring a fresh perspective to the evaluation and new ideas for improvements. At the same time, ISAB provides consulting and specialized recommendations for the CEITEC management.
The evaluators receive detailed materials approximately one month in advance, including bibliometric assessment. Thus, they have a detailed insight of the functioning, performance and future plans of the research group and are able to ask very specific questions during the personal meetings that last about an hour. During this time, each head of the research group has fifteen minutes to present their research and intentions. The meetings of evaluators and researchers take place in the form of a controlled discussion. The structure of the interview and evaluation methods are always the same and consistent. The evaluators also spend a quarter of an hour with PhD students without the presence of any other researchers or the research group leader. Thus, the evaluators are able to obtain the most accurate information ensuring that their evaluation is objective to the maximum extent possible.
"I would like to congratulate CEITEC and its employees to the professional organization of our meeting that meets all the criteria of the best international evaluation and that is being carried out in a highly equitable manner,” Dirk Inzé, the scientific director of the department of plant biology systems of VIB (the Flanders Institute for Biotechnology), who is also the chairman of the ISAB, commented on the entire course of the evaluation process.
Karel Říha, the head of the Plant Molecular Biology research group, adds another aspect: “Let’s not forget that this evaluation method creates unique personalopportunities for our researchers. Just imagine that you spend an hour discussing your work with often the best experts in the given field of specialization. This represents, in particular in the case of junior researchers, a valuable opportunity to establish new contacts and networks. When they meet in the future at any conference abroad, they will already be known to these experts. And last, but not the least, the evaluation also has a significant impact on the overall internal culture of our institution.“