The pandemic we are currently experiencing has made it necessary to introduce changes to adapt medical education to this ’new normal’ in which we are immersed, and will continue to do so in the near future. In a previous editorial, we discussed how medical schools should respond to the new challenges posed by this situation, especially with regard to the contents to be taught, how to teach them and how to evaluate them. This last aspect, evaluation, is undoubtedly one of the main challenges that we will have to face without further ado [¹].

In the evaluation of students we have suddenly been forced to perform tests online since it was no longer possible to do them face-to-face. However, examining students at a distance, mostly by means of multiple choice tests, entails a number of technical difficulties because it is necessary to set up mechanisms that allow all students to access them under equal and secure conditions. It also generates ethical conflicts because there is a fear that students may resort to fraudulent procedures while taking them, thus reducing the reliability, discrimination and fairness of the tests. It is true that there are systems in place to prevent cheating, such as limiting the amount of time available, verifying the identity of examinees, visually tracking them or ensuring that they do not resort to information available in different formats or from a variety of sources. Yet, it is also true that lecturers cannot become strict controllers, as this would convey the feeling that we do not trust our students. Therefore, we must think of other alternatives that complement those we already have and are capable of perhaps not completely solving but at least reducing the negative impact we were referring to.

In this regard, in some countries, the current situation has led many medical schools to progressively implement what are known as ’open-book examinations', as opposed to the usual ’closed-book examinations'. An open-book examination is defined as a test in which students are allowed to use study materials, the internet, textbooks, their own notes, and so on [²]. This kind of exam can be taken face-to-face and also online. In each case, it should be made quite clear what kind of materials can and cannot be used, that is, the rules of the game, as well as the time available to complete the test.

In contrast to closed-book exams, which usually assess students' ability to understand and remember information and their capacity to memorise information, in open-book exams the type of questions used should assess their ability to apply knowledge to particular situations and to solve problems, as well as to evaluate cognitive and critical reasoning skills and their capacity to deal with uncertainty. Yet, they should also evaluate the ability to search for quality information by selecting the most appropriate sources. In addition, if the questions ask the student to justify their answer, it will be more difficult to find the solution in books and this will make it possible to evaluate the way students think. These exams are usually based on posing questions about case studies or clinical cases. Their characteristics make them tests that are appropriate for evaluating clinical practice [³].

However, in our context, as we are used to multiple choice exams, without the possibility of using reference material, open-book exams are an option that may come as a surprise. But we could argue that the use of this type of exam is justified by the fact that, when faced with a given case in their daily practice, professionals can usually consult different sources of information, and in fact do so quite frequently. So why can’t our students do the same when they are set a problem to solvé It is also true that professionals sometimes have to solve problems urgently and without time to look up information, and for this reason knowledge is an essential competence of medical professionals that must be evaluated. Open-book examinations are not intended to completely replace traditional closed-book exams, which primarily explore knowledge, but should complement them.

In any case, open-book examinations also offer a number of difficulties. First of all, they are not easy to prepare. The practical or hypothetical case studies must be designed with care. Appropriate, usually open-ended questions should also be asked in order to evaluate the competencies mentioned above, avoiding questions that evaluate knowledge in an abstract way. Furthermore, this type of exam is usually a novelty for students who, on taking one for the first time, are surprised to find that it is not the type of exam they are used to. Therefore, they may fall into the error of thinking that, being allowed to consult various sources, they will not need to study so much. In fact, several studies have shown that this does occur, but many of these studies have been conducted with students who are not used to this type of examination, and since they initially think that they will be easier, they spend less time studying [⁴]. However, this kind of exams are not necessarily easier than the traditional ones and very often they can actually turn out to be more difficult.

In recent years, we have had some experience with this type of testing with second-year medical students, and the initiative was well received. But we have detected that students, in order to feel as secure as they can, try to look for specific and exhaustive information to solve each and every one of the questions, so as to ensure they have the correct answer, when perhaps they could answer some of them perfectly well with only the knowledge that they have inside their heads. This exhaustive search for information for each of the questions in the exam involves a great investment of time that sometimes makes it difficult for them to finish the test within the stipulated time. They should be warned of this difficulty and that they should calculate the time available very well. A good solution, as proposed by some authors, would be to carry out some mock, or instructional, evaluations beforehand so that students can get used to the new type of evaluation [⁴].

Given the difficulties arising from the current situation, it might be advisable to carefully assess the possibility of introducing this type of evaluation in order to determine its effectiveness, its acceptance by lecturers and students, and the performance achieved, comparing it with the results obtained in traditional closed-book exams. Likewise, according to some authors, it would be very appropriate to combine the two types of evaluation [⁴].

Evaluation is a great challenge in medical education, and more so in today's circumstances, which represent a good reason to reflect upon how to improve it and move towards a more comprehensive evaluation of our students. Open-book examinations can be another tool with which to achieve this. But in any case, whatever kind of exams are used, we can never be 100% sure that students will not resort to some kind of fraudulent procedure. We must therefore insist on educating students in the culture of integrity and honesty that has to govern all their behaviour both as a student and as a professional, in accordance with the values of medical professionalism.