In a recent issue of the journal Medical Teacher [¹], some renowned experts in the processes used to assess health professionals published a commentary entitled 'The pursuit of fairness in assessment: looking beyond the objective'. In that article, they argue that the development of the competency-based medical education paradigm has now led to a shift away from traditional assessments towards workplace-based assessments and an emphasis on what are known as programmes of assessment.

Based on this, these experts reflect on how to ensure fairer and more equitable assessment processes in the new scenario, which poses the dilemma of whether fairness and equity are best achieved with objective tests or whether, on the contrary, assessment in which decisions are made on the basis of expert judgement, and subject to a certain degree of subjectivity, are more relevant.

The history of assessment in medical education has been characterised by the search for instruments that ensure fairness and equity, relying especially on objective tests. Clear examples are the use of multiple choice questions to assess knowledge and, later, objective structured clinical examinations (OSCE), which basically sought to minimise human judgement as much as possible in the interests of fairness. However, the authors point out that the use of objectivity as the gold standard for assessing trainees and residents also has its limitations.

Different experts argue that even in tests that we consider objective, there is always a certain degree of subjectivity in different aspects, such as the topics to be included, the choice of questions and the cases or stations to be developed, the minimum scores to be achieved to pass the assessments, etc. In addition, it is usually necessary to reach agreements among the evaluators, which is not always easy to achieve. Even in the OSCE, despite the use of checklists or rubrics, it is very difficult to completely eliminate subjectivity.

When it comes to assessment in the workplace, i.e. in a real-life situation, which is generally less structured, less predictable and, above all, more complex, objective assessment does not guarantee overall fairness and equity. This situation arises in the assessment of residents, where the expert's subjective judgement would become relevant, despite the possible existence of limitations or biases.

Given the limitations of both objective (multiple choice and OSCE) and subjective tests, experts recommend that, to ensure fairness, a suitable balance should be struck between objectivity and subjectivity.

In our context, objectivity is still seen as the only way to achieve fairness in assessment and no subjective component is allowed. This means that the main instruments used for assessment are multiple-choice tests. The appearance of the OSCE has dazzled the academic world as an objective, and therefore fair, instrument for assessing clinical competency. Nevertheless, in many cases, what is gained in fairness through objectivity is lost in validity and reliability because these objective instruments are not used in the right way. Multiple-choice tests often contain significant technical shortcomings owing to a lack of knowledge of the principles guiding the proper preparation of tests. As regards the OSCE, which are complex to develop and difficult to analyse, it is often observed that they fail to meet the psychometric guarantee required to ensure their total reliability when used for summative purposes.

To improve this situation it is necessary to promote the training of university lecturers and residents' tutors in the assessment process. This would bring to the attention of both teachers and tutors the wide range of assessment instruments available and the conditions for their appropriate use. It would also make them aware of the fact that fairness and justice in assessment is not only a matter of using objective tests, but that the use of expert judgement as a criterion in assessment is relevant. Applying expert judgement has its limitations and entails the danger of possible biases, but these can be lessened and offset by appropriate mentoring and effective feedback. Assessment is too important to be left to the teacher alone, so decision-making, which is currently carried out individually by the teacher/tutor, should become a concerted process, as advocated by the new paradigm of programmatic assessment.