The Ottawa Conference is an international congress that is held every two years in different countries around the world and whose main theme is assessment in medical education. Barcelona hosted it in 2004. The different Ottawa Conferences have always produced recommendations and consensus statements on how to develop increasingly more appropriate forms of assessment in medical education, at the undergraduate and postgraduate levels, as well as in specialised training.

Some of the documents that emerged from the Ottawa Conferences include: Criteria for Good Assessment (2011); Performance Assessment (2011); Assessment of Professionalism (2011 and 2019); Framework for good assessment (2018); International consensus statement of the assessment of interprofessional learning outcomes (2017), and Ottawa 2020 consensus statement for programmatic assessment (2021), which are highly recommended reading.

At the last Ottawa Conference in Lyon (France) in 2022, experts addressed various aspects of assessment in medical education and identified challenges to be addressed in the near future in order to improve assessment practices. The aim of this editorial is to summarise the main issues raised in this forum [¹].

The experts analysed the evidence not only on the use and effectiveness of assessment and its instruments, but also on the ways in which assessment is designed and implemented. They also discussed the impact assessment has on improving learning, learner engagement with assessment, the feedback given, the effect of new forms of assessment in terms of achieving better learning outcomes and the challenges posed by the use of new technologies in assessment practices. In reference to this last aspect, attention was drawn to what happened during the pandemic, when it became necessary to implement the Objective Structured Clinical Examinations (OSCE) remotely, and they sometimes failed to reach the required standards of quality, which raises doubts about their validity, especially when they are used to make decisions that are of the utmost importance for the student (high-stakes assessment).

The experts also pointed out that the OSCE remains a valid method of assessing clinical and communication skills, but its design and implementation on a large scale and with sufficient validity remains a major challenge. They stressed that the main problems are still the inadequate number of stations resulting in insufficient sampling and data to assess students' performance properly, the need to be very clear about the purpose and objective of the test, while establishing clear standards for its assessment, as well as ensuring a variety of examiners and appropriate training to reduce possible biases. This is clearly expressed by Boursicot, one of the experts participating in the conference of the Association for Medical Education in Europe in Lyon (2022), when she stated in her presentation that ‘an OSCE is not an OSCE simply because it is called an OSCE, but because they must meet clearly defined requirements'.

The conference also focused on the increasing role of workplace assessment, the use of other methodologies such as the electronic portfolio and the implementation of an integrated set of different assessments and feedback within an overall system of programmatic assessment. How to carry out the assessment of so-called entrustable professional activities was also addressed as one of the challenges to be overcome in the future.

When considering a particular type of assessment, teachers must be very clear about its purpose and students' participation in it longitudinally throughout their learning process. Likewise, they also have to take into account the judgements of expert observers as a key element in assessment, and move progressively from decision-making based on traditional numerical scores towards decision-making based on narrative feedback and the use of all the information provided by the different types of assessment employed, whether for formative or summative purposes, such as OSCE. This becomes even more crucial when decisions of great importance to the student are to be made (high stakes assessment).

If we look at what is going on in our setting, there is no doubt that the assessment of the competences of our medical students has improved enormously in recent times. The adoption of OSCE by all the faculties of medicine in our country has been a very important step forward in the assessment of our undergraduate students. However, if there is one field in medical education that is constantly developing and is the subject of educational research, it is precisely that of assessment. The content of this article that we have summarised demonstrates this. Therefore, we cannot be content with what we do, but must be able to constantly analyse in a critical manner how we do it in order to improve the effectiveness and validity of the assessment instruments we use, complementing them with other methodologies. What was discussed at the last Ottawa Conference is a good roadmap to guide us in this necessary analysis.