REVISIONES

 

Instruments for primary health care evaluation: a narrative review

Instrumentos para la evaluación de la atención primaria de salud: una revisión narrativa

 

 

Vélez Álvarez, Consuelo^* and Giraldo Osorio, Alexandra^**

^* Department of Public Health, Research group of Health Promotion and Disease Prevention, University of Caldas. E-mail: consuelo.velez@ucaldas.edu.co
^** Research group of Health Promotion and Disease Prevention, Young researchers and innovators program, University of Caldas, Colciencias. Colombia.

 

 

---

ABSTRACT

Objective: To conduct a narrative review about articles that present instruments used for primary health care evaluation worldwide.
Materials and methods: The search was performed in PubMed, Embase, Cochrane, and Lilacs of articles published between 2008 and 2013. The descriptors used were: primary health care, health care evaluation mechanisms, questionnaires and validation study. The eligibility criteria for articles were three languages and validation studies. An analysis matrix was constructed by database with the characteristics of each item, then a final matrix was consolidated with the articles that met the inclusion criteria and the found instruments were analyzed. The data analysis was carried out by the two authors independently for greater objectivity.
Results: 128 articles that met the inclusion criteria for the study were obtained. The percentage of instruments identified by analysis categories were: Lifestyle and chronic diseases 20.3%, PHC -Management 5.4% PHC - Relationship between Health equipment and User 23.4 %, Mental health 28, 9% and other subject areas 21.9%.
Conclusions: There is a wide and varied availability of generic and specific instruments that have been validated in the global context for evaluating the primary health care strategy, from which it is possible to reconstruct the best evidence as a working tool.

Key words: primary health care; evaluation; validity of tests; questionnaires; validation studies; health evaluation; review.

---

RESUMEN

Objetivo: Realizar una revisión narrativa sobre artículos que presentan instrumentos utilizados para la evaluación de la atención primaria de salud a nivel mundial.
Materiales y métodos: La búsqueda se realizó en Pubmed, Embase, Cochrane y Lilacs de artículos publicados entre 2008 y 2013. Los descriptores utilizados fueron: primary health care, health care evaluation mechanisms, questionnaires and validation study. Como criterios de elegibilidad de los artículos se incluyeron tres idiomas y estudios de validación. Se construyó una matriz de análisis por base de datos con las características de cada artículo, luego se consolidó una matriz final con los artículos que cumplieron con los criterios de inclusión y se analizaron los instrumentos encontrados. El análisis de la información se llevó a cabo por las dos autoras de forma independiente, para una mayor objetividad.
Resultados: Se obtuvieron 128 artículos que cumplieron con los criterios de inclusión establecidos en el estudio. El porcentaje de instrumentos identificados por categorías de análisis fueron: Estilos de vida y enfermedades crónicas 20,3%, Estrategia de APS - Administración 5,4%, Estrategia de APS -Relación Equipo de salud Usuario 23,4%, Salud Mental 28,9% y otras áreas temáticas 21,9%.
Conclusiones: Existe una amplia y variada disponibilidad de instrumentos genéricos y específicos que han sido validados en el contexto mundial para evaluar la estrategia de atención primaria de salud, sobre los cuales es posible reconstruir la mejor evidencia como herramienta de trabajo.

Palabras clave: Atención primaria de salud; evaluación; validez de las pruebas; cuestionarios; estudios de validación; evaluación en salud; revisión.

---

 

Introduction

Reorienting health services after the renewal of primary health care (PHC) in the Americas, it has been suggested as the core of health systems ^(1-2). This has led to the inclusion of mechanisms to allow conceptual, organizational and service offerings appropriation in different countries of Latin America, as each must find their own way to create a sustainable strategy in order to firmly base their health system on PHC ⁽¹⁾. In this regard, one of the main challenges arising from the implementation of health systems based on PHC, is the evaluation of the results of actions in terms of efficiency and changes in the health status of the population, in order to ensure the quality of services ^(3-4). Thus, the assessment is established as an essential component of management and potential source of inputs for the improvement of processes in general and quality in particular. ^(3,5).

The literature has reported that the theoretical models on which PHC is evaluated regularly emphasize structural and organizational aspects ^(6-8) and scarcely the extent to which they perform their duties ⁽⁹⁾ and in this sense, the volume of studies published in scientific journals grows enormously, however, the scientific quality of research articles is varied. The importance of taking into account the quality of the studies for decision-making, is that acting on the basis of invalid or wrong scientific evidence is probably worse than acting without evidence. In the best of cases, the practice is unconsciously distorted and at worst of them the decision may cause harm to patients or negatively affect the users of the health system ^(10-11). In the area of biomedical sciences and health studies research studies with social impact have been conducted but few focused on PHC and although there are studies focused on bibliometric analysis an update of scientific production level base don PHC ⁽¹²⁾ is required.

Scientific evidence rebuilt in this review will allow decision-makers and researchers have a comprehensive view on the instruments that have been used in different contexts, thematic areas and priority lines for PHC and in this way to identify the information about tools for their easy location, consultation and selection. Thus, as discussed above perform a narrative review articles presenting instruments used for the evaluation of PHC worldwide.

 

Materials and methods

The narrative review articles are spacious publications appropriate to describe and discuss the development or the "state of the art" in a particular case, from a theoretical or contextual point of view ⁽¹³⁾. To this end, for the literature search of instruments that assess PHC four electronic databases were used: Pubmed, Embase, Cochrane and Lilacs; articles published between 2008 and 2013 were included. In addition to the databases cross-references and literature in specialized journals was sought, where some of the previously selected articles were found. The Mesh, Decs and Emtree search terms were: Related with PHC (eg. Primary health care) related with assessment (eg, health evaluation mechanisms and questionnaires) and related with the type of study (eg, validation studies). An analysis matrix was built where the characteristics of each of the items were recorded and variables such as author, title, original language, journal, year of publication, objective, name and evaluated area were included; as a final result of this process four matrices (one for each database) were obtained. As eligibility criteria there were used: validation articles for tools assessing PHC in Spanish, Portuguese and English and that the researches presented a tool to evaluate PHC.

Information Processing: data extraction from the review of studies was carried out with the participation of two researchers who worked independently, reviewed the studies and extracted the data required to fill the matrices defined for analysis; the type of study and the instrument used to evaluate PHC. Subsequently, the results of both were compared and in case of discrepancies, these were resolved by consensus. Of the four databases a second matrix was developed consisting on articles that met the inclusion criteria. The cut point to start the review was Articles published up to June 2013.

Analysis Plan: from the information provided in the final consolidation matrix, full-text articles were obtained, the variables included in this study were analyzed and a categorization was made according to the evaluation objective proposed by the tools identified (actors, programs and thematic areas that were targeted).

Ethical aspects: this review follows the ethical parameters established by Resolution 8430 of 1993 and the Declaration of Helsinki and does not represent any risk on humans because it is not performed on individuals but on published articles.

 

Results

Search of the different articles in databases PubMed, EMBASE, Cochrane and Lilacs was performed, for which different search strategies were used and are described in Table I.

 

 

In the specified search in each database the terms primary health care and health care evaluation and questionnaires were used, except in the Pubmed database, where instead of using the term health care evaluation it was used health care quality. As a result of this first search 4141 articles were obtained. Then the search was refined and validation studies were selected, so the results were restricted to 222 from which 128 met the inclusion criteria and were annexed to the matrix of the evaluators (Table II). Finally 78 articles were rejected for not meeting the inclusion criteria and 16 for being duplicated in different databases. Following the evaluation of the instruments found, a categorization was made considering the ones assesing directly PHC functions, the ones assesing the functions of PHC related to a subject area, and the ones evaluating specific thematic areas without being bound by any of the functions of PHC. After analysis the categories that emerged were: Lifestyle and chronic diseases (20.3%), PHC Strategy - Management (5.4%), PHC Strategy - health team and user relationship (23.4%), Mental health (28.9%) and other thematic areas (21.9%) In the latter category, instruments assessing thematic areas related to physical activity, the skeletal, muscle, digestive and nervous systems; occupational health, general health, sexual and reproductive health and health care of the elderly were included.

 

 

In Tables III to VII, the instruments found in each of the categories that emerged are presented. When analyzing the assessment ares notice how in the category lifestyles and chronic diseases the tendency is that to evaluate behavior (psychosocial and personal aspects) 14 of the 26 instruments were found (53%). Among the tools to evaluate the PHC strategy 37 were found of which 7 (19%) were found in the subject area of Administration and 30 (81%) to assess the health team-user relationship. Consistent with the development that has had PHC worldwide lots of tools to assess mental health have been designed (37) and in other subject areas 28 tools were found. Figure 1 shows the trend of the tools in each subject per year.

 

 

 

 

 

 

 

 

 

 

Discussion

In this review 128 articles that met the inclusion criteria were included; different types of validated instruments which are framed within the implementation of the PHC strategy were found. The categories obtained post-analysis were Lifestyle and chronic diseases, PHC Strategy - Management, Strategy PHC - Relationship health team and user, Mental Health and Other thematic areas. The results show a high heterogeneity in thematic areas that the tools assess related to PHC, and little focus on measuring the extent to which it meets its functions⁽¹⁴⁾. Additionally, they are focused on the evaluation of specific programs developed within the framework of the PHC strategy and pathologies attended, according to the results presented on a research conducted in countries with low per capita income, where most evaluations focused on specific programs such as maternal and child indicators and indicators bounded to structure and processes, but have lacked a clear conceptualization about the functions of PHC and a consistent measurement of them to assess their degree of achievement or attainment ^(15-16).

Results of other studies have shown a trend in research of PHC focused on nonspecific problems and in second and third place on psychosocial and clinical aspects ⁽¹²⁾. Within this variety of approaches of the strategy, 23.4% of the revised instruments refer to articles that evaluated it related to the health team and the user relationship, in this regard in Canada a research was conducted to identify the instruments evaluating the attributes of the PHC from the patient's perspective, against which it was concluded that it is necessary to adjust existing tools and / or develop new instruments ⁽¹⁷⁾. In the category other thematic areas, instruments to assess physical activity, the skeletal, muscle, digestive and nervous systems; occupational health, general health, sexual and reproductive health and health care of the elderly were included, which some studies have shown such activities or services provided within the framework of the APS ^(18-19).

The lower percentage category addressed in the instruments analyzed was the PHC Strategy - Management (5.4%), data that matches the issues raised by Cuba-Fuentes M, et al, who claim that the evaluation gives more importance to aspects such as time and service availability, accessibility, continuity of care, information they receive about their health problems or the doctor - patient relationship, while putting less emphasis on administrative aspects ⁽²⁰⁾. In this regard it is important to note that in this category the tools found are more recent which would suggest that it is so far that this aspect is being studied as as relevant to the operation of the strategy increasing the number of publications in this field. With regard to the scientific production of articles that demonstrate validated tools by years, the trend showed an increase between 2008 and 2011 of instruments in mental health, lifestyles and chronic diseases, consistent with the results of work carried out in Spain where they described an increase in PHC publications between 2008 and 2012 ⁽²¹⁾.

The findings indicate the need to move forward in the process of cultural adaptation of existing tools to assess different areas of PHC as one of the difficulties of the international use of these tools is the magnitude of the changes they suffer when they are adapted to different contexts without regard to recommendations on this type of processes ⁽²²⁾. In addition, from these results, it is evident that there are difficulties in the standardization of tools to assess PHC, this same difficulty has been reported in other assessments in the health sector ^(23-24).

 

Conclusions

There is a wide and varied availability of generic and specific tools that have been validated in the global context to evaluate the PHC strategy, on which it is possible to reconstruct the best evidence as a working tool. Despite the importance given to the strategy and the emphasis on the need to evaluate it, the tools used for this purpose have been mixed, which highlights the diversity of interests and researchers in the area. This heterogeneity in the categories that the instruments assess, poses a methodological challenge focused on cross-cultural adaptation to be performed prior to application of the instruments in contexts other than its origin and understanding of the functions or dimensions that PHC should fulfill. The trend in the production of instruments each year reflects the interest in the global context of initiating the evaluation of the strategy and its management, since it is in 2012 where the start of publication of tools relevant to this analysis category. The increase in the number of published evidence shows the need to carry out regular systematic reviews of the literature, which also assesses the quality of the tools based on their metric properties.

 

 

Received: April 20, 2014
Accepted July 11, 2014

 

References

1. Organización Panamericana de la Salud, Organización Mundial de la Salud. Renovación de la atención primaria de salud en la Américas: documento de posición de la Organización Panamericana de la Salud/Organización Mundial de la Salud (OPS/OMS) Washington, D.C.: Biblioteca Sede OPS; 2007.         [ Links ]

2. Autor; 2013.         [ Links ]

3. Pizzanelli M, Ponzo J, Buglioli M, Toledo A, Casinelli M, Gómez A. Validación de Primary Care Assessment Tool (PCAT) en Uruguay. Rev Med Urug. (Carta al Editor). 2011;27(3):187-2.         [ Links ]

4. Navarro López V, Martín-Zurro A, Violán Fors C. Proyecto de Investigación: La Atención Primaria de Salud en España y sus comunidades autónomas. Barcelona: semfyc ediciones; 2009.         [ Links ]

5. Macinko J, Almeida C, Klingelhoefer de Sá P. A rapid assessment methodology for the evaluation of primary care organization and performance in Brazil. Health Policy and Planning. (Search). 2007;22:167-10.         [ Links ]

6. Donabedian A. Aspects of Medical Care Administration: Specifying Requirements for Health Care. Cambridge: Harvard University Press; 1973.         [ Links ]

7. Sibthorpe B, Gardner K. A Conceptual Framework for Performance Assessment in Primary Health Care. Aust J Prim Health. 2007;13(2):96-7.         [ Links ]

8. Hogg W, Rowan M, Russell G, Geneau R, Muldoon L. Framework for primary care organizations: the importance of a structural domain. Int J Qual Health Care. 2008;20(5):308-5.         [ Links ]

9. Campbell S, Roland M, Buetow S. Defining quality of care. Soc Sci Med. 2000;51(11):1611-25.         [ Links ]

10. Ministerio de Sanidad y Consumo. Revisión externa y validación de instrumentos metodológicos para la lectura crítica y la síntesis de la evidencia científica. Madrid: Plan Nacional para el SNS del MSC. Servicio de Evaluación de Tecnologías Sanitarias del País Vasco (Osteba);2006. Report No.: OSTEBA N^o 2006/02.         [ Links ]

11. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. Cosmin checklist manual. Amsterdam: University Medical Center 2012.         [ Links ]

12. Carratalá-Munuera M, Orozco-Beltrán D, Gil-Guillen V, Navarro-Perez J, Quirce F, Merino J, et al. Análisis Bibliométrico de la producción Científica Internacional sobre atención primaria. Atención Primaria. 2012;44(11):651-8.         [ Links ]

13. Bernardo WM, Nobre MRC, Jatene FB. A prática clínica baseada em evidências: parte II-buscando as evidências em fontes de informação. Rev Assoc Med Bras. 2004;50(1):104-8.         [ Links ]

14. Berra S. El estudio de las funciones de la atención primaria de la salud: Adaptación e implementación de los cuestionarios PCAT en Argentina. 1era ed: CONICET; 2012.         [ Links ]

15. Macinko J, Starfield B, Erinosho T. The impact of primary healthcare on population health in low- and middle-income countries. J Ambul Care Manage. 2009;32(2):150-21.         [ Links ]

16. Haggerty JL, Lévesque J-F, Hogg W, Wong S. The strength of primary care systems. BMJ: British Medical Journal. 2013;346.         [ Links ]

17. Lévesque J-F, Haggerty J, Beninguissé G, Burge F, Gass D, Beaulieu M-D, et al. Mapping the coverage of attributes in validated instruments that evaluate primary healthcare from the patient perspective. BMC Family Practice. 2012;13(1):20.         [ Links ]

18. Autor; 2014.         [ Links ]

19. Feldman L, Vivas E, Lugli Z, Alviarez V, Pérez MG, S B. La satisfacción del paciente hospitalario: una propuesta de evaluación. Rev Calid Asist. 2007;22:133-40.         [ Links ]

20. Cuba-Fuentes M, Jurado A, Estrella E. Evaluación del cumplimiento de los atributos de la Atención Primaria y el grado de satisfacción de los usuarios en un establecimiento de primer nivel de atención. Rev Med Hered. [Investigación ]. 2011;22(1):4-5.         [ Links ]

21. López-Torres Hidalgo J, Basora Gallisà J, Orozco Beltrán D, Bellón Saameño JÁ. Mapa bibliométrico de la investigación realizada en atención primaria en España durante el periodo 2008-2012. Atención Primaria. (0).         [ Links ]

22. Berra S, Hauser L, Audisio Y, Mántaras J, Nicora V, Celestina de Oliveira MM, et al. Validez y fiabilidad de la versión argentina del PCAT-AE para evaluar la atención primaria de salud. Rev Panam Salud Publica. (Investigación Original). 2013;33(1):30-9.         [ Links ]

23. Atehortúa S, Ceballos M, Gaviria CF, Mejía A. Evaluación de la calidad metodológica de la literatura en evaluación económica en salud en Colombia: una revisión sistemática. Biomédica. 2013;33(4):615-30.         [ Links ]

24. Pane S, Solans M, Gaite L, Serra-Sutton V, Estrada M, Rajmil L. Instrumentos de calidad de vida relacionada con la salud en la edad pediátrica. Revisión sistemática de la literatura: actualización Agencia d'Avaluació de Tecnología i Recerca Mediques Barcelona. 2006.         [ Links ]