SciELO - Scientific Electronic Library Online

 
vol.17 issue49Epidemiological aspects of child mortality in a state in Northeastern BrazilBlog of use as continuing education strategy in nursing: an integrative review of literature author indexsubject indexarticles search
Home Pagealphabetic serial listing  

My SciELO

Services on Demand

Journal

Article

Indicators

Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google

Share


Enfermería Global

On-line version ISSN 1695-6141

Enferm. glob. vol.17 n.49 Murcia Jan. 2018  Epub Jan 01, 2018

http://dx.doi.org/10.6018/eglobal.17.1.276091 

Revisiones

Evaluation of randomized clinical essays developed by nurses according to the consort declaration criteria

Edith Guzmán-Mendoza1  , María de la Luz Santos-Díaz1  , Gisela Cabañas-Benitez1  , Dulce María Pérez-Calderón1  , Celsa Chávez-Aguilar1  , María Claudia Morales-Rodríguez2  , Erick Alberto Landeros-Olvera3 

1 Licenciada en Enfermería, Benemérita Universidad Autónoma de Puebla. México.

2Maestra en Ciencias de Enfermería, profesora investigadora, integrante del Cuerpo Académico Intervenciones de Enfermería, Benemérita Universidad Autónoma de Puebla. México.

3 Doctor en Ciencias de Enfermería, Integrante del Sistema Nacional de Investigadores, profesor investigador de tiempo completo, integrante del Cuerpo Académico Intervenciones de Enfermería, Benemérita Universidad Autónoma de Puebla. México.

ABSTRACT:

Objective

Evaluate if the Randomized Clinical Trials (RCTs) carried out by nurses in the last seven years fulfill the methodological rigor established by the CONSORT declaration criteria.

Methods

Nursing journals literature integral review; sixty-six RCTs carried out by nurses and published in 11 indexed journals within recognized data bases were analyzed and where their title, key words or design allowed them to be recognized as a randomized clinical essay. A 48 item instrument was realized to evaluate the characteristics of the 66 published RCTs. Such instrument is divides in two segments: the first one evaluates general information in the articles, and the second one includes the CONSORT Declaration characteristics.

Results

The 57.6% of RCTs used an equivalent control group, 87.9% used randomized sampling, 28.8% blind, 54.5% presented flow chart, groups.

Conclusions

In general, the RCTs published by nursing in the last seven years do not fulfill the CONSORT Declaration criteria. These findings present an opportunity area so nursing journals publishers request from the authors the most attachment to the methodological rigor in their articles, according to the CONSORT Declaration criteria.

Keywords: Evaluation; Clinical Trial; Nursing Research

INTRODUCTION

Randomized clinical trials (RCTs) are controlled experiments that regularly are used to evaluate treatment or interventions security and efficiency1. Well designed and correctly executed, RCTs provide the best evidence of the sanitary interventions effect2. However, the methodological rigor in executing a controlled experiment with parallel groups is not always performed. Diverse groups of researchers have interested on identifying which are the systematic procedures to carry on this experimentation and on obtaining reliable results and perform replicas in other contexts or populations3. Scientific studies quality evaluation can be considered as essential to the production process and the health scientific literature selection4. The most representative guide, Consolidated Norms for Clinical Essays Publications (CONSORT), was published in 1996 aiming to unify criteria to publish RCTs. CONSORT Declaration criteria have been constantly modified to polish design methodological details and procedures through a check list and a flow chart. The list has 25 specific items to communicate results or evaluate reports about RCTs of two or more parallel groups5)(6.

Regarding the published literature, there is no recent work evaluating if the RCTs carried out in certain disciplines fulfill the CONSORT Declaration criteria; it seems it is implied that the published essays have been evaluated by an editorial committee based on the CONSORT criteria. The evaluation performed to the essays published in the Revista Española de Anestesiología y Reanimación (Spaniard Journal on Anesthesiology and Re-animation) concluded on the lack of relevant data about the essays performance as well as a low methodological quality according to the actual criteria7; similar results report that the evaluation of RCTs published in Chile bio-medical journals have serious deficiencies but it is difficult to know if those inconsistencies are due to incomplete reports or to poor methodological designs.8

The presence of these methodological weaknesses is well known by the scientific journals editors, who point out that RCTs biases limit the available information and condition knowledge, affecting particularly to evidence-based medicine.9

In the case of the Nursing Science, RCTs represent one of the best scientific evidence for better caring procedures in communities, families, hospitals and independent practice as well.10)(11) Nursing publications with an experimental design, however, are minimum, so it is appropriate them would be evaluated with the rigor stated by the CONSORT declaration criteria.

There are no published works evaluating the RCTs developed by nursing; a study carried out the analysis of 358 nursing studies published in Mexican journals and identified that 14% of the published articles designs are experimental. The authors don’t specify the methodological characteristics of this 14% nor tell if they are RCTs, only point out methodological weaknesses in general like no randomized sampling, small samples, instruments which reliability is unknown and statistical tests wrongly employed.12

At the face of the evidence published actually, the following purpose may be stated: To evaluate if the RCTs performed by nurses in the last seven years fulfill the methodological rigor established by the CONSORT declaration criteria.

METHODOLOGY

This is an integral review of the nursing journals literature through the virtual library in the Meritorious Autonomous University of Puebla. The searching criteria were as follows: indexed journals (in databases SciElo, EBSCO, CINAHL, CUIDEN, LILACS, Medline, PubMed and Thomson Reuters), research articles published from 2008 first semester to 2014 second semester; in Spanish, English and Portuguese languages; articles which title, key words or design allow them to be recognized as a randomized clinical essay.

Eighty seven journals were analyzed in the determined period. Results rendered 9616 articles from which 66 researches were selected as they fulfill the inclusion criteria; they were located in 11 journals. To notice the selection criteria, go to Figure 1.

Figure 1 Selection process flow chart 

A 48 item instrument was carried out to evaluate the characteristics of the 66 RCTs published. Such instrument is divided in two segments: the first one evaluated the articles general information (six items) and the second includes the CONSORT Declaration characteristics. The instrument was pilot tested and the final review was performed within a 90 days period. Each articles was reviewed twice by different researchers to compare results and diminish biases. This procedure was done by the complete reading of the articles, focusing attention on the reagents the instrument evaluated.

Ethical considerations. This work protocol and procedures were evaluated for its registry in the Pre-Grade Investigation Coordination of the Meritorious Autonomous University of Puebla, Mexico under the reference number A-2015-0111-CIP. Statistical analysis. The results analysis was performed with descriptive statistics (frequencies and percentages) through de SPSS program, version 21.

RESULTS

Sixty six nursing RCTs published within the 2008 to 2014 in indexed journals in Spanish and English were analyzed. Those analyzed RCTs came from Spain (6), Colombia (3), México (3), Brazil (2), Turkey (2), USA (30), South Korea (2), Australia (2), Iran (6), China (2), Canada (1), Portugal (1), Netherlands (1) and some that don’t specify their origin (5).

The descriptive findings were: 34 RCTs (51.5%) present a nursing sciences PhD as first author; an MD for 13 RCTs (19.7%), one publication for each a specialist and a graduate (1.5% respectively), while 17 RCTs (25.7%) mention nothing about it.

Places where RCTs were performed show that 38 (57.6%) come from universities, 22 (33.3%) from health institutions, while the remaining 6 (9%) are divided from research clinics and institutes. Regarding collaboration, 25 RCTs (37.9%) are multidisciplinary, and the rest are performed exclusively by nursing professionals.

The main theme prevalent was clinic in 46 RCTs (69.7%), education in 17 RCTs (25.8%), and the family theme among other themes in 3 RCTs (4.5%).

The findings regarding the CONSORT Declaration criteria are shown in the following table.

Table 1 Criteria results: CONSORT Declaration. Puebla, México, 2014. 

IMRyD Format= Introduction, Methodology, Results, and Discussion

PS*= Parametric statistics; NPS†=Non Parametric statistics; PS/NPS‡= Parametric statistics/non parametric statistics; MVS§= Multivariate statistics; DS¶= Descriptive statistics.

Table II Main characteristics of the Randomized Clinical Essays analyzed. Puebla, México, 2014 

DISCUSSION

General characteristics of the study

The present study aimed to evaluate if the RCTs performed by nurses in the last seven years fulfill the methodological rigor established by the CONSORT Declaration criteria.

Authorship. Half of the RCTs analyzed have as first author a doctor in sciences, which is congruent due the upbringing of this academic degree, which main objective is facilitate the transition from knowledge to practice through variables manipulation. 13

Themes. The prevalent theme was clinic; is seems logic to think that this type of design has more feasibility in hospitals than in community or family interventions. This finding is similar to that reported by García Rodríguez12 where the most used variables were clinical type, indicating that the evaluations included in RCTs have a clear interest and repercussion on the hospital environment patients.14) However, a larger presence is required specially in indexed journals publication and with a good impact factor, since RCTs publications by nurses are scarce, and the figure of the nurse in clinical essays is not clearly described and remains as an unknown entity for most of the professionals working in a hospital.11

Researchers. Most of the RCTs in this study do not include other disciplines different from nursing; the figures coincide with the findings of García Rodríguez, where the minority of articles show no multi-disciplinary collaboration, but the situation is similar in other disciplines.7)(8) In this sense we believe that to afford the study problems in RCTs, researchers necessarily must pertain to diverse disciplines that might contribute to clarify the research phenomenon.

Title and Abstract. Most of the evaluated works didn’t facilitate the search since in the title and abstract they could be identified as a randomized clinical essay as is set out by the CONSORT Declaration. However, they could be included because in the methodological section were mentioned as RCTs. If you wish to make a systematic review or meta-analysis of a particular theme that had been approached as a randomized clinical essay, the simple situation of not mention this in the title could be an exclusion motive, even though the RCE is well structured.

Background. One of the guidelines evaluated in any research work is the reference to previous studies and in our evaluation, most of the articles do so; it is clear that the articles that don’t show background in their research lack the power to make neutral conjectures when comparing their parallel groups and thus in these articles the results don’t give the certainty that the interventions or treatments are reliable; situations that can be notice not only in nursing discipline but is well known in the published reports in other types of journals.9

Pilot test.Just a minority RCTs report applying a pilot test, which is backed by the García Rodríguez and Abeille Mora’s report where most of the experimental studies don’t express a previous training or omit it,12)(15 however, this result must be considered with reserve because is possible that the pilot test was not necessary due the instruments have already been validated with previous studies or the researchers are experts and they simply didn’t mention it.

Financing. In the case of financing (equipment or material), most RCTs didn’t specify their financing source.7 This situation allows for subjectivity, possibly they are researches which are financed by the researchers themselves; anyhow, the budget line is not clear.

Methodological aspects.

Participants selection. Most of the analyzed articles present selection criteria, diminishing confounding variables. However, in a bit more than the half of the sample, the elimination criteria are not mentioned, coinciding with the scores performed by Gonzalez Barahona16 who mentions that not detail exclusion criteria represents a bias that influences the results external validity.

Equivalence and randomization. More than half of the analyzed RCTs show the basal- demographic characteristics and these results accord with Manriquez’ report8 about 66 RCTs where 80.3% clearly informed on the demographic characteristics; this fact facilitates locating if the groups are equivalent, apparently showing a correct randomization. In our evaluation, however, most of the RCTs don’t mention the method used to generate the randomized designation sequence and they don’t guarantee knowing if the groups are equivalent. This results are similar to García Alamino’s report7 where he mentions that the exposition to treatment or interventions maybe were not comparable.

Interventions. Most of the analyzed RCTs detail the interventions and specify they have a response variable after the intervention; this allows to know which were the changes from the statistic point of view were obtained. Other similar studies, however, found that in the analyzed studies the process and the analysis were not detailed.7)(8

Sample size. It has been evident now that most works don’t clarify the method to calculate the sample size8) such is the case of our study; this situation diminishes the statistical power and may lead to a wrong conclusion that there are no differences between parallel groups and highlighting the type I or alpha error. Since a small sample size gives reliable answers to the stated questions or the study hypothesis that need to be proven. On the other hand a large sample size may make the study hard to handle, waste much time and effort and is essential for producing useful research results.17

The principal variable measurements number. Most RCTs didn’t carry out an intermediate measurement of the variables, since most of them were designed under a test re-test model. It would be important to have a comparison between each one of these variables in order to observe the phenomenon evolution, have more control over the variable being manipulated and watch for changes as they evolve; but this depends on several facts, both the study design and the financing to carry out repeated measurements model.

Blind. In experiments, the blind or masked interventions are important to avoid biases and make more accurate comparisons. In our review most of the RCTs don’t mention blind, and as a consequence it is possible that comparisons between parallel groups are, deliberately or unwittingly, impartial inducing a sample contamination that involves a bias in the final analysis; this agrees with Manriquez and Flores Pineda,14 who found that the way the studies participants were masked is not clearly specified.

Flow chart. The study progression was not schematically showed in more than half of the researchers. As is mentioned in the CONSORT Declaration, show it allows to observe the way in which the sample wears through the subjects loss in order to keep the statistical power, or if there was intention of treatment, or to understand the results analysis. The reviewed literature does not make evident the evaluation of this parameter from the CONSORT in the analysis of RCTs in other studies.

Ethics considerations. More than half of the RCTs evaluated don’t mention having used a written informed consent. Getting a signed paper, however, is not a guarantee that the participant has understand the research objectives;19 this situation could give the RCE more credibility if the protocol had been registered and evaluated by an ethics committee, since this committee’s responsibility is to assure the rights, security, and welfare of the possible participants in the essay;18 most of the evaluated works, however, don’t mention such a registry and this situation must be required by journals specially in this kind of designs.

Limitations. Most of the evaluated essays don’t mention the limitations occurred during their development; without these data, carrying out improved replicas of protocols is not allowed, since some factors acting as impediment and reduce the clinical applicability of the study might be allowed to be established.19

CONCLUSION

In a general form, the RCTs published by nursing in the last seven years don’t fulfill the CONSORT Declaration criteria. The results might be considered as methodological limitations in design and statistics that contribute nothing to present a reliable evidence.

This must be considered just now that the nursing work with experimental designs is recent. If the RCTs are performed with the pertinent statistical reliability and with quality in their procedures, they might represent a clear scientific evidence in the future of nursing interventions.

This findings represent an opportunity area for nursing journals editors to require from the authors more adherence to the methodological rigor in their articles according to the CONSORT declaration criteria.

REFERENCIAS

1. Manterola C, Otzen T. Estudios Experimentales 1ª Parte. El Ensayo Clínico. Int. J. Morphol. 2015; 33(1):342-349. [ Links ]

2. Cobos Carbò A, Augustovski F. Declaración CONSORT 2010: actualización de la comprobación para informar ensayos clínicos aleatorizados de grupos paralelo. Med Clin. 2011: 137(5): 213-214. [ Links ]

3. Pérez Fuillerat N. Enfermería Basada en Evidencia y Transferencia de Conocimiento. Index de Enfermería. 2015; 24(1-2): 5-6. [ Links ]

4. Cascaes da Silva, Valdivia BA, da Rosa R, Barbosa Gutierres PJ, da Silva R. Escalas y listas de evaluación de la calidad de estudios científicos. Revista Cubana de Información en Ciencias de la Salud [revista en internet]. 2013 [citado 2016 agosto 31];24(3):295-312. [ Links ]

5. CONSORT.[Internet]: Ottawa: The Ottawa Hospital Research Institute; c2014 [consultado 05 mayo de 2014]. Disponible en http://www.consort-statement.org/. [ Links ]

6. González de Dios J, González Muñoz M, Alonso Arroyo A, Aleixandie Benavent R. Comunicación Científica (XIX) conocimientos básicos para leer (y escribir) un artículo científico (6): Listas de comprobación de estudios experimentales. Acta Pediatr Esp. 2015; 73(1):19-24. [ Links ]

7. García Alamino JM, Parera A, Olle G, Bonfil X. Características y calidad metodológica de los ensayos clínicos publicados en la Revista Española de Anestesiología y Reanimación. Rev. Esp. Anestisiol. Reanim. 2007; 54:333-339. [ Links ]

8. Manríquez J, Valdivia G, Rada G, Letelier LM. Análisis crítico de ensayos clínicos randomizados publicados revistas biomédicas chilenas. Rev. Méd. Chile. 2005; 133:439-446. [ Links ]

9. Alfonso F, Segovia J, Heras M, Bermejo J. Publicación en ensayos clínicos en revistas científicas: consideraciones editoriales. Rev. Esp. Cardiol. 2006; 59(11): 1206-1214. [ Links ]

10. Cárdenas Becerril L. Desarrollo del Pensamiento Reflexivo y Crítico. Ciudad de México: Cigome, 2014. [ Links ]

11. Guerrero Molina A, Millán Vázquez G, Cruzado Álvarez C, Medina Fernández M. La enfermera de ensayos clínicos en el ámbito hospitalario: una figura desconocida. Rev. Cardiocore. 2013; 48(2):75-78. [ Links ]

12. Garcia Rodríguez MG, Gómez Ávila MG, Aguilar Pérez I, Pérez Martínez GP, Velázquez Díaz L, Soriano Sotomayor M, Landeros Olvera E. Tendencias y características de la investigación en enfermería publicada en México. Rev. Enf Univ ENEO-UNAM. 2011; 8(1): 7-16. [ Links ]

13. Scaular Judith M. LA IMPORTANCIA DEL DOCTORADO EN CIENCIAS DE LA ENFERMERIA. Rev haban cienc méd [Internet]. 2008 Sep [citado 2016 Sep 05] ; 7(3):. Disponible en: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S1729-519X2008000300001&lng=es. [ Links ]

14. Flores M, Márquez V, Candido A, Rodríguez J, Soriano M, Landeros E. Diseños Experimentales: ¿La Ciencia del Cuidado está Preparada? Rev. Horiz Enferm. 2015; 26(1):21-28. [ Links ]

15. Abeille Mora E, Soto Carrasco AA, Carrera Huerta S, Muñoz Muñoz VP, Sánchez Salinas R, Pérez Noriega E, Landeros Olvera E. Características de la Prueba Piloto: Revisión de Artículos Publicados en Enfermería. Rev. Enf Neurol. 2015; 14(3). [ Links ]

16. González J, Fernández A B, Lucio F J (2013). Análisis crítico de ensayos clínicos de preparaciones de medicina herbal china. Dianas 2(1): e20130304. ISSN 1886-8746 dianas.20130304 URI [ Links ]

17. Shapiro H. Aspectos éticos de los ensayos clínicos en los países en desarrollo. Rev. Hosp. Materno Infantil. 2002; 21(1): 43-44. [ Links ]

18. Karlberg J, Speers M. Revisión de ensayos clínicos: una guía para el Comité de ética. Hong Kong: board, 2010 marzo. ISBN 978-988-19041-1-9. [ Links ]

19. Donado Gómez JH, Higuita Duque LN, Castro Palacio JJ. Limitaciones más frecuentes de los ensayos clínicos con asignación aleatoria (ECA) en el área de medicina interna. Rev. Med. 2015; 23(2):35-40. [ Links ]

Received: November 24, 2016; Accepted: February 23, 2017

Creative Commons License Este es un artículo publicado en acceso abierto bajo una licencia Creative Commons