Abstract

LAPRESA, D et al. The consensus agreement method in observational methodology: impact of sample size on time requirements and dataset quality. CPD [online]. 2021, vol.21, n.2, pp.47-58.  Epub May 16, 2022. ISSN 1989-5879.

The reliability of datasets in observational methodology is typically tested using coefficients of agreement, correlation coefficients, or generalizability theory. Another increasingly popular method used to demonstrate the quality of data is the consensus agreement method, in which two or more observers agree on their coding decisions while creating the dataset. Although the consensus agreement method is being increasingly used in observational studies, few studies have conducted an in-depth analysis of how this qualitative procedure is approached or of how it can be optimized. In this study, in addition to presenting a practical example of the application of the consensus agreement method, we compare the results from three groups (of two, three, and four observers) to analyze performance in terms of time required to code the data and goodness of fit with respect to an optimal dataset. No significant differences were found between the three groups for either of the variables analyzed. Prior calculation of the sample size required to detect significant differences between the groups adds strength to our conclusions regarding the efficiency of the consensus agreement method.

Keywords : observational methodology; consensus agreement; sample size; Cohen's kappa; time; ANOVA.

        · abstract in Spanish | Portuguese     · text in Spanish     · Spanish ( pdf )