Mi SciELO
Servicios Personalizados
Revista
Articulo
Indicadores
- Citado por SciELO
- Accesos
Links relacionados
- Citado por Google
- Similares en SciELO
- Similares en Google
Compartir
Pharmacy Practice (Granada)
versión On-line ISSN 1886-3655versión impresa ISSN 1885-642X
Pharmacy Pract (Granada) vol.6 no.3 Redondela jul./sep. 2008
Original Research |
Validation of a survey tool for use in cross-cultural studies
Filipa A. COSTA, Catherine DUGGAN, Ian BATES.
ABSTRACT There is a need for tools to measure the information patients need in order for healthcare professionals in general, and particularly pharmacists, to communicate effectively and play an active part in the way patients manage their medicines. Previous research has developed and validated constructs to measure patients desires for information and their perceptions of how useful their medicines are. It is important to develop these tools for use in different settings and countries so that best practice is shared and is based on the best available evidence. Key words: Cross-Cultural Comparison. Validation Studies as Topic. Portugal. | RESUMEN Se necesitan herramientas para medir la información que necesitan los pacientes para que los profesionales de la salud en general, y los farmacéuticos en particular, comuniquen efectivamente y jueguen un papel activo en como los pacientes manejan su medicación. Anteriores estudios han desarrollado instrumentos para medir los deseos de los pacientes de información y sus percepciones sobre la utilidad de los medicamentos. Es importante desarrollar estas herramientas en diferentes escenarios y países para compartir la mejor práctica y que se base en la evidencia disponible. Palabras clave: Comparación trans-cultural. Estudios de validación como tema. Portugal. |
Filipa Alves da COSTA. MSc, PhD. Auxiliary Professor, Instituto Superior de Ciências da Saúde, ISCSEM. Caparica (Portugal).
Catherine DUGGAN, BPharm(Hons), PhD, MRPharmS. Senior Clinical Lecturer - School of Pharmacy, University of London, and Associate Director for Clinical Pharmacy, Development and Evaluation for East and South East England Specialist Services (NHS). London (United Kingdom).
Ian BATES. BPharm, MSc, FRSH, MRPharmS. Professor and Head of Educational Development. School of Pharmacy, University of London. London (United Kingdom).
Received (first version): 17-Mar-2008
Accepted: 20-Jun-2008
INTRODUCTION
The development of standardised scales to assess the information each patient wants can be of major importance to effectively target health care interventions. A survey tool has been developed to explore patients desires for information and their perceptions about prescribed therapy and illness. The initial tool, comprising 5 constructs, has been used in several studies and has been shown to describe and predict patients information needs and perceptions of their medicines. Further validation of three of these constructs has led to factors describing the Intrinsic Desire for Information (IDI), the Perceived Utility of Medicines (PUM), and the Anxiety about Illness (AI).2 Additional work on the initial 12-item IDI scale resulted in a 6-item scale: Extent of Information Desired (EID).1,3
An important attribute when developing a practical scale is its length; this scale uses the minimum number of items comprising three important dimensions: the way the individual perceives his illness; the way he perceives his medicines; and takes into account the importance of patient-health care professional communication. Considering the potential utility of such tools for health-care professionals to increase their awareness of patients desires, perceptions and feelings, the applicability of such a scale in other countries and settings was evaluated.
There is a problem when directly applying scales developed in English speaking countries for use in non-English speaking countries (eg: English scales for use in Portugal) and will increase as more countries enter the European Union. Networks to foster collaborative work are established across Europe to share and compare data.4 It is therefore important to adapt scales in a robust way for use across cultures that can assist health care professionals to target interventions to improve patient care.
The aim of this study was to validate a survey tool to measure patients information desires and perceptions about medicines and illness in a Portuguese patient sample.
METHODS
Ethics
Ethical approval had previously been obtained for the UK data. In Portugal, no official ethics approval systems were in place applicable to this study design and content, so a written authorisation was obtained from the head of each service or hospital, accordingly.
Study design
An initial exploratory study using a cross-sectional design evaluated two subsequent versions of the translated tool in regards to validity and internal consistency. The tools consistency was tested over time through a test-retest longitudinal study.
Sample
Previous research showed that 5 factors emerged from the pool of 50 items. The sample size was calculated to be 250 recruits for validation purposes and another 200 for purposes of test-retest reliability analysis. The latter sample would detect differences between mean scores to the scales for a 95% confidence interval (5% error and an 80% power). This paper describes part of a larger study but all data collected by the time of submission is presented.
A pilot study was undertaken in GEM patients to ensure that data collection materials and methods were appropriate for the main study. The main study involved testing the final version in an independent medical patient sample. The inclusion criteria restricted participants to adult patients prescribed with medication, understanding written and/or spoken Portuguese and agreeing to participate in the study once informed. Patients were invited to participate and were then provided with additional information about the research by means of a written invitation. Prior to the interview, patients read and signed an informed consent form.
Participants recruited in the hospital setting were interviewed at their bedside using a standardised questionnaire, comprising three main sections: demographic data, diagnoses and prescribed medicines; the survey tools items, which were statements to be answered using a 5-point Likert scale, ranging from strongly agree to strongly disagree; and a set of open questions to address the concepts explored through the scales. In the community pharmacy setting, questionnaires were self-administered while similar in content. Patients who were unwilling to participate were given a refusal form, seeking to explore different characteristics of non-respondents.
Data analysis
Data were analysed using descriptive statistics to characterise the sample and responses to the scales. The Kolmogorov-Smirnov Test was used to explore the distribution of scores to scales before deciding on the use of parametric or non-parametric tests. Construct validity was explored in factor analysis, using Principal Components Analysis (PCA) with Oblimin rotation. The emerging subscales were subsequently compared with retrospective UK data. Criterion validity was explored by correlating scores to the anxiety scale and patients responses to the SF-36 general question included in the questionnaire using Spearmans rho. The tools discriminatory power was assessed using ANOVA or independent samples t-test, as appropriate. Internal consistency was analysed using Cronbachs coefficient alpha and inter-item correlations (r) and again compared with the UK retrospective data. Test-retest reliability was explored in a sub-sample of patients who responded to the questionnaire at two time points one month apart. Data were treated as paired samples, using a t-test to compare their mean scores.
RESULTS
A sample of 62 patients was recruited for the pilot phase. This highlighted difficulties in recruitment associated with some of the study sites where access was restricted to acute care wards. Therefore, inclusion criteria were reviewed to widen the recruitment possibilities.
Table 1 summarises the demographic, medical and therapeutic characteristics of the 596 cases included in the sample. Gender was evenly distributed; the mean age was as expected given the focus on chronically ill patients. There was a high proportion of people without education and an extremely high proportion of individuals with primary school education. Cardiovascular and endocrine patients (57%) were the most common diagnoses. The number of non-respondents was very low (n=25; 4%), hence no statistical tests were performed to evaluate differences between the two samples.
Validity of the scales
Criterion validity was judged to be good. A negative and statistically significant correlation was found when comparing the AI scale with the external measure (rho=-0.318; p<0.001), indicating that patients who perceived their health status as worse were more anxious.
Construct validity: Principal components analysis resulted in 6 factors, rather than the predicted 5 (table 2). Factor 6, when compared to the original tool, was judged to be the combination of those items that did not work in a Portuguese sample, despite modifications during translation. This factor comprised the items A4 (I just want to blame someone for the way I feel) and T1 (I find my medicines easy to take, I am used to them).
Most items fell into the expected domains for the AI and EID scales. The Tolerance scale (Ti), a subscale of the Anxiety about Illness scale originally had weak internal consistency (alpha=0.470) and contained only 3 items; one of which did not load into this factor and was then dropped, leaving this scale with only two items, insufficient to form a scale. As such, the Ti scale was dropped and considered unsuitable for the Portuguese population.
The items of the PUM scale loaded well into the expected sub-factors. The exception was the previously described item T1 and was dropped. Both the PHM and PBM subscales comprised only 3 items, rather than the UK sample 4. The version to be used in Portugal had therefore the tolerance scale removed as well as one item of the PUM scale. The final scale comprised a total of 16 items.
Reliability of the scales
Both the EID and PHM scales showed acceptable internal consistency (alpha=0.607 and 0.536, respectively), which could be further improved by additional modifications. The AI and the PBM scales revealed good internal consistency estimates (alpha=0.695 and 0.756, respectively). In all scales, the removal of any item would result in a decrease of its overall internal consistency. Table 3 displays the Cronbachs alpha values and the inter-item correlations range for each scale and item in the Portuguese sample and their comparison with retrospective UK data.
All scales, except the EID were consistent over time, since their mean scores did not differ between baseline and after one month. The correlation between the PBM, PHM, and AI scores (at baseline and after one month) was high and statistically significant, indicating the concepts are being measured consistently. Furthermore, their mean scores were not significantly different, confirming these scales are consistent over time. Scores to the EID scale were strongly correlated between the two time points, but mean scores were significantly higher at baseline (25.9) than after one month (24.2) which implies patients seem to want less information as time goes on- maybe they are more used to their medicines by this stage
Discriminatory power
The scores to the four valid scales were explored in samples of inpatients, outpatients and community pharmacy patients. Significant differences between mean scores to scales were detected for the EID scale between patients in the three settings (F=9.891; p<0.001) and the AI scale (F=4.916; p=0.008). This implies patients recruited from different study sites had different desires for information and levels of anxiety about their illness.
To ensure that the setting had no further influence on the scales discriminatory power, analysis was restricted to community pharmacy patients. Females scored significantly higher than males to the PBM scale (t=-2.174; p=0.031). Age was negatively correlated with scores to the EID scale (r=-0.24; p<0.001) and positively correlated with scores to the AI scale (r=0.16; p=0.001), both correlations weak but statistically significant. Categorizing age into two broad groups additionally showed that younger patients expressed a higher desire for information (t=4.19; p<0.001) and lower anxiety about their illness (t=-2.00; p=0.047).
The mean scores to the EID, PHM and the AI scales varied significantly between the lowest educational level and all others (EID: F=3.92; p<0.001); (PHM: F=2.10; p=0.011); (AI: F=5.18; p<0.001). The desire for information increased, the perception of harm and anxiety decreased as years of formal education increased. Patients prescribed 5 or more drugs scored significantly higher on the Anxiety scale (t=-3.22; p=0.001). It may be assumed that when patients are prescribed more drugs, they interpret this as a signal of deterioration in their condition, hence increasing their anxiety.
In summary, the translated survey tool was found to be both valid and reliable after being tested in a patient sample of 596 patients with a wide representation of demographic and medical characteristics. The sample reflected the general characteristics of the Portuguese population well.
The five scales described in the original UK research behaved differently in the Portuguese population following translation. The tolerance scale was dropped during the course of validation work as it was deemed to be unsuitable for use in this patient sample. Additionally, the two subscales in the PUM scale were transformed by dropping one common item as it did not load into any of the expected factors and internal consistency of both scales increased when it was deleted. Both the EID and the AI scales were left unaltered and found to have good construct validity.
The EID scale showed to have an acceptable internal consistency (alpha=0.607) but not being consistent over time (p<0.05; p<0.01). The EID scale was able to discriminate between settings, age group and educational levels (p<0.05).
The PBM scale showed to have a good internal consistency (alpha=0.695) and being consistent over time (p>0.05; p<0.01). Construct validity results led to dropping one of the items, whilst not compromising its reliability. This scale showed to discriminate between different genders (p<0.05). The distribution of scores to the PBM scale showed small spread, indicating a possibility for little added-value in its use.
The PHM scale was the one with the lowest internal consistency (alpha=0.536), considered fair, but being consistent over time (p>0.05; p<0.01). Construct validity results led to dropping one of the items, whilst not compromising its reliability. This scale showed to discriminate between educational levels (p<0.05).
The AI scale was the one with the highest internal consistency (alpha=0.756), considered good, and also showing consistency over time (p>0.05; p<0.01). This scale showed good results in regards to criterion validity, when compared with the SF-36 (p<0.05). Additionally it showed to discriminate between settings, age group, educational levels and number of prescribed drugs (p<0.05).
DISCUSSION
The characteristics of the sample were compared to national data, where illiterate people represent 8% of the population (compared to 7% in the study sample) and people with university education account for 11% (compared to 12% in this sample). Study recruits with primary school education accounted for 47% of the sample, compared to 35% in the national population. Nonetheless, it may be concluded that the sample educational level represents the Portuguese population fairly well.5
Internal consistency of the EID was not as high as expected from previous UK studies.1-3 A Cronbachs alpha value of 0.7 has been suggested as good6 and the fact that the value reported for the EID in the Portuguese sample roughly passes 0.6 is an issue of concern. However, this scale has been further tested in another study including 112 hypertensive patients recruited from Lisbon pharmacies, where the value obtained was 0.683.7 This increase may be related to distinct characteristics of the samples, the latter study was conducted in a particularly developed area, where educational levels are generally higher. This may imply that this scales performance may improve as the Portuguese populations literacy increases. This trend is supported by successive legislation reviewing the minimum compulsory education.8
Consistency over time for the EID scale was not detected which may be due to the changes made just by asking the patients; and possible interventions made by the pharmacists while administering the questionnaire, simply by being aware of patients desires. Alternatively, information needs may be easily changed dependent on situations, time, severity of symptoms, whereas anxiety and perceptions of medicines may be less so.
Face and content validity have also been examined, albeit reported elsewhere9, where problems were revealed for the EID, PHM and Ti scales. The latter was dropped; the others may need additional work when being tested in practice. Additional problems were encountered with the Anxiety scale as it led to anguish in some patients, which could potentially compromise its use in pharmacy practice. Nonetheless, it may be worth exploring its applicability in medical settings, where scales such as the Becks Depression Inventory (BDI) have been employed.10
Criterion validity is often not reported due to lack of comparable measures in this area (which further validates the need to develop them). In this study, criterion validity was explored in three different ways, using qualitative data, interfacing qualitative and quantitative data and external measures to correlate with some of the scores to scales, although only the latter was expanded in this manuscript. Other external measures could have been used for comparison with the PUM scale, such as the BMQ.11 This would have undisputed advantages, but also an additional burden for respondents which could potentially have an impact on the response rate, so was disregarded in this study. Additionally, this questionnaire is not validated to Portuguese. There is no tool comparable to the EID that could have been used, leaving patients own words as the only possible option.
Some of the limitations of this study include not having used an external measure for each scale to explore criterion validity, limiting quantitative analysis of this type of validity to the Anxiety scale. The limited length of hospital stay resulted in the impossibility of having repeated the questionnaire also in hospital patients to explore consistency over time, which would have the advantage of ensuring the same interviewer, minimising probabilities of interviewer-related bias.
Selection bias in community pharmacy occurred since participating pharmacists had to combine participating in a research study with minimal impact on normal service delivery. The inability to guarantee response of refusal forms limited the usefulness of the analysis undertaken to explore comparability of respondent and non-respondents, but this was a limitation impossible to overcome as people cannot be forced to answer.
CONCLUSIONS
The main conclusions are that the translated tool was valid and reliable in Portuguese patients- excluding the Tolerance scale. Some of the scales may benefit from further refinement, such as the PHM subscale. Successive modifications are needed to increase the tools validity and reliability and confirm that scales are not directly transferable from one country to another. This study demonstrates the importance of robust translation and validation processes to measure the concepts of interest in a credible manner.
ACKNOWLEDGEMENTS
The authors acknowledge all the patients interviewed, the hospital staff, and all the community pharmacists who actively participated in patient recruitment.
CONFLICT OF INTEREST
The authors declare they have no conflict of interests. This project was entirely funded by CEFAR (Centre for Pharmacoepidemiological Research), ANF (National Association of Pharmacies), Portugal.
References |
1. Aström K, Carlsson J, Bates I, Webb DG, Duggan C, Sanghani P, McRobbie D. Desire for information about drugs. A multi-method study in general medical inpatients. Pharm World Sci. 2000;22(4):159-164. [ Links ]
2. Duggan C, Bates I. Development and evaluation of a survey tool to explore patients' perceptions of their prescribed drugs and need for drug information. Int J Pharm Pract 2000; 8(1): 42-52. [ Links ]
3. Duggan C, Bates I, Sturman S, Andersson E, Astrom K, Carlsson J. Validation of a 'desire for information' scale. Int J Pharm Pract. 2002;10(1):31-37. [ Links ]
4. PCNE. Pharmaceutical Care Network Europe. Available at www.pcne.org assessed 22/05/2006 [ Links ]
5. Instituto Nacional de Estatística. 2003, "Recenseamento Geral da População e Habitação (2001) - Resultados Definitivos." Lisboa. Data File ("General Licensing of the Population and Housing - Final Results") (original report in Portuguese). [ Links ]
6. DeVellis RF. Factor analysis. Scale development, theory and applications. London: SAGE Publications, 2003: 102-137. [ Links ]
7. Costa F, Couto S, Ferreira AP, et al. 2005a, "Hypertensive patients' perceptions about prescribed medicines. Proceedings of the FIP Congress (Abstract 108; CPS-P-083). Presented at the FIP Congress 2005 [ Links ]
8. Ministério da Educação. Portaria. Diário da República Portuguesa. 27-5-2005. n º 41/2005 ("Education Ministry. Law ") (original in Portuguese). [ Links ]
9. Costa FA, Duggan C, Bates I. A systematic approach to cross-cultural adaptation of survey tools. Pharm Pract. 2007;5(3):115-124. [ Links ]
10. Coelho R, Ramos E, Prata J, Barros H. Psychosocial indexes and cardiovascular risk factors in a community sample. Psychother Psychosom. 2000;69(5):261-74. [ Links ]
11. Horne R, Weinman J, Hankins M. The beliefs about medicines questionnaire: the development and evaluation of a new method for assessing the cognitive representation of medication. Psychol Health 1999;14:1-24. [ Links ]