SciELO - Scientific Electronic Library Online

 
vol.43 número3Receta electrónica: diferencias entre comunidades autónomas que afectan al acceso a los tratamientos y a la calidad de la atención farmacéuticaDimensiones del arco posterior de C2 para el guiado de tornillos translaminares. Estudio radiológico en población española índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


Anales del Sistema Sanitario de Navarra

versión impresa ISSN 1137-6627

Anales Sis San Navarra vol.43 no.3 Pamplona sep./dic. 2020  Epub 21-Jun-2021

https://dx.doi.org/10.23938/assn.0921 

Artículos Originales

A Spanish adaptation of the Quality in Psychiatric Care - Inpatient Staff (QPC-IPS) instrument: Psychometric properties and factor structure

Adaptación española de Calidad en la Atención en Salud Mental - Instrumento para los Profesionales de Hospitalización (QPC-IPS): propiedades psicométricas y estructura factorial

S Sanchez-Balcells1  , LO Lundqvist2  , JF Roldán-Merino3  , MT Lluch-Canut4  , M Callarisa Roca5  , NF Rodríguez Zunino1  , M Tomás-Jiménez1  , A Schröder6  , M Puig-Llobet4 

1Parc Sanitari Sant Joan de Déu. Sant Boi de Llobregat. Spain.

2University Health Care Research Center. Faculty of Medicine and Health. Örebro University. Örebro. Sweden.

3Department of Mental Health. Campus Docent Sant Joan de Déu - Fundació Privada. University of Barcelona. Barcelona. Spain.

4Department of Public Health, Mental Health and Maternal-Child Nursing. School of Nursing. University of Barcelona. Health Sciences Campus Bellvitge. L'Hospitalet de Llobregat. Barcelona. Spain.

5Department of Mental Health. Sant Joan de Déu School of Nursing. University of Barcelona. Barcelona. Spain.

6Faculty of Medicine and Health. Institute of Health sciences. Norwegian University of Science and Technology (NTNU). Trondheim. Norway.

Abstract

Background

Western countries share an interest in evaluating quality of care in the healthcare field. In spite of this, there is a lack of intercultural comparison of the perceptions of professionals. One reason for this may be the lack of standardized instruments. The objective of this study was to investigate the psychometric properties and dimensions of the Spanish version of the Quality in Psychiatric Care-Inpatients Staff (QPC-IPS) instrument.

Methods

After translation and revision of the instrument by a panel of experts, a questionnaire was obtained in Spanish that was administered to a pilot sample. A total of 163 professionals participated in the study.

Results

After conducting pilot testing and a cognitive interview with 30 professionals, it was determined that the QPC-IPS was adequate and could be self-administered. Confirmatory factor analysis confirmed six factors that explained 60.9% of the variation. In terms of internal consistency, a Cronbach's alpha of 0.92 was obtained for the full instrument. For test re-test reliability, the intraclass correlation coefficient for the overall questionnaire was 0.91. Convergent validity was analyzed using the NTP394 satisfaction instrument, yielding a positive correlation (0.58).

Conclusions

The results demonstrated that the psychometric properties in terms of internal consistency, temporal stability (test-retest), content validity, and construct validity (confirmatory factor analysis) were adequate. These results confirm that the structure of the Spanish version is similar to the original Swedish version of the QPC-IP.

Keywords: Inpatient care; Psychometric properties; Mental health professionals; Nursing; Quality of care

Resumen

Fundamento

Los países occidentales comparten un interés en evaluar la calidad de la atención en el campo de la salud. A pesar de esto, existe una falta de comparación intercultural de las percepciones de los profesionales. Una razón para esto puede ser la falta de instrumentos estandarizados. El objetivo de este estudio fue investigar las propiedades psicométricas y las dimensiones de la versión española del instrumento Quality in Psychiatric Care-Inpatients Staff (QPC-IPS).

Métodos

Tras la traducción y revisión del instrumento por un panel de expertos se obtuvo un cuestionario en español que fue administrado a una muestra piloto. La muestra estuvo configurada por 163 profesionales.

Resultados

Después de realizar una prueba piloto y una entrevista cognitiva con 30 profesionales, se determinó que el QPC-IPS era adecuado y podía autoadministrarse. El análisis factorial confirmatorio confirmó seis factores que explican el 60,9% de la varianza. Mediante el análisis de consistencia interna, se obtuvo un alfa de Cronbach de 0,92 para el total del instrumento. Para la fiabilidad del test-retest, el coeficiente de correlación intraclase para el cuestionario general fue de 0,91. La validez convergente se analizó utilizando el instrumento de satisfacción NTP394, produciendo una correlación positiva (0,58).

Conclusiones

Los resultados demostraron que las propiedades psicométricas en términos de consistencia interna, estabilidad temporal (test-retest), validez de contenido y validez de constructo (análisis factorial confirmatorio) fueron adecuadas.

Palabras clave: Atención hospitalaria; Propiedades psicométricas; Profesionales de la salud mental; Enfermería; Calidad de la atención

Introduction

There is growing interest in studying the quality and efficacy of mental health services1. Perceived quality in mental health has commonly been evaluated by assessing care practices and interventions based on a set of clinical practice guidelines2 (CPGs). However, these evaluation systems have been criticized for not taking the preferences of users into consideration3.

The generic instruments that exist for general evaluation of health services may not be relevant in a mental health context4 since mental health differs from other specializations regarding the interaction between patient and professional ascribed as a therapeutic tool, due to the element of compulsion in mental health care5.

There is no widely accepted definition of the concept of quality of care. Rather, the term refers to a multi-dimensional concept6 that is perceived by mental health care users as a positive concept, specifically in reference to good quality of care7. Unlike the concept of patient satisfaction, quality of care includes the perspectives of all the interested parties8. The experience of the mental health professional may be seen as something to bear in mind and may be used as an additional indicator of the quality of mental healthcare8. The essential components that make up this quality are the therapeutic setting, the therapeutic relationship and support, assessment, professional performance, assessment of practice, and environmental health9.

However, it has been demonstrated that the various professional disciplines involved in mental healthcare have differing points of view regarding exactly what characterizes quality in care10. The results of earlier studies11 suggested that the perspective of nurses in relation to the quality of care was focused on interpersonal relationships, while other professionals were centered on organizational structures.

Western countries share an interest in evaluating and improving quality of care in the healthcare field12. In spite of this, there is a lack of intercultural comparison of the perceptions of patients and staff regarding quality of care13. This is due primarily to the fact that cross-cultural research in psychiatric care lacks standardized instruments12.

Instruments for measuring mental health care need to be valid and reliable. Some of the instruments used have been ad hoc in nature and have been criticized because their psychometric properties have rarely been documented14. Nevertheless, a review of the literature revealed a large number of patient satisfaction instruments for hospitalized mental health patients, although the psychometric properties of these instruments demonstrated variable results15.

The experience of the mental health professional should be seen as something to bear in mind and may be used as an additional indicator of the quality of mental healthcare16. Assessment of quality of care by professionals provides information on the therapeutic relationship formed with patients, the setting and its impact on care practice, and the relationships with teams and managers17. However, it has been demonstrated that the various professional disciplines involved in mental healthcare have differing points of view regarding what constitutes quality in care18. The study by Mason et al11 suggested that the perspective of nurses in relation to the quality of care was focused on interpersonal relationships, while that of other professionals was centred on organizational structures.

To date, there has been no Spanish-language instrument for measuring the perspective of mental health professionals in terms of quality in psychiatry care. The Quality in Psychiatric Care-Inpatients Staff (QPC-IPS) instrument has the potential to fill this gap in research. This study is part of a larger research project to adapt the QPC-IPS to different international settings, test the psychometric properties and equivalence of dimensionality of the different language versions, and describe and compare the quality of inpatient psychiatric care across different countries.

Materials and methods

The objective of the present study was to describe the translation of the QPC-IPS into Spanish and the stages in its validation process and to test its psychometric properties.

A psychometric study was conducted, translating the QPC-IPS instrument into Spanish and assessing its psychometric properties, including content validity, convergent validity, construct validity, internal consistency, and test-retest reliability.

Adaptation of the Spanish instrument

Participants. The process of adaptation, translation and back-translation were carried out by a panel of experts made up of nurses, psychiatrists, a psychologist with expertise in psychometry, and care managers. After obtaining the definitive version of the instrument, a pilot test was carried out with 30 professionals from mental health hospital units.

Instrument. The perceptions of mental health professionals regarding the quality of care were obtained using the QPC-IPS instrument. The QPC-IPS consists of 30 items and measures six dimensions: Encounter (eight items), Participation (eight items), Discharge (four items), Support (four items), Secluded environment (three items), and Secure environment (three items). The QPC-IPS is based on the QPC-IP instrument by Schröder19. The definition was developed from a phenomenographic interview study7, and the instrument was tested for face validity in a pilot study and also empirically tested19.

Data collection. All of the items in QPC-IP were reformulated to adapt them to the context of staff working in psychiatric inpatient care. Each item begins with the sentence I experience that... and is assessed using a Likert-type scale with four options, going from 1 (totally disagree) to 4 (totally agree). The maximum score is 120 points and the minimum is 30 points. A high score in each dimension or for the scale as a whole indicates a perception of high quality on the part of the professional. In contrast, a low score would justify the need for intervention in the areas found to be lacking. For each item, there was also the possibility of answering not applicable. In addition, the questionnaire includes a number of background questions covering demographics and general clinical characteristics, and at the end of the questionnaire, there is an open-ended question inviting professionals to make comments regarding the quality of care being provided.

The original QPC-IPS instrument in Swedish was translated into Spanish using a translation/back-translation process. First, the Swedish version was translated into Spanish. The research group reviewed the translation and checked that the meaning of each item had been transmitted and translated correctly (cultural validation). Each item was rated on a scale of from 1 to 4 (minimum-maximum) with regard to its coherence, clarity, and relevance. Thereafter, the Spanish version was translated back into Swedish in order to check that the Spanish translation corresponded to the original Swedish text. The Swedish research group, the authors of the original QPC-IPS instrument, discussed the back-translation and compared it with the original Swedish versions to examine the convergence between the translation and the back-translation. After this discussion, the preliminary Spanish version of the QPC-IPS was created, paying attention to semantic equivalence (face validity). A pilot test was later conducted with the participation of 30 professionals, who underwent a cognitive interview (Fig. 1). They confirmed that the translated version of the instrument was easy to understand and complete.

QPC-IPS: Quality in Psychiatric Care - Inpatient Staff version; NTP 394 Job satisfaction: Overall Job Satisfaction Scale.

Figure 1.  Overview of the three-phase validation study. 

Analysis of psychometric properties

Participants. The sample size was estimated based on the directions in the Consensus-based Standards for the selection of Health Measurement Instruments (Cosmin)20, the Standards for Educational and Psychological Testing21, and criteria set by experts22. To determine internal consistency, it was estimated that a minimum of five individuals needed to be included for each item used (i.e., a minimum of 150 participants). To analyze temporal stability, it was estimated that a minimum of 61 professionals were needed to detect an intraclass correlation coefficient (ICC) of around 0.70 between the two administrations, assuming a confidence level of 95% and power of 80% in a bilateral comparison23. Finally, internal consistency of the full QPC-IPS and its dimensions was analyzed in a sample of 163 professionals (psychiatrists, psychologists, nurses, social workers, occupational therapists, coaches, nursing assistants) who were employed in psychiatric units at two psychiatric hospitals and voluntarily agreed to participate. Mental health professionals with less than six months' experience in mental health were excluded. Temporal stability was analyzed in a sample of 92 professionals, drawn from the total of 163 professionals, who completed the scale a second time, with an interval of 7-14 days. The study was approved by the hospital's independent ethics committee. All mental health professionals were informed of the study and signed the informed consent form as established under Spanish law. The data were anonymous.

Instrument. The definitive Spanish version SpanishQPC-IP was chosen along with the NTP394 General Satisfaction developed by Warr et al24, and validated in Spanish by Pérez and Fidalgo25 with a Cronbach's alpha (α) coefficient of 0.85-0.88 and ICC of 0.63. It is designed to cover all intrinsic and extrinsic aspects of working conditions. It is a self-reported scale consisting of fifteen items. The total score is obtained from the sum of the responses to each of the fifteen items, assigning values ranging from 1 (very unsatisfied) to 7 (very satisfied). The total score for the scale varies between 15 and 105, such that a higher score reflects greater general satisfaction. This scale has been extensively translated and adapted to other languages.

Data Collection. Meetings were first held with the mental health professionals of the different psychiatric units. In these meetings, they were informed of the study, and the voluntary collaboration of all mental health professionals who met the inclusion criteria was sought. Subjects were enrolled consecutively. After providing informed consent, each participant received an envelope containing a letter with detailed information about the study, two questionnaires, and instructions on how to complete them. The Spanish QPC-IPS scale was administered again after from seven to fourteen days to determine test-retest reliability.

Background data on the participants were collected: age, sex, nationality, professional category, years working in the ward, and duty day. The data collection took place during a six-month period from September 2017 to December 2017.

The item analyses included calculation of item mean, standard deviation (SD), percentage ceiling and floor effects, and corrected item-total correlation. A ceiling or floor effect was signalled by a response percentage equal to or greater than 20%26. The corrected item-total correlation for the items was calculated, estimating the correlation of each item with the scale as a whole and with each corresponding subscale, accepting a correlation of 0.30 as the lower limit23.

Cronbach's α coefficient was used to evaluate the instrument's internal consistency, both for the full instrument and for each dimension. The reliability was considered to be adequate if the index values were above 0.70. Test-retest reliability and temporal stability were analyzed using the intraclass correlation coefficient, with values between 0 and 1, where the value needed to be equal to or greater than 0.90 to be considered a good concordance22. Convergent validity was analyzed using Spearman's correlation coefficient with the NTP394 General Satisfaction scale. Construct validity was analyzed using a confirmatory factor analysis (CFA) with parameters estimated using the generalized least squares method with a polychoric correlation matrix. This method has the same properties as the maximum likelihood method, although with criteria that are less strict than normal. It is used primarily to measure ordinal items. GFI (Goodness-of-fit Index) higher than 0.95 and RMSE (Root Mean Standard Error) absolute fit indices lower than 0.08, as well as the AGFI (Adjusted Goodness-of-fit Index), BBNFI (Bentler Bonnet Normed Fit Index), and BBNNFI (Bentler Bonnet Non-Normed Fit Index) incremental fit indices, were calculated higher than 0.95. Parsimony-based indices were calculated using the reduced Chi-squared, defined as the ratio between the Chi-squared value and the number of degrees of freedom; values between 2 and 6 were considered acceptable27.

In all of the statistical tests mentioned, the confidence level used was 95%. The SPSS statistics package was used for statistical analysis of the data, and EQS version 6.1 for the CFA28.

The study was approved by the clinical research ethics committee in a hospital (PIC-128-15), and permission to carry it out was granted by the coordinators and supervisors of the respective psychiatric units. All questionnaires were confidential, and all the patients signed informed consent in accordance with existing Spanish legislation. Their participation was voluntary.

Results

For the adaptation of the Spanish instrument, a panel of experts produced the results based on coherence, clarity, and relevance greater than 3. No items required modification. After conducting a pilot testing and a cognitive interview with 30 professionals, it was determined that the QPC-IPS was adequate and could be self-administered. The results of this phase were positive, and there were no problems in the comprehension or administration of the questionnaire (Table 1).

Table 1.  Semantin equivalence of items from English that were metrically validated 

Item English Spanish
1 The patients have influence over their own care and treatment Los pacientes deciden sobre sus cuidados y tratamiento
2 There is a high level of security at the ward La seguridad de la unidad es elevada
3 The patients have access to a private place where they can withdraw when they want to be left in peace and quiet Los pacientes tienen acceso a un lugar privado, al que pueden ir cuando quieran estar tranquilos
4 The patients can feel secure together with their fellow patients Los pacientes pueden sentirse seguros junto con los demás pacientes
5 The patients' opinions about the correct care and treatment for them are respected La opinión de los pacientes se respeta a la hora de establecer sus cuidados y tratamiento
6 The patients are involved in deciding about their care Los pacientes participan en las decisiones de sus cuidados.
7 The patients receive support and the opportunity to talk when they need to Los pacientes reciben apoyo y escucha cuando lo necesitan
8 There is cooperation in planning the patients' future care and day-to-day activities Los pacientes participan en el tratamiento de forma continuada y de las actividades diarias
9 The patients are not disturbed by their fellow patients Los pacientes no se sienten molestados por otros pacientes
10 The staff are involved and are out among the patients in the ward Los profesionales se implican y se encuentran en la unidad junto a los pacientes
11 The staff treat the patients with warmth and consideration Los profesionales tratan a los pacientes con consideración y proximidad
12 If patients are angry and irritated, the staff are concerned enough to want to know why Los profesionales se preocupan por conocer el motivo por el cual el paciente está enfadado o irritado
13 The patients' previous experiences of medical treatment are utilised in the best possible way La experiencia del paciente se tiene en cuenta a la hora de establecer el tratamiento médico
14 The patients learn to recognise signs of deterioration in their mental health Se ofrece a los pacientes educación sanitaria para aprender a identificar los síntomas de empeoramiento de la enfermedad
15 The staff respect the patients Los profesionales tratan con respeto al paciente
16 The patients are offered follow-up after discharge Se ofrecen seguimiento a los pacientes después del alta
17 The patients are given help to find an occupation before discharge Los pacientes reciben ayuda para buscar trabajo u otras ocupaciones
18 The staff show that they understand the patients' feelings Los profesionales comprenden los sentimientos de los pacientes
19 The staff prevent patients from hurting others, if the patients have such thoughts Los profesionales ofrecen estrategias a los pacientes para evitar que hagan daño a los demás, si tienen estas ideas
20 The staff have the time to listen to the patients Los profesionales dedican tiempo a escuchar a los pacientes
21 The patients are given information about where they can go if they need help following discharge Los pacientes reciben información sobre dónde pueden dirigirse si necesitan ayuda una vez finalizado el ingreso
22 The staff prevent patients from hurting others, if the patients have such thoughts Los profesionales ofrecen estrategias a los pacientes para evitar que se hagan daño a sí mismos, si tuviesen ese tipo de ideas
23 The staff help the patients understand that it is not shameful to suffer from mental health problems Los profesionales ayudan a entender que no es vergonzoso tener una enfermedad mental
24 The staff help the patients understand that the feelings of guilt and shame must never prevent them from seeking care Los profesionales ayudan a entender que los sentimientos de culpa y de vergüenza no debe impedir que pida ayuda
25 The staff are concerned about the patients' care and treatment Los profesionales se preocupan por los cuidados y tratamiento de los pacientes
26 Private rooms are available Hay posibilidades de tener habitación individual
27 The patients are informed in an understandable way about their mental health problems/diagnosis Se informa al paciente de forma clara sobre su enfermedad mental y su diagnóstico
28 There is a private place where patients can receive visits from their next of kin Hay un sitio íntimo donde los pacientes pueden recibir visitas de sus familiares y amigos
29 There is a private place where patients can receive visits from their next of kin Se ofrece información a los pacientes sobre su enfermedad mental para que puedan participar en su tratamiento
30 There is a private place where patients can receive visits from their next of kin Los pacientes reciben información sobre diferentes opciones de tratamiento para poder decidir cuál les parece la más adecuada

The analysis of psychometric properties was carried out on responses obtained from 173 professionals. A total of ten returned questionnaires were excluded as they had 30% or more missing data. A total of 163 mental health professionals belonging to two hospitals (46.6 and 56.8%) in two districts of the province of Barcelona participated. The average age was 38.7 years (SD: 10.3), and 57.1% were women. In relation to nationality, 96.9% were Spanish. With regard to professional category, 39.9% were nurses and 47.2% nursing assistants. Psychology and psychiatry professionals accounted for 7.4% of the sample. More than half of the professionals worked in day-shift (60.1%). The mean number of years spent working in the current unit was 4.4 (SD: 5.6).

The mean item value ranged from 2.21 to 3.63, and the standard deviation ranged from 0.58 to 1.03. The item with the highest percentage ceiling rankings was item 15 (the staff respect the patients, 68.1%) and only two items presented floor ranking: item 2 (there is a high level of security at the ward, 25.2%) and item 28 (there is a private place where patients can receive visits from their next-of-kin, 25.8%) (Table 2).

Table 2.  Descriptive statistics of the items of the SPANISH-QPC-IPS 

Content of the summarized items Total sub-scale Mean SD % Floor % Ceiling Corrected ítem-total correlation
Encounter 0.86 3.38 0,65
P7 The patients receive support and the opportunity to talk when they need to 3.33 0.69 0 45,4 0,51
P10 The staff are involved and are out among the patients in the ward 3.34 0.69 1,8 4,8 0,55
P11 The staff treat the patients with warmth and consideration 3.43 0.64 0 50,9 0,66
P12 If patients are angry and irritated, the staff are concerned enough to want to know why 3.42 0.64 0,6 49,1 0,7
P15 The staff respect the patients 3.63 0.59 0,6 68,1 0,67
P18 The staff show that they understand the patients' feelings 3.15 0.67 0,6 30,1 0,55
P20 The staff have the time to listen to the patients 3.26 0.73 1,8 41,1 0,63
P25 The staff are concerned about the patients' care and treatment 3.53 0.58 4,3 57,7 0,65
Participation 0.82 2.74 0.70
P1 The patients have influence over their own care and treatment 2.26 0.72 12.9 3.7 0.51
P5 The patients' opinions about the correct care and treatment for them are respected 2.67 0.68 3.1 9.2 0.56
P6 The patients are involved in deciding about their care 2.60 0.68 6.1 4.9 0.53
P13 The patients' previous experiences of medical treatment are utilized in the best possible way 2.76 0.75 4.9 14.7 0.65
P14 The patients learn to recognize signs of deterioration in their mental health 3.07 0.77 3.1 30.1 0.51
P27 The patients are informed in a comprehensible way about their mental health problems/diagnosis 3.12 0.63 1.2 24.5 0.44
P29 The patients receive information about their mental health problems so that they can participate in their care 3.08 0.63 0.6 25.2 0.61
P30 The patients are given information about different treatment alternatives so that they can decide what is best for them 2.38 0.76 11.0 7.4 0.58
Support 0.86 3.44 0.64
P19 The staff prevent patients from hurting others, if the patients have such thoughts 3.39 0.69 1.2 49.1 0.63
P22 The staff prevent patients from hurting others, if the patients have such thoughts 3.44 0.60 0.6 48.5 0.73
P23 The staff help the patients understand that it is not shameful to suffer from mental health problems 3.44 0.66 1.2 52.1 0.74
P24 The staff help the patients understand that feelings of guilt and shame must never prevent them from seeking care 3.49 0.63 0.6 55.2 0.76
Secluded environment 0.66 2.5 0.97
P3 The patients have access to a private place where they can withdraw when they want to be left in peace and quiet 2.61 0.95 13.5 19.6 0.44
P26 Private rooms are available 2.59 0.96 14.7 19.6 0.43
P28 There is a private place where patients can receive visits from their next-of-kin 2.30 1.00 25.8 13.5 0.55
Secure Environment 0.76 2.47 0.89
P2 There is a high level of security at the ward 2.43 1.03 25.2 15.3 0.57
P4 The patients can feel secure together with their fellow patients 2.78 0.88 9.2 20.9 0.70
P9 The patients are not disturbed by their fellow patients 2.21 0.78 18.4 3.7 0.55
Discharge 0.62 2.99 0.72
P8 There is cooperation in planning the patients' future care and day-to-day activities 2.90 0.64 1.2 14.7 0.39
P16 The patients are offered follow-up after discharge 2.98 0.86 6.1 30.1 0.33
P17 The patients are given help to find an occupation before discharge 2.80 0.81 6.7 19.0 0.42
P21 The patients are given information about where they can go if they need help following discharge 3.30 0.59 7.4 35.6 0.49

SD: Standard deviation

The Cronbach's α internal consistency coefficient for the full scale was 0.92, with values above 0.70 being achieved in four of the six dimensions of the Spanish QPC-IPS (Table 3). In the Discharge and Secluded Environment dimensions Cronbach's α values of 0.62 and 0.66 were obtained, respectively. These levels are lower than desirable but close to 0.70. The α values were also calculated excluding each item or question from the scale, and no exclusion of an item was found to improve the internal consistency of the scale in a relevant way. All items had item-total correlations > 0.30.

Table 3.  Cronbach's α and intraclass correlation coefficients test-retest 

Factors/dimensions of the questionnaire Cronbach's alpha ICC CI 95%
F1: Encounter 0.86 0.86 0.80-0.91
F2: Participation 0.82 0.89 0.83-0.92
F3: Support 0.86 0.72 0.58-0.82
F4: Secluded environment 0.66 0.85 0.77-0.90
F5: Secure environment 0.76 0.85 0.78-0.90
F6: Discharge 0.62 0.68 0.52-0.79
Total 0.92 0.91 0.86-0.94

ICC: intraclass correlation coefficient; CI: confidence interval.

The analysis of the ICC, shown in Table 2, demonstrated that test-retest reliability was 0.91 (95% CI: 0.87-0.94; n=92) and that the test was adequate for all dimensions except for the Discharge dimension, which scored 0.68 (95% CI: 0.52-0.79).

Convergent validity was analyzed using Spearman's correlation coefficient with the NTP394 General Satisfaction scale. The correlation obtained was rho=0.58 (p<0.0001).

The CFA revealed that the Encounter dimension and the Support dimension presented the largest factor loadings, while the Participation dimension and the Discharge dimension presented the lowest (Fig. 2). However, all of the loadings were statistically significant.

*: p < 0.05; LS: least squares.

Figure 2.  Factor loadings derived from the least square estimation (least squares). Confirmatory factor analysis (λij). 

All QPC-IPS dimensions showed high inter-correlations, except for the Support dimension, which scored below 0.50 with the Secluded environment and Secure environment dimensions (0.42 and 0.41 respectively).

The Chi-squared test was statistically significant but the chi-squared by degrees of freedom ratio was 2.08 (X2/df). As shown in Table 4, the other absolute fit, incremental fit, and the parsimony-based indices analyzed also showed a reasonably good fit.

Table 4.  Goodness-of-fit indices of the confirmatory model 

Index Value
Bentler Bonnet Normed Fit Index 0.69
Bentler Bonnet Non-normed Fit Index 0.78
Goodness-of-Fit 0.96
Adjusted Goodness of Fit Index 0.96
Comparative Fit Index 0.81
Root Mean Standard Error 0.08
Cronbach's alpha 0.92
Adjusted goodness test χ2390= 813.54*
Adjustment reason χ2/df = 2.08

df: degrees of freedom; *: p<0.0001.

Discussion

The objective of the present study was to describe the translation of the QPC-IPS into Spanish and the stages of the validation process, and to test the psychometric properties of the Spanish QPC-IPS.

During the adaptation process of the QPC-IPS, a translation and back translation was performed, obtaining an instrument adapted to Spanish. Other studies29 of the QPC family have also obtained original instruments adapted with a similar process. The results of this phase were positive and there were no problems in the comprehension or administration of the questionnaire.

The results demonstrated that the psychometric properties in terms of internal consistency, temporal stability (test-retest), content validity, and construct validity (confirmatory factor analysis) were adequate. These results confirm that the structure of the Spanish version is similar to that of the original Swedish version of the QPC-IP.

The variability of most items was substantial, although most scores showed a ceiling or floor effect.

Cronbach's α for internal consistency was 0.92 for the full scale, with values close to or above 0.70 obtained for all dimensions. These values are considered adequate by Nunnally & Bernstein30, and are higher than or very similar to those obtained in the original version19 and in other studies that have used other versions of the QPC instrument29. The Discharge dimension obtained a Cronbach's α of 0.62 and the Secluded environment dimension an α of 0.66. These dimensions have shown less than adequate internal consistency in previous studies29. Given the influence of several elements on Cronbach's α coefficient, the low α value in these dimensions is probably due to the small number of items (two and three items, respectively). It is also worth pointing out that the reliability results obtained for the Spanish QPC-IPS are similar to those for other quality of care measures reported in a recent systematic review15.

Temporal stability has not previously been analyzed on any QPC-instruments. The analysis of the Spanish version of QPC-IPS is the first. The ICC indicates good temporal stability. The Discharge dimension showed the lowest ICC values (ICC = 0.68). One of the reasons for these low ICC values may be that patient discharges from the ward were not scheduled in advance. This means that there are different types of discharges according to the care pressure, and therefore the professionals respond with different scores depending on the level of pressure. Another explanation may be that the mental health professionals give the patients information on discharge intermittently. This means that one week the mental health professional may have time to inform, and the next week not. This could account for the low values for the Discharge dimension.

In comparison to the instruments considered in a recent systematic review15, our results for Temporal stability were similar to those for the Spanish QPC-IP: the Combined Assessment of Psychiatric Environments instrument (CAPE)31, with a Pearson correlation coefficient of 0.82 for the staff version, the General Practitioner Experiences Questionnaire instrument (GPEQ)32, with an ICC of 0.72-0.87, and the Psychiatric Out-Patient Experiences Questionnaire (POPEQ)33, with an ICC of 0.90.

The convergent validity of the Spanish QPC-IPS was examined by calculating Spearman's rho correlation coefficient with the NTP 394. A positive and moderate correlation was obtained with the NTP 394 as, according to Martínez et al34, values between 0.31 and 0.70 present a moderate association. In this sense, as the results move in a positive direction, it may be inferred that the greater the job satisfaction, the more positive the perception of the quality of care is. This is the only study in which convergent validity has been analyzed with one satisfaction instrument.

With respect to construct validation, the CFA of the Spanish QPC-IPS showed the same factor structure as the original version of the QPC-IP, i.e., six dimensions of quality in mental health care7. In terms of the proposed criteria, from Worthington and Whittaker35, the simple size was deemed sufficiently large to carry out CFA. It may be concluded that the dimensions of the Spanish QPC-IPS are identical to those of the original scale. In the Indonesian version of the QPC-IPS29, the same six dimensions were observed. Thus, the concept of quality of mental health professionals from a Spanish perspective is similar to that among Indonesian mental health professionals. It is likely that mental health professionals are more observant about activities performed by themselves or other staff members compared to the patients, who may not fully observe what mental health professionals actually do.

The fact that the QPS-IPS has six dimensions is consistent with the assumption that the concept of quality is multidimensional7. In relation to the individual items, in all cases factor loading was greater than 0.40, which is generally considered to be the minimum level36. The CFA revealed that all items presented adequate and statistically significant factor loading, resulting in adequate validity levels. The goodness-of-fit indices showed a reasonably good fit27. These goodness-of-fit results were very similar to those found in previous studies of the Quality in Psychiatric Care-In-Patients14, the Quality in Psychiatric Care-Out-Patients37, and the Quality in Psychiatric Care-Forensic In-Patients Staff instrument29.

Finally, we should note that the mental health professionals in this study gave the highest quality rating to Support, which is in the line with previous studies on patients and mental health professionals29 in an inpatient psychiatric care setting. The Encounter dimension was the second highest in terms of quality. Secluded environment was rated the lowest in terms of quality of care, as in the study by Lundqvist et al29 performed in Indonesia. One possible explanation for this is that in Spain there are no individual rooms available, except for isolation rooms for emergency crises. However, it should be noted that the Secluded environment dimension in the Spanish QPC-IP14 fit the original Swedish model and the Indonesian model29. This result indicates that the concept of secluded environment is more or less the same in Spain, Sweden, and Indonesia.

The limitations of this study are that QPC-IPS instrument was adapted to the inpatient context in Spain. Therefore, any generalized use of this adapted instrument must take into account that it should be applied to a similar population. Second, although the minimum sample required for the analysis was estimated22, the sample size was small. Finally, it should be noted that it was not possible to evaluate sensitivity to change or predictive validity, since the study design was transversal. This needs to be taken into account and addressed in the design of future studies.

The Spanish QPC-IPS is a simple and easily administered tool for measuring various aspects of quality in psychiatric inpatient care from the perspective of mental health professionals. Its six-factor structure and psychometric properties are consistent with those of the original instrument, lending support to its use to measure quality of care in Spanish-speaking populations. The results of such measurements could be used to improve the quality of the service provided. Future studies will need to look at the psychometric properties of this instrument in relation to other variables and other samples of mental health professionals, both in the community and in other settings.

Acknowledgments

We are grateful to the Psychiatric Research Centre, Örebro University, Örebro, Sweden, the Sant Joan de Déu Research Foundation, and the Parc Sanitari Sant Joan de Déu, Barcelona, Spain.

References

1. Aimola L, Jasim S, Tripathi N, Bassett P, Quirk A, Worrall A et al. Impact of a peer-review network on the quality of inpatient low secure mental health services: cluster randomised control trial. BMC Health Serv Res 2018; 18: 994. https://doi.org/10.1186/s12913-018-3797-zLinks ]

2. Boyer L, Lanco̧n C, Baumstarck K, Parola N, Berbis J, Auquier P. Evaluating the impact of a quality of life assessment with feedback to clinicians in patients with schizophrenia: randomised controlled trial. Br J Psychiatry 2013. https://doi.org/10.1192/bjp.bp.112.123463Links ]

3. Paludetto M, Camuccio CA, Cutrone F, Cocchio S, Baldo V. Can we have routine measurement of patient satisfaction in acute psychiatric settings? A cross sectional study. Arch Psychiatr Nurs 2015; 29: 447-453. https://doi.org/10.1016/j.apnu.2015.07.006Links ]

4. Nugter Ma, Hermens MLM, Robbers S, Van Son G, Theunissen J, Engelsbel F. Use of outcome measurements in clinical practice: how specific should one be? Psychother Res 2019; 29: 432-444. https://doi.org/10.1080/10503307.2017.1408975Links ]

5. Bartlett, P, Sandland, R. Mental health law: policy and practice. Oxford: Oxford University Press, 2013. [ Links ]

6. Information Resources Management Association. Healthcare ethics and training : concepts, methodologies, tools, and applications. Hershey, PA: IGI Global, 2017. [ Links ]

7. Schröder A, Ahlström G, Larsson BW. Patients' perceptions of the concept of the quality of care in the psychiatric setting: a phenomenographic study. J Clin Nurs 2006; 15: 93-102. https://doi.org/10.1111/j.1365-2702.2005.01241.xLinks ]

8. Farr M, Cressey P. Understanding staff perspectives of quality in practice in healthcare. BMC Health Serv Res 2015; 15: 123. https://doi.org/10.1186/s12913-015-0788-1Links ]

9. Kane CF. The 2014 Scope and standards of practice for psychiatric mental health nursing: key updates. Online J Issues Nurs 2015; 20: 1. [ Links ]

10. Bee P, Brooks H, Fraser C, Lovell K. Professional perspectives on service user and carer involvement in mental health care planning: a qualitative study. Int J Nurs Stud 2015; 52: 1834-1845. https://doi.org/10.1016/j.ijnurstu.2015.07.008Links ]

11. Mason T, Lovell A, Coyle D. Forensic psychiatric nursing: skills and competencies: I role dimensions. J Psychiatr Ment Health Nurs 2008; 15: 118-130. https://doi.org/10.1111/j.1365-2850.2007.01191.xLinks ]

12. Salyers MP, Fukui S, Rollins AL, Firmin R, Gearhart T, Noll JP et al. Burnout and self-reported quality of care in community mental health. Adm Policy Ment Health 2015; 42: 61-69. https://doi.org/10.1007/s10488-014-0544-6Links ]

13. Henderson C, Hales H, Ruggeri M. Cross-cultural differences in the conceptualisation of patients' satisfaction with psychiatric services-content validity of the English version of the Verona Service Satisfaction Scale. Soc Psychiatry Psychiatr Epidemiol 2003; 38: 142-148. https://doi.org/10.1007/s00127-003-0606-7Links ]

14. Lundqvist LO, Suryani AN, Rafiyah I, Schröder A. Indonesian adaptation of the Quality in Psychiatric Care - Inpatient (QPC-IP) instrument: psychometric properties and factor structure. Asian J Psychiatr 2018; 34: 1-5. https://doi.org/10.1016/j.ajp.2018.03.006Links ]

15. Sanchez-Balcells S, Callarisa Roca M, Rodriguez-Zunino N, Puig-Llobet M, Lluch-Canut MT, Roldan-Merino JF. Psychometric properties of instruments measuring quality and satisfaction in mental health: a systematic review. J Adv Nurs 2018; 74: 2497-2510. https://doi.org/10.1111/jan.13813Links ]

16. Arnetz BB. Staff perception of the impact of health care transformation on quality of care. Int J Qual Heal Care 1999; 11: 345-351. https://doi.org/10.1093/intqhc/11.4.345Links ]

17. Laker C, Rose D, Flach C, Csipke E, Mccrone P, Craig T et al. Views of the Therapeutic Environment (VOTE): stakeholder involvement in measuring staff perceptions of acute in-patient care. Int J Nurs Stud 2012; 49: 1403-1410. https://doi.org/10.1016/j.ijnurstu.2012.06.001Links ]

18. Demarco R, Flaherty L, Glod C, Merrill N, Terk K, Plasse M. Staff & client perceptions of unit quality: a pilot study. J Psychosoc Nurs Ment Heal Serv 2004; 42: 36. https://doi.org/10.3928/02793695-20040501-06Links ]

19. Schröder A, Wilde-Larsson B, Ahlstrom G. Quality in psychiatric care: an instrument evaluating patients' expectations and experiences. Int J Health Care Qual Assur 2007; 20: 141-160. http://doi.org/10.1108/09526860710731834Links ]

20. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol 2010; 10: 22. https://doi.org/10.1186/1471-2288-10-22Links ]

21. American Educational Research Association. Standards for educational and psychological testing. Washington, DC: American Educational Research Association, 2014; 33-47. [ Links ]

22. Streiner DL, Norman Gr, Cairney J. Health measurement scales : a practical guide to their development and use. 5ª ed. Oxford: Oxford University Press , 2015; 399. [ Links ]

23. Zou GY. Sample size formulas for estimating intraclass correlation coefficients with precision and assurance. Stat Med 2012; 31: 3972-3981. https://doi.org/10.1002/sim.5466Links ]

24. Warr PB, Cook JD, Wall TD. Scales for the measurement of some work attitudes and aspects of psychological well-being. J Occup Psychol 1979; 131: 241-259. https://doi.org/10.1111/j.2044-8325.1979.tb00448.xLinks ]

25. Pérez J, Fidalgo M. NTP 394: Satisfacción laboral: escala general de satisfacción. Madrid: Instituto Nacional de Seguridad e Higiene en el Trabajo, 1994. https://saludlaboralydiscapacidad.org/wp-content/uploads/2019/05/NTP-394-Satisfacci%C3%B3n-laboral-escala-general-de-satisfacci%C3%B3n.pdfLinks ]

26. Ware JE, Gandek B. Methods for testing data quality, scaling assumptions, and reliability: the IQOLA Project approach. International Quality of Life Assessment. J Clin Epidemiol 1998; 51: 945-952. [ Links ]

27. Rial A, Varela J, Abalo J, Lévy JP. El análisis factorial confirmatorio. En: Varela Mallou J, editor. Modelización con estructuras de covarianzas en ciencias sociales : temas esenciales, avanzados y aportaciones especiales. La Coruña: Netbiblo, 2006: 119-154. [ Links ]

28. EQS 6.2 for Windows. Encino: Multivariate Software Inc, 2006. [ Links ]

29. Lundqvist LO, Suryani Y, Hermiati D, Sutini T, Schröder A. A psychometric evaluation of the Indonesian version of the Quality in Psychiatric Care - Inpatient Staff (QPC-IPS) instrument. Asian J Psychiatr 2019; 46: 29-33. https://doi.org/10.1016/j.ajp.2019.09.027Links ]

30. Nunnally JC, Bernstein IH. The theory of measurement error. Psychometric Theory. New York: McGraw-Hill, 1994; 209-247. [ Links ]

31. Delaney KR, Johnson ME, Fogg L. Development and testing of the Combined Assessment of Psychiatric Environments: a patient-centered quality measure for inpatient psychiatric treatment. J Am Psychiatr Nurses Assoc 2015; 21: 134-147. https://doi.org/10.1177/1078390315581338Links ]

32. Bjertnaes OA, Garratt A, Nessa J. The GPS' Experiences Questionnaire (GPEQ): reliability and validity following a national survey to assess GPs' views of district psychiatric services. Fam Pract 2007; 24: 336-342. https://doi.org/10.1093/fampra/cmm025Links ]

33. Garratt A, Bjørngaard JH, Dahle KA, Bjertnæs ØA, Saunes IS, Ruud T. The Psychiatric Out-Patient Experiences Questionnaire (POPEQ): data quality, reliability and validity in patients attending 90 Norwegian clinics. Nord J Psychiatry 2006; 60: 89-96. https://doi.org/10.1080/08039480600583464Links ]

34. Martínez González MA, Sánchez Villegas A, Toledo Atucha EA, Faulin-Fajardo J. Bioestadística amigable. Barcelona: Elsevier, 2014. [ Links ]

35. Worthington R, Whittaker T. Scale development research: a content analysis and recommendations for best practices. Couns Psychol 2006; 34: 806-838. [ Links ]

35. Hair JF, Anderson RE, Tatham RL, Black WC. Multivariate data analysis: with readings. Statistics 1995; 151: 1-5. [ Links ]

37. Schröder A, Ahlström G, Larsson BW, Lundqvist L-O. Psychometric properties of the Quality in Psychiatric Care - Outpatient (QPC-OP) instrument. Int J Ment Health Nurs 2011; 20: 445-453. https://doi.org/10.1111/j.1447-0349.2011.00741.xLinks ]

Source of funding This work was supported by the Hospital Universitario Marqués de Valdecilla, the Asociación Nacional de Enfermería de Salud Mental, and the Fundació Privada per a la Recerca i la Docència Sant Joan de Déu FSJD. The study was also funded in part by the Nurse and Society Foundation under Nurse Research Projects Grants [PR-011/16].

Received: May 15, 2020; Revised: June 03, 2020; Accepted: July 31, 2020

Corresponding author: Juan Roldán-Merino, Campus Docent Sant Joan de Déu - Edifici Esade-3, C/ Miret i Sans, 10-16, 08034, Barcelona. E-mail: jroldan@santjoandedeu.edu.es

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License