SciELO - Scientific Electronic Library Online

 
vol.10 número4Conocimiento, percepción y práctica de la farmacovigilancia entre farmacéuticos comunitarios del sur de India índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Indicadores

Links relacionados

  • Em processo de indexaçãoCitado por Google
  • Não possue artigos similaresSimilares em SciELO
  • Em processo de indexaçãoSimilares em Google

Compartilhar


Pharmacy Practice (Granada)

versão On-line ISSN 1886-3655versão impressa ISSN 1885-642X

Pharmacy Pract (Granada) vol.10 no.4 Redondela Out./Dez. 2012

 

ORIGINAL RESEARCH

 

Job satisfaction among chain community pharmacists: results from a pilot study

Satisfacción con el trabajo entre farmacéuticos comunitarios de cadenas: resultados de un estudio piloto

 

 

Ana L. Hincapie1, Stephanie Yandow2, Stephanie Hines2, Megan Martineau2 and Terri Warholak3

1MS. Graduate Associate. College of Pharmacy, University of Arizona. Tucson, AZ (United States)
2PharmD. College of Pharmacy, University of Arizona. Tucson, AZ (United States)
3PhD. Assistant Professor. College of Pharmacy, University of Arizona. Tucson, AZ (United States)

 

 


SUMMARY

Objectives: The objectives of this study were to obtain pilot data concerning the job satisfaction of Tucson area retail chain setting and to identify the facets of community practice that have the greatest contribution to job satisfaction.
Methods: This was a cross-sectional study of chain pharmacists in the Tucson area. The Warr-Cook-Wall questionnaire of job satisfaction was used to evaluate community pharmacists' satisfaction with their current position. This study used Rasch analysis to assess the validity and reliability of the questionnaire. The Rasch scores obtained for each respondent were used as a dependent variable in univariate and bivariate analyses to evaluate differences in job satisfaction.
Results: A total of 32 pharmacists responded from 129 chain community pharmacies in the cities of Tucson, Marana and Oro Valley, Arizona. The mean (SD) Rasch score for job satisfaction was 0.93 (2.1). Results from bivariate analysis indicate that pharmacists in the Tucson area with practice experience outside community pharmacy were less satisfied with their job compared to those without experience outside community pharmacy (p<0.01).
Conclusions: This pilot evaluation suggests that having pharmacy experience outside community practice affects pharmacist job satisfaction. Additionally, findings from this study indicate that there is reliability and validity evidence to support the use of the modified Warr-Cook-Wall questionnaire for assessing overall job satisfaction in chain community pharmacy practice.

Key words: Job Satisfaction. Professional Role. Professional Practice. Pharmacists. United States.


RESUMEN

Objetivos: Los objetivos de este estudio fueron obtener datos piloto sobre la satisfacción en el trabajo de establecimientos de cadena en el área de Tucson e identificar las facetas de la práctica comunitaria que tienen mayor contribución a la satisfacción con el trabajo.
Métodos: Este fue un estudio transversal en farmacias de cadena del área de Tucson. Se utilizó el cuestionario de Warr-Cook-Wall de satisfacción con el trabajo para evaluar la satisfacción de los farmacéuticos comunitarios con su puesto actual. Este estudio usó el análisis de Rasch para evaluar la validez y la fiabilidad del cuestionario. Las puntuaciones de Rasch obtenidas de cada respondente fueron usadas como variable dependiente en los análisis univariado y bivariado para evaluar las diferencias en la satisfacción con el trabajo.
Resultados: Un total de 32 farmacéuticos respondieron de las 129 farmacias comunitarias de cadena en las ciudades de Tucson, Marana and Oro Valley, Arizona. La media (DE) de la puntuación de satisfacción con el trabajo fue de 0,93 (2,1). Los resultados del análisis bivariado indican que los farmacéuticos del área de Tucson con práctica fuera de la farmacia comunitaria estaban menos satisfechos que los que no la tenían (p<0,01).
Conclusiones: Este estudio piloto sugiere que tener experiencia fuera de la farmacia comunitaria afecta a la satisfacción con el trabajo. Además, los hallazgos de este estudio indican que existe evidencia fiable y válida para apoyar el uso del cuestionario Warr-Cook-Wall modificado para evaluar la satisfacción general con el trabajo en la práctica de la farmacia comunitaria en cadenas.

Palabras clave: Satisfacción en el Trabajo. Rol Profesional. Práctica Profesional. Farmacéuticos. Estados Unidos.


 

Introduction

Pharmacist job satisfaction, or dissatisfaction, plays a major role in many aspects of community pharmacy practice.1,2 An article by Saari et al. demonstrates that poor job satisfaction is directly associated with performance, especially for professionals like pharmacists.1 These performance problems can include incorrectly filling prescriptions, not detecting drug interactions and poor patient counseling. Thus, dissatisfaction may also affect the way that patients view the pharmacist and patients may then be inclined limit their interactions with the pharmacist.3 It is important to realize that a decrease in pharmacist performance could cause patient harm or even death.4,5

If a pharmacist is dissatisfied with his/her career, there is a potential for increased job turnover.1,6 Pharmacists job satisfaction has been found to be negatively associated with job turnover6, that is pharmacists with low levels job of satisfaction are more likely to resign their positions.6-8 Pharmacist job satisfaction does not only affect the pharmacist at his/her workplace, but has the potential to affect many other aspects of his/her pharmacist life. Research suggests that there is a strong link between job satisfaction and overall life satisfaction.4 If a pharmacist is not satisfied with his/her work he/she may bring those ill feelings from their job home and allow them to affect his/her life outside of the work place.

There are limited preliminary studies exploring aspects that impact community pharmacists job satisfaction and most of current evidence has been conducted in hospital settings. National surveys of community pharmacists show that perceived workload, and information technologies may have an effect on job satisfaction.3,9,10 In addition, continuing pharmacy education and preceptorship have been identified to be significant predictors of career and job satisfaction among community pharmacists.2,11 Other factors that have been shown to affect job satisfaction are treatment by management10,12 and other interpersonal interactions including patient contact and coworkers relationships8,13, compensation7,14, pharmacy ownership4, and practice setting.14-16 Community and chain pharmacists were found to be less satisfied with their jobs than pharmacists in other settings.15 The effect of demographic characteristics such age, sex, and education level on community pharmacist job satisfaction is still contradictory in the published literature.7,10,15,16

Most of the studies examining the factors affecting job satisfaction for community pharmacists have used instruments developed, validated and used in hospital settings17, which may not capture factors within the realm of community pharmacy. Additionally, previous studies have measured satisfaction with single-item or a limited number of Likert type rating scales within larger studies that assess general aspects of pharmacy worklife4,8,18 and thus they may not capture the different traits of the job satisfaction multidimensional construct. Finally, some studies have adapted instruments from other settings to measure community pharmacy facets not assessed with available standardized tools.14,19-21 However these studies used narrow population samples, thus more evidenced is needed to support validity of such instruments in community pharmacy.

Using the Warr-Cook-Wall questionnaire of job satisfaction, the objectives of this study are to: (1) obtain pilot data concerning the job satisfaction of Tucson area retail chain setting; 2) identify the facets of community practice that have the greatest contribution to job satisfaction; and 3) investigate the validity and reliability of the Warr-Cook-Wall questionnaire within a US community pharmacy study sample.

 

Methods

Sample

This investigation involved an anonymous cross-sectional study of chain pharmacists in the Tucson area. A comprehensive list of all chain pharmacies in the Tucson area, including Oro Valley and Marana, was compiled from online lists of stores obtained from each respective chain's website. There were 129 chain community pharmacies within this geographical area, all of which were included in questionnaire distribution. Each of the chain community pharmacies were faxed a questionnaire and cover letter a total of three times with approximately one month between distribution waves. Each store was sent one copy of the questionnaire and a cover letter explaining the study during each wave. The cover letter explained study goals, intent of the investigators, information about anonymity and included instructions on how to return the questionnaire (i.e. fax number). The cover letter also included a web address where the survey was available online. The cover letter encouraged all pharmacists working at each pharmacy to complete the survey. The store was responsible for making enough copies so that each pharmacist could complete his or her own survey. Responses were faxed back to a phone number that put the response directly into an email account. Received data were introduced into an Excel spreadsheet for analysis. This project was approved by the IRB at the University of Arizona.

Instrumentation

The questionnaire used for this study consisted of nine questions regarding job satisfaction and eleven questions regarding pharmacist demographics. The nine questions regarding job satisfaction were taken from the Warr-Cook-Wall scale job satisfaction questionnaire modified version20 that includes a "patient interaction" item and were used to evaluate pharmacists' satisfaction with their current position. All nine items measured job satisfaction using a 4 point Likert-scale with response options "very satisfied," "satisfied," "somewhat satisfied," and "not at all satisfied." Although the original scale measure satisfaction in a 7-point scale, a 4-point scale was used to decrease misinterpretation and mid-point selection. In previous studies in United Kingdom and New Zealand community pharmacists, the Warr-Cook-Wall job satisfaction scale obtained a Cronbach's alpha coefficient of 0.90.20,21

Pharmacists were asked what type of degree they held, the number of years of experience they had in pharmacy practice, and whether they had outside community pharmacy experience. Pharmacist were also asked five questions related to pharmacists work status: 1) type of chain pharmacy at which they were employed; 2) number of work hours worked per week; 3) the length of their longest work day (in hours); 4) average number of prescriptions filled per week; 5) if he/she is the pharmacists in charge; and 6) availability of patient services at their pharmacy. The inclusion of these questions was based on previous research on job satisfaction in community pharmacy.2,3,12,20

Data analysis

We used Rasch analysis to assess the validity and reliability of the questionnaire.19 The general consensus on the minimum sample size for Rasch analyses is 30 subjects.22 In Rasch analysis two types of fit statistics are used to assess how well the model fit the empirical data: Infit and Outfit for items mean squares (MNSQ). Infit MNSQ is an index weighted by the difference between persons' ability and item difficulty, which makes it more sensitive to unexpected responses. Outfit MNSQ is an unweighted index more sensitive to extreme responses. High infit and outfit (underfit) reflect lack of predictability of an item. Low infit and outfit (overfit) indicate over-predictability of an item.23 Mean-squares are expected to be around 1.0. An aspect of validity is unidimensionality, which indicates that items in a scale are measuring the same construct. To assess the unidimensionality of the questionnaire, the items Infit and Outfit MNSQ statistics were evaluated. An Infit MNSQ or Outfit MNSQ greater than 1.4, which denotes 40 percent more variance than expected, indicates that the item may be measuring a different construct; and an Infit MNSQ or Outfit MNSQ lower that 0.6 indicates item redundancy.24,25 Items that do not meet the Infit and Outfit MNSQ criteria were removed from the model. To evaluate whether the 4 point-Likert rating scaled used ("very satisfied" to "not satisfied at all") fits the Rasch model criteria, the following characteristics were assessed (Linacre): (1) at least 10 observations per category; (2) regular observation distribution; (3) average measures advance monotonically with category; (4) outfit MNSQ less than 2; (5) step calibration advance; (6) ratings imply measures and measure imply ratings; and (7) step difficulties advance greater than or equal to 1.4 logits and less than 5 logits.

Construct validity was determined by examining the gaps in the measurement of each item's calibration value. Z-test analysis was used to determine the significance of gaps. The presence of significant gaps indicates that some domains of the job satisfaction construct are not assessed.25 Reliability refers to the level of true and error variance in an observed score. The Rasch person reliability index denotes the reproducibility of person placement across other items measuring the same construct, which is equivalent to the Cronbach's alpha (test reliability) in classical test theory.22,23,26

An overall score of job satisfaction among community pharmacists in Tucson was calculated converting the ordinal responses from the Warr-Cook-Wall questionnaire to interval data (logits) with the Rasch Model if the Rasch model's requirements mentioned above were met. A higher score in logits represents greater job satisfaction (logits typically ranges from -4 to 4). The Rasch overall score was used as a continuous variable in descriptive, and non-parametric bivariate analyses to explore differences in job satisfaction and age, type of degree (PharmD/Non-PharmD), years of experience, workload (Avg. prescription volume/week, and whether they worked more than 36 hours/week), type of pharmacy chain (Grocery/non-grocery), having experience outside of community practice (yes/no), provision of patient care services at the pharmacy (yes/no) and job position (Pharmacist in charge yes/no) the selection of these variables was based on previous studies.15 Statistical significance was set at 0.05. The statistical software used were Winsteps for Rasch analyses and STATA 11.

 

Results

TDuring the period from August 2011 through November 2011, a total of 32 pharmacists responded from 129 chain community pharmacies in the cities of Tucson, Marana and Oro Valley, Arizona. Twenty-two questionnaires were received via fax while 10 were submitted online. Our estimated response rate was roughly 12%, which was calculated using an average of 2 pharmacists per store. The true count of pharmacists working at each store was not available. Pharmacists holding a Doctor of Pharmacy degree (PharmD) made up 68% (n=21) of respondents and the remaining 32% (n=11) of respondents held a Bachelor of Science in Pharmacy (BS) degree. Fifty-five percent of pharmacists surveyed (n=16) were employed at a grocery chain and 45% (n=13) were based out of a non-grocery chain store and almost two thirds (n=30) of participants were pharmacists in charge. Most of respondents indicated that they provided patient care services (94% n=30). Additional demographic data can be found in Table 1.

 

 

Validity and Reliability Results for the modified Warr-Cook-Wall questionnaire

Total fit (Infit and Outfit MNSQ) for persons and items were 1.01 and 1.02 respectively, which are around the expected 1.0.This indicates that the data fit the model appropriately and responses show a logical pattern. The item measures, standard errors, and fit statistics for each item are described in Table 2. All but items 3 and 8 fit the expected Infit and Outfit MNSQ27 range between 0.6 and 1.4, which means that they were not overfitting or misfitting items; the two items that did not meet these criteria were removed from the analysis. Additionally, the model reliability estimate was 0.83, which is higher than the accepted 0.7 threshold.28 The persons' mean score of job satisfaction was 0.93, which indicates that pharmacists found items moderately easy to agree with.

 

 

Table 3 and Figure 1 show that the instrument demonstrated appropriate functioning of the rating scale categories used. The number of observations in each category was greater than 10 and the average category measures increased with the rating scale categories. Pharmacists responding with a "1-Not Satisfied at all" showed an average measure (-1.64) lower than those responding with a "2-Satisfied" (average measure= -0.36), etc. There was no evidence of misfit categories (Outfit MNSQ <2), and the probability curve in Figure 1 depicts that all category values were more likely to be used at some point of the continuum thus no inversions were found.

 

 

 

Figure 2 shows the order of the items relative to the optimized rating categories and measures and what would be expected to be more or less challenging. From this Figure, it appears that there were gaps in the measurement continuum (between items 5 and 7, and items 1 and 9), which were not statistically significant (p=0.2). Additionally, the item related to patient contact was the easiest to endorse, which means that the respondents were more likely to agree with this statement.

 

 

Job satisfaction analysis

The mean (Standard Deviation) Rasch score for job satisfaction was 0.93 (2.1) (See Table 2), which indicates that pharmacists found items moderately easy to agree with. Table 4 displays the results of the analysis exploring differences between respondents' overall Rasch job satisfaction scores and multiple work related factors. It was found that pharmacists' job satisfaction score was statistically significantly smaller for those who reported having experience outside community practice.

 

 

Discussion

This paper reports the results of a pilot study that assessed the job satisfaction of Tucson area retail chain setting. Findings from this study indicate that responding pharmacists in the Tucson area with practice experience outside community pharmacy were less satisfied with their job. These findings are consistent with those of previously published research.3,15,29 Several reasons can be thought to explain these results. First, there could be a perception of over-qualification for the job that may not be present while working in hospital settings. This is reflected in the responses to the item "opportunities to use abilities" where only one quarter of respondents indicated, "very satisfied" to this item. Second, pharmacist in community settings may be have limited opportunities to growth within the organization.2

The instrument used in this study showed validity and reliability as determined by the Rasch model when two items were removed from the original questionnaire: satisfaction with colleagues and amount of variety in the job. Examination of the items that misfit the model reveled a fit MNSQ greater than two, which indicates that these items showed excessive unexplained noise. This may be explained by misunderstanding of the items.

"Recognition you get for good work" was the item most challenging to agree with. This is consistent with the Maslow's Theory of Motivation of job satisfaction where "recognition" is a factor located at a higher level; and can be achieved only after social, safety and psychological needs are met.30 On the other hand, "Patient contact" was the easiest item to endorse. This could occur because community pharmacists in chain settings have more patient contact, which favors interpersonal interactions. Research has found that pharmacist value these interpersonal interactions and determine their intention to stay on a job.8

Due to an unintentional inversion of the response scale during questionnaire development, we expected to find an inversion of the probability curve in the Rash model. However, the analysis showed that respondents did not detect the inversion of the questionnaire scale. That is, respondents selected the option "satisfied" as it was "somewhat satisfied." This could be explained by a potential familiarity with 4-point Likert-scale surveys that may facilitate the identification of the anchors in the scale without carefully reading options in between. Additionally, research shows that people tend to respond on the left side of a scale and response order bias is affected by readers' habits.31,32

An area for future research may include broadening the variable "experience outside community practice" to determine to what pharmacists are comparing to, when assessing their level of satisfaction. Aspects to consider include the extent to which the use of clinical skills in other settings affects job satisfaction and years and related activities of experience working outside community practice.

As several of the previously reviewed studies have confirmed, the lack of satisfaction with job can have negative impacts on patient care and safety and can even be a motivating factor for leaving the job or the profession all together. Conversely, researchers have found that pharmacists that are more satisfied with their work are highly motivated to do a good job and make patients and employees encounters with the pharmacist more positive.2 For this reason, pharmacy job satisfaction surveys are an important part of identifying the weaknesses of a specific job so that those weaknesses can be corrected or minimized, while maximizing the strengths, which leads to a better and safer experience for everyone involved.

Limitations

This study analyzed information from pharmacists working within the cities of Tucson, Marana and Oro Valley and results may not be generalizable to pharmacists in other geographic locations. The compiled list of all the pharmacies in the geographic location did not include independent pharmacies, so the study cannot be generalized to other pharmacy types. Another limitation is that some pharmacists might not have been in the pharmacy on the day of questionnaire dissemination, which means it is possible that not all of the pharmacists received a questionnaire or to know if a pharmacist actually filled the questionnaire. Even if a pharmacist received a questionnaire he or she may not have completed it, or the pharmacy manager may not have distributed the questionnaires to all of the pharmacists working at the store.

 

Conclusions

This pilot evaluation suggests that experience outside community practice affects pharmacist job satisfaction. Additionally, findings from this study indicate that there is reliability and validity evidence to support the use of the modified Warr-Cook-Wall questionnaire for assessing overall job satisfaction in chain community pharmacy practice.

 

Conflict of interest

The authors declare no conflict of interest.

No funding was provided for this project.

 

References

1. Saari LM, Judge TA. Employee attitudes and job satisfaction. Hum Resour Manage. 2004;43(4):395-407.         [ Links ]

2. Murawski MM, Payakachat N, Koh-Knox C. Factors affecting job and career satisfaction among community pharmacists: a structural equation modeling approach. J Am Pharm Assoc. 2008;48(5):610-620.         [ Links ]

3. Kreling DH, Doucette WR, Mott DA, Gaither CA, Pedersen CA, Schommer JC. Community pharmacists' work environments: evidence from the 2004 National Pharmacist Workforce Study. J Am Pharm Assoc. 2006;46(3):331-339.         [ Links ]

4. Mott DA, Doucette WR, Gaither CA, Pedersen CA, Schommer JC. Pharmacists' Attitudes Toward Worklife: Results from a National Survey of Pharmacists. J Am Pharm Assoc. 2004;44(3):326-336.         [ Links ]

5. Bond CA, Raehl CL. Pharmacists' assessment of dispensing errors: Risk factors, practice sites, professional functions, and satisfaction. Pharmacotherapy. 2001;21(5):614-626.         [ Links ]

6. Gaither CA. Career commitment: a mediator of the effects of job stress on pharmacists' work-related attitudes. J Am Pharm Assoc. 1999;39(3):353-361.         [ Links ]

7. Seston E, Hassell K, Ferguson J, Hann M. Exploring the relationship between pharmacists' job satisfaction, intention to quit the profession, and actual quitting. Res Social Adm Pharm. 2009;5(2):121-32.         [ Links ]

8. Gaither CA, Kahaleh AA, Doucette WR, Mott DA, Pederson CA, Schommer JC. A modified model of pharmacists' job stress: The role of organizational, extra-role, and individual factors on work-related outcomes. Res Social Adm Pharm. 2008;4(3):231-43.         [ Links ]

9. Lerkiatbundit S. Predictors of job satisfaction in pharmacists. J Soc Adm Pharm. 2000;17(1):45-50.         [ Links ]

10. Gubbins TV, Rascati KL. Satisfaction with management and overall job satisfaction of Texas chain store pharmacists. J Pharm Mark Manage. 1992;6(3):59-74.         [ Links ]

11. Payakachat N, Ounpraseuth S, Ragland D, Murawski MM. Job and career satisfaction among pharmacy preceptors. Am J Pharm Educ. 2011;75(8):153.         [ Links ]

12. Ferguson J, Ashcroft D, Hassell K. Qualitative insights into job satisfaction and dissatisfaction with management among community and hospital pharmacists. Res Social Adm Pharm. 2011;7(3):306-316.         [ Links ]

13. Mihm DJ, Mihm LB, Lonie JM, Dolinsky D. Selected perceptual determinants of pharmacy students' expected job satisfaction: a pilot study. Currents in Pharmacy Teaching and Learning. 2011;3(3):185-191.         [ Links ]

14. Hardigan P, Carvajal M. Job satisfaction among practicing pharmacists: a Rasch analysis. The Internet Journal of Allied Health Sciences and Practice. 2007;5(4):1-9.         [ Links ]

15. Maio V, Goldfarb NI, Hartmann CW. Pharmacists' job satisfaction: variation by practice setting. P & T. 2004;29(3):184-190.         [ Links ]

16. Carvajal MJ, Hardigan PC. Pharmacists' sources of job satisfaction: inter-gender differences in response. Am J Pharm Educ. 2000;64(4):420-425.         [ Links ]

17. Kerschen AM, Armstrong EP, Hillman TN. Job satisfaction among staff, clinical, and integrated hospital pharmacists. J Pharm Pract. 2006;19(5):306-312.         [ Links ]

18. Mott DA, Doucette WR, Gaither CA, Kreling DH, Pedersen CA, Schommer JC. Pharmacist Participation in the Workforce: 1990, 2000, and 2004. J Am Pharm Assoc. 2006 May-Jun;46(3):322-330.         [ Links ]

19. Hardigan PCM. Application of Rasch rating scale model to analysis of job satisfaction among practicing pharmacists. J Am Pharm Assoc (2003). 2008;48(4):522-529.         [ Links ]

20. Hassell K, Seston E, Shann P. Measuring job satisfaction of UK pharmacists: a pilot study. Int J Pharm Prac. 2007;15(4):259-264.         [ Links ]

21. Dowell AC, Westcott T, McLeod DK, Hamilton S. A survey of job satisfaction, sources of stress and psychological symptoms among New Zealand health professionals. N Z Med J. 2001;114(1145):540-543.         [ Links ]

22. Linacre JM. Optimizing rating scale category effectiveness. J Appl Meas. 2002;3(1):85-106.         [ Links ]

23. Bond TG, Fox CM. Applying the Rasch model: Fundamental measurement in the human sciences. Lawrence Erlbaum; 2001.         [ Links ]

24. Wright BD, Linacre JM. Reasonable mean-square fit values. Rasch Measurement Transactions.1994;8(3):370.         [ Links ]

25. Baghaei P. The Rasch model as a construct validation tool. Rasch Measurement Transactions. 2008;22(1):1145-1146.         [ Links ]

26. Linacre JM. Relating Cronback and Rasch reliabilities. Rasch Measurement Transactions. 1999;13(2):696.         [ Links ]

27. Linacre JM, Wright B. Interpreting output tables. A User's Guide to BIGSTEPS WINSTEPS Rasch Model Computer Program. Chicago: MESA Press; 2001:94-115.         [ Links ]

28. Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD. What Is Sufficient Evidence for the Reliability and Validity of Patient-Reported Outcome Measures? Value Health. 2007;10(Suppl 2):S94-S105.         [ Links ]

29. Cox F, Carroll N. Comparison of practice patterns and job satisfaction of entry level PharmD and BS level graduates in hospital and community practice. Am J Pharm Educ. 1988;52(1):47.         [ Links ]

30. Salancik GR, Pfeffer J. An examination of need-satisfaction models of job attitudes. Adm Sci Q. 1977;22:427-456.         [ Links ]

31. Weng LJ, Cheng CP. Effects of response order on Likert-type scales. Educ Psychol Meas. 2000;60(6):908-924.         [ Links ]

32. Chan JC. Response-order effects in Likert-type scales. Educ Psychol Meas. 1991;51(3):531-540.         [ Links ]

 

 

Received (first version): 27.07.12
Accepted: 15.12.12

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons