ADMINISTRACIÓN-GESTIÓN-CALIDAD

 

Content Validity: The Human Papillomavirus Vulnerability Questionnaire

Validez de Contenido: Cuestionario de Vulnerabilidad al Papillomavirus Humano

 

 

Sepúlveda Carrillo, Gloria Judith*; Meneses Báez, Alba Lucia** and Goldenberg, Paulete***

* Nurse. Assistant Professor School of Nursing, Pontificia Universidad Javeriana, Bogotá-Colombia. E-mail: gloseca@hotmail.com
**Psychologist. Titular Professor School of Psychology Universidad El Bosque, Bogotá-Colombia
*** Sociologist. Doctor of Public Health. Professor, Preventive Medicine Department-Federal University of São Paulo / School of Medicine. Universidad Federal de São Paulo. Brazil.

 

 

---

ABSTRACT

Background: Accurate diagnosis about knowledge, behavior and risk perception relative to the HPV virus is a cornerstone to design effective interventions related to the promotion, prevention and reduction of the individual vulnerability to HPV among adolescents and young adults.
Objective: Obtain evidence related with content validity of a questionnaire that measures knowledge and willingness to act preventively against HPV in adolescents and young adults.
Method: 16 experts evaluated the questionnaire conceptual structure as well as the degree of agreement on items clarity and wording. Additionally, 42 college students underwent a cognitive interview in order to adjust instructions, vocabulary, and wording of items.
Results: The experts' agreement index estimated for wording and clarity of the questionnaire items was 0.88 and for the four dimensions was between 0.81 to 0.96. The estimated experts' agreement index for wording and clarity of the questionnaire items was 0.88 and for the four dimensions range from 0.81 to 0.96. The questionnaire conceptual structure was determined by applying the concept mapping methodology, using an alternating least-squares algorithm (ALSCAL) to perform non-metrical multidimensional scaling (MDS) technique with a two-dimension solution (Stress = 12%), and a hierarchical cluster analysis (HCA) using Ward's method; also it estimated a bridging statistic for each item in relationship to its cluster. The obtain data suggest that the questionnaire measures four dimensions: socio-demographic profile, sexual behavior, knowledge of transmission and prevention knowledge.
Conclusions: The findings for content validity support that the questionnaire measures knowledge and willingness to act preventively against HPV in adolescents and young adults from the vulnerability perspective.

Key words: Human papillomavirus; Vulnerability; Adolescent; Young Adult; Validity of Tests.

---

RESUMEN

Antecedentes: Es necesario realizar diagnósticos sobre conocimientos, comportamientos y percepción del riesgo a la infección del virus de Papiloma Humano (VPH), como base para desarrollar intervenciones efectivas que reduzcan la vulnerabilidad al VPH entre adolescentes y adultos jóvenes.
Objetivo: Establecer indicadores de validez de contenido de un cuestionario que mide conocimientos y disposición para actuar preventivamente frente al VPH en adolescentes y adultos jóvenes.
Metodología: El cuestionario fue evaluado por 16 expertos con el fin de determinar su estructura conceptual y el nivel de concordancia sobre la claridad y la redacción de los ítems. Este también se aplicó a 42 universitarios a quienes se les realizó entrevista cognitiva con el fin de ajustar léxico, instrucciones y forma de redacción de los ítems.
Resultados: Se estimó el índice de concordancia entre expertos en cuanto a redacción y claridad de los ítems cuestionario (0.88) y para las cuatro dimensiones (0.81-0.96). La estructura conceptual del cuestionario se determinó mediante la aplicación de la metodología de concept mapping, para ello se realizó un Análisis de Escalamiento Multidimensional no métrico de ALSCAL (Alternating Least Squared Scaling) con una solución para dos dimensiones (Stress = 12%) y un análisis jerárquico de conglomerados utilizando el método de Ward; además se estimó el estadístico de bridging para cada ítem con respecto al conglomerado. Estos datos sugieren que el cuestionario mide cuatro dimensiones: perfil sociodemográfico, comportamiento sexual, conocimiento de transmisión y conocimiento de prevención.
Conclusiones: El cuestionario presenta indicadores adecuados de validez de contenido para medir conocimientos y prácticas preventivas asociadas al (VPH) desde la perspectiva de la vulnerabilidad en adolescentes y adultos jóvenes.

Palabras clave: Papillomavirus Humano; Vulnerabilidad; Adolescente; Adulto Joven; Validez de las Pruebas.

---

 

Introduction

The human papillomavirus (HPV) is the most prevalent sexually transmitted disease (STD) in the sexually active population. The HPV is considered an important public health issue for its causal relationship with the cervical cancer^(1-3). Unlike others STDs, the HPV can be transmitted by genitals direct skin contact and during the anal and oral intercourse^(1,2,4,5,9). There are factors related to the sexual behavior which are directly associated with the infection such as: early onset of sexual intercourse, and the background risk of the sexual partner(s).

The most effective way to prevent the infection requires the absolute sexual abstinence but it is considered impractical. However, it is possible to reduce the likelihood of becoming infected with a monogamy relationship, circumcision, and the use of condom that only provides some degree of protection; and the use of vaccines that provide immunity to certain types of HPV⁽¹⁰⁾.

Scientific evidence indicates that most adolescents and young adults have information about HPV infection and its consequences; however, the majority do not know about ways of transmission, prevention methods and its consequences; in particular, the relationship between the virus and the development of several cancer types. Therefore, this age group has a low risk perception of acquiring HPV virus^(1.11-14).

In general, it has been found a good level of knowledge about other STDs among this population^(6,15); however, there is much misinformation about HPV despite its serious consequences^(11,12). It is possible that HIV education has eclipsed the teachings over other STDs, including HPV^{(6, 9,15,16)}.

Studies carried out with sexually active college students have shown that most of them have a low knowledge about HPV and its sexual risk behavior^{(12, 14, 17-22)}. A low knowledge combined with low risk perception about HPV infection increases the likelihood of infection⁽¹⁹⁾. Therefore, the first step to change the risk behavior is to get people to recognize the existence of HPV infection, in order to increase their risk perception of infection acquisition^(13,14,23); the second step is for people to use health services for diagnosis, treatment and monitoring of the damages caused by HPV evolution^{(12-14,17-27)}.

The population of adolescents and young adults require educational interventions that strengthen HPV knowledge through effective, accurate and truthful information based on their local community behavioral characteristics. The acknowledgment of specific conditions about HPV dissemination among this population is a key requirement in order to restrain the spread of this infection. Therefore, it is important to consider the behavioral dimension along with biological risk perception in order to point out the mediation between the ways of thinking and acting. The prevention models should take into account the concept of vulnerability for the development, implementation and evaluation of interventions that face HPV because it includes the dynamics of individual, social and programmatic dispositions.

The concept of vulnerability was developed by Mann, Tarantola & Netter, in the field of HIV prevention and has to do with the factors that need to be considered to avoid the exposition of individuals to the risk of infection transmission⁽²⁷⁾. Ayres & al. conceptualize vulnerability as "different susceptibility degrees and characteristics of individuals and communities to acquire the infection or be ill, that depend on social , programmatic and individual aspects as a whole, that come along with the problem and the resources to cope with it "^(28,29).

The assessment of individual variables is fundamental to address vulnerability for HPV infection given that personal vulnerability for the infection increases by the lack of precise, relevant and exhaustive information, the lack of skills to make adequate decisions, the difficulty to access to health services and supplies, and the grade of confidence to support or introduce behavioral changes⁽²⁷⁾.

Social vulnerability refers to the grade of protection that society owns related with the infection, it focuses on the economic, political and cultural aspects, along with other issues such as the social and gender relations, in particular⁽³⁰⁾.

The variables cited before, affect the programmatic vulnerability as well. This vulnerability evaluates the advantages and limitations of the prevention and control programs, as well as the assistance to the health programs, regardless if they come from governmental, non-governmental or private field, and constitutes a set of actions closely related to the social and individual scope⁽³⁰⁾. In this sense, the vulnerability is a relative dimension, that is, everybody is vulnerable, but each person has his/her own level and type of vulnerability according to his/her socio-economic, cultural, political, and religious and gender circumstances that go beyond the exclusively individual responsibility⁽³¹⁾.

It is evident the need to promote health interventions that will improve HPV knowledge in order to encourage behavioral changes in the communities. In this sense, the cited premises must address the diagnosis about knowledge, behavior and risk perception relative to the HPV virus in adolescents and young adults; as a starting point to design effective interventions related to the promotion, prevention and reduction of the individual vulnerability to HPV infection at local, regional and national communities.

In the literature reviewed for this work, there is no evidence about the use of questionnaires for measuring knowledge and behavior in order to design and carry out prevention programs to HPV infection in adolescents and young adults from the vulnerability perspective. This findings point out the need of an assessment tool that measures individual and social vulnerability, as well as actions of a programmatic plan. The individual vulnerability has to do with those behaviors that increase the likelihood of getting the HPV infection or be sick at VPH transmission contexts, along with the risk perception of developing the infection; the social vulnerability refers to the ability of the individual to access to health services, to obtain information, and make effective use of them; and the programmatic level is the strength of health actions in individuals for the prevention and control to infection⁽²⁹⁾.

In relation to the findings, the authors designed a questionnaire that measures the following aspects: a demographic profile (work, educational settings, family and type of health insurance); issues related to sexual behaviors (those can interfere with the vulnerability to infection of the virus); knowledge related to HIV transmission; and knowledge about HIV prevention (access to health services and perceived vulnerability to infection).

In consequence, it is necessary to obtain evidence that supports is the items of the questionnaire are properly written and relevant for measuring knowledge and willingness to act preventively against HPV in college students who are building their sexual experiences in the context of Bogotá-Colombia. So, the purpose of this study is to get evidence for the content validity of the questionnaire that measures knowledge and willingness to act preventively against HPV in adolescents and young adults.

 

Method

The authors designed a questionnaire that measures knowledge and willingness to act preventively against HPV in adolescents and young adults base on a review of literature. For the design of this questionnaire, the authors followed Lynn's methodology⁽³²⁾: definition of the construct to be measured and its dimensions, the development of the questionnaire items and the content validity indicators through expert assessment. In addition, a cognitive interview with potential users of the questionnaire was carried out to adjust vocabulary, instructions and items⁽³³⁾.

The content validation procedure of the questionnaire involved three stages: in the first stage, the questionnaire was evaluated by three thematic experts and two psychometrics experts, in order to assess relevance, clarity, accuracy and sufficiency of the items. They also examined if any item needed to be modified or if it was necessary to remove one or more items.

In the second stage, a new version of the questionnaire was designed taking in consideration the experts suggestions. This instrument had 47 items and four dimensions. 42 college students answered this questionnaire after they signed an informed consent to participate in this study.

A cognitive interview was carried out with the students in order to optimize the items structure in terms of clarity and comprehension. The authors also evaluate social desirability and based on the results, decided which items needed to be modified or had to be eliminated. In order to do that, students were given the instruction to identify grammar errors as well as difficulties regarding items comprehension; they asked for suggest different ways to improve item's wording; identify the items that were uncomfortable to answer and make suggestions to improve them; identify specific word that couldn't be understood; identify items that could and could not be answered honestly from the start; and make suggestions to make improve items wording so that participants can answer them in an honest way. Once the participants had completed the questionnaire, one of the researchers reviewed all the aspects mentioned before regarding the structure of items.

Based on the previous results, the authors developed the third version of the questionnaire which had 36 items, 14 open ended questions and 22 closed-ended questions. The items were organized into four dimensions: socio-demographic profile, sexual behavior, knowledge of HPV transmission and knowledge of HPV prevention.

In the third stage, 16 experts evaluated the questionnaire; five of them had participated in the first stage of this study. Two forms were assigned to each expert, the first one evaluated items appropriateness, regarding the four dimensions and each expert was asked to classify items in one of the four dimensions of the questionnaire⁽³⁴⁾, considering the following rules: each item can only measure one dimension; if necessary, new dimensions could be suggested; there is no limit of dimensions; the total number of dimensions cannot exceed the number of items; all items must belong to a dimension.

The second form had criteria to evaluate clarity and wording of items, so each expert was asked to rate each item in a numeric scale ranging from 1 to 5 as follows: 1= the item cannot be understood, has semantic, syntactic and grammar errors; 2= most words in the item have to be modified according to their meaning; 3= some words in the item have to be modified according to their meaning; 4= there is need for some modifications in specific words of the item; 5= the item is understandable, its semantics, syntax and grammar are adequate.

Data gathered form experts were analyzed in order to find out if item's wording was appropriate. Data analysis was carried out through the statistical package SPSS 19.0 version and EXCEL.

Content validity indicators involved the content agreement index proposed by Lynn (³²⁾ to determine the agreement level among experts regarding clarity and wording of items.

On other hand, the concept mapping methodology proposed by Trochim⁽³⁵⁾ and adapted by Carpenter⁽³⁴⁾ and Bedi⁽³⁶⁾ was used to evaluate the conceptual structure of the questionnaire, and for the data analysis was used SPSS 19 version. This methodology represents data through the shaping of points in a Cartesian coordinated system when there is information about similarities among items. In order to apply this methodology, first a data analysis was conducted using an alternating least-squares algorithm (ALSCAL) to perform non-metrical multidimensional scaling (MDS) technique with a two-dimension solution, following the procedure described by Carpenter⁽³⁴⁾.

For the MDS, the stress index was estimated. This index is used to determine the number of dimensions that better represent data, so values closer to zero indicate better fix, that is, when there is little discrepancy between the similarity matrix and the distances in the map resulting from the MDS. The stress index was estimated with different solutions (six) and the graph showing the stress values versus the number of dimensions analyzed. According to Trochim⁽³⁵⁾ when complex phenomena are addressed, stress values below 25% for two dimensions are considered acceptable. This index is used to determine how well the number of resulting dimensions (two in this case) represent data (Stress < .10 is considered ideal; values that range between >.10 and <.25 are considered adequate for studies like this).

Data obtained from X and y Cartesian coordinates, resulting from the non-metric MDS analysis were used to run the hierarchical cluster analysis (HCA) using the Ward algorithm⁽³⁵⁾. The results of HCA were used to select the ideal number of clusters (conceptual group) taking into account the semantic analysis of each one⁽³⁷⁾ and the estimated value of the bridging index for item and cluster^{(36, 37)}.

The bridging index is a mathematical index that is estimated for each item that configures a specific conceptual group, it has a range between zero and one and it points out how many times an item was classified with other items that were near in the map resulting from the non-metric MDS^{(36, 37)}. Bridging index values for a particular item (i) closer to zero show that the item is related to items in the corresponding cluster and values near >.70 show that the item has a weak relation with items in the conceptual group. Thus, it is assumed that items with bridging values closer to zero are conceptual related to the dimension that structures the cluster where the items were assigned to^{(37, 38)}.

Additionally, It was estimated the average bridging index per cluster using the method proposed by Bedi⁽³⁶⁾ values close to zero indicate that experts had a greater level of conceptual homogeneity respect to the items of a given cluster; by contrast, a high value of this index indicates that experts had different meanings for items of a particular cluster⁽³⁷⁾ for example, a cluster with low conceptual homogeneity would have an average bridging index greater than 0.7.

 

Results

Firstly, the results obtained for the 16 experts regarding the conceptual structure of the questionnaire using the concept mapping methodology are presented. Secondly, the concordance level related to the clarity and wording of the questionnaire.

Evidence of the conceptual structure of HPV vulnerability questionnaire.

To determine the frequency the items were grouped in the same conceptual dimension by experts, non-metric MSA ALSCAL was performed with different number of solutions (six) and the graphic that showed the values of stress versus the number of dimensions with a solution of two dimensions was analyzed. The value of stress obtained was 12% (<25% adequate) for a two dimensional solution (see figure 1)

 

 

The data of the coordinate obtained in the MSA of ALSCAL, were entered for the hierarchical cluster analysis using Ward's algorithm, these results were used to select the number of conceptual groups that best represented the data (see figure 2)

 

 

Finally, it was calculated the Bridging index for each item per cluster and it was estimated the Bridging index average per cluster (see table 1).

 

 

According to the observed data in table 1, the average values of the Bridging index per cluster obtained for this study, indicate that experts have conceptual homogeneity per cluster in general (38-59), data suggest that the test has an adequate conceptual structure in terms of the dimensions that comprise (demographic profile, sexual behavior, knowledge of transmission and prevention knowledge) (see Figure 2). However, the values obtained for the Bridging index per item (> 0.7) suggest from the perspective of experts that some reagents may not belong conceptually to the dimensions of the test as in the case of items 9, 10, 11 and 16 (demographic profile); 23 and 36 (sexual behavior); and 17 and 29 (knowledge of prevention).

Evidence of clarity and wording of the items of the vulnerability to HPV questionnaire

The results of the concordance rate regarding the wording and clarity of the items between experts (see table 2) were adequate for the questionnaire in general (0.88) and for each one of the dimensions (from 0.81 to 0.96). According to Lynn⁽³²⁾, a rate of 0.80 or more indicates agreement between the judges as for dimension of the questionnaire (p<0,05).

 

 

Thirty-one items showed an adequate concordance rate >0,80. The items 14, 17, 18, 20 and 21 presented a concordance rate <0,80, so they require modifications to improve its wording and clarity (see table 3). The judges did not make any comment in relation to the design and the form of register of the answers.

 

 

Discussion

This study explored the conceptual structure of the questionnaire on knowledge and willingness to act preventively against Human Papillomavirus (HPV), in adolescents and young adults, using Concept Mapping methodology.^(34-36).

The evidence in this regard through the multidimensional scaling analysis, the hierarchical cluster analysis and the statistical Bridging suggests that the instrument presents four dimensions: socio-demographic profile, sexual behavior, knowledge of transmission and knowledge of prevention^{(34, 36)}.

Moreover, evidence obtained by Lynn's concordance rate⁽³²⁾, regarding wording and clarity of the items from the experts' perspective suggests that the majority of the items do not present problems of wording; however, the items 14, 17, 18, 20 and 21 require some modifications in certain words, so these items will be revised in the construction of the new version of the questionnaire.

Besides, according to the estimated of the statistical Bridging (>7) it is not clearly determined the relevance of the dimension of the items 9, 10, 11 and 16, 17, 23, 29 and 36 which are presented in the current questionnaire. Consequently, the location of these items will be also reviewed to the new version of the questionnaire⁽³⁶⁾.

Evidence obtained with respect to the content validity of the questionnaire suggests that its structure is relevant for measuring vulnerability factors regarding HPV in undergraduate students^{(27, 30)}, although the suggested agreements must be done prior to using the questionnaire in the study field.

 

Conclusion

The social and the epidemiological relevance of having an instrument as the one designed in this study is undeniable to make diagnosis related to knowledge and risk perception around HPV infection between adolescents and young adults, as a baseline for conducting later interventions able to report the coordination between the individual, social and programmatic scope that optics vulnerability requires.

 

References

1. Castellsagué X. Natural history and epidemiology of HPV infection and cervical cancer. Gynecologic Oncology. 2008;110:S4-7.         [ Links ]

2. Bosch FX, Burchell AN, Schiffman M, Giuliano AR, de Sanjose S, Bruni L, et al. Epidemiology and natural history of human papillomavirus infections and type-specific implications in cervical neoplasia. Vaccine. 2008;26 (Supl 10):K1-16.         [ Links ]

3. León G, Bosques D. Infección por el virus del papiloma humano y factores Relacionados con la actividad sexual en la génesis del cáncer de cuello uterino. Revista Cubana de Obstet y Ginecología. 2005;31(1):0-0.         [ Links ]

4. Hidalgo MI, Catellano G. Virus del papiloma humano y adolescencia. Bol. Pediatr. 2007;47:213-218.         [ Links ]

5. López A, Lizano M. Cáncer cérvico uterino y el virus del papiloma humano: La historia que no termina. Cancerologia. 2006;1:31-55.         [ Links ]

6. Burk RD, Chen Z, Van Doorslaer K. Human Papillomaviruses: Genetic Basis of Carcinogenicity. Public Health Genomics. 2009;12:281-290.         [ Links ]

7. Unger E, Duarte-Franco E. Human papillomaviruses. Into the new millennium. Obstet Gynecol Clin North Am. 2001;28(4):653-66.         [ Links ]

8. Oviedo G, Arpaia AL, Ratia E, Seco N, Rodríguez I, Ramírez Z. Factores de riesgo en mujeres con infección del virus papiloma humano. Rev chil obstet. Ginecol. 2004; 69(5):343-346.         [ Links ]

9. Da Rosa MI, Medeiros LR, Rosa DD, Bozzeti MC, Rosa F, Rosa B. Human papillomavirus and cervical neoplasia. Cad. Saúde Pública. 2009;25(5):953-964.         [ Links ]

10. Munoz N, Reina JC, Sanchez GI. The human papillomavirus vaccine: A powerful tool for the primary prevention of cervical cancer. Colombia Médica. 2008;39(2):196-204.         [ Links ]

11. Lambert EC. College students' knowledge of human papillomavirus and effectiveness of a brief educational intervention. J Am Board Fam Med. 2001; 14(3):178-183.         [ Links ]

12. McNair R, Power J. Comparing knowledge and perceived risk related to the human papilloma virus among Australian women of diverse sexual orientations. Aust N Z Public Health. 2009;33:87-93.         [ Links ]

13. Tristram A. HPV information needs. Best practice & research clinical obstetrics and gynecology. 2006;20(2):267-277.         [ Links ]

14. Stark A, Gregoire L, Pilarski R, Zarbo A, Gaba A, Lancaster WD. Human papillomavirus, cervical cancer and women's knowledge. Cancer Detect Prev. 2008; 32(1):15-22.         [ Links ]

15. Economist Intelillence Unit. Breakaway: The global burden of cancer-challenges and opportunities. The Economist, London: Livestrong; (2009).         [ Links ]

16. Moreno V, Bosch FX, Muñoz N, Meijer CJ, Shah KV, Walboomers JM, et al. Effect of oral contraceptives on risk of cervical cancer in women with human papillomavirus infection: the IARC multicentric case-control study. Lancet. 2002;359(9312):1085-1092.         [ Links ]

17. Sandfort JS, Pleasant A. Knowledge, attitudes, and informational behaviors of college students in regard to the human papillomavirus. Journal of American College Health. 2009;58(2):141-149.         [ Links ]

18. Giuseppe G, Abbate R, Liguori G, Albano L, Angelillo IF. Human papillomavirus and vaccination: knowledge, attitudes, and behavioural intention in adolescents and young women in Italy. British Journal of Cancer. 2008;99:225-229. Br J Cancer. 2008;99(2):225-229.         [ Links ]

19. Denny-Smith T, Bairan A, Page MC. A survey of female nursing students' knowledge, health beliefs, perceptions of risk, and risk behaviors regarding human papillomavirus and cervical cancer. Journal of the American Academy of Nurse Practitioners. 2006;18(2):62-69.         [ Links ]

20. Dell DL, Chen H, Ahmad F, Stewart DE. Knowledge About Human Papillomavirus Among Adolescents. Obstet Gynecol. 2000;96(5):653-656.         [ Links ]

21. Waller J, McCaffery K, Forrest S, Szarewski A, Cadman L, Wardle J. Awareness of human papillomavirus among women attending a well woman clinic. Sex Transm Infect. 2003;79(4):320-322.         [ Links ]

22. Simas F, Yasuko L, Vilela A. Conhecimento, Atitude e Práticas na Prevenção do Câncer de Colo Uterino e HPV em Adolescentes. Esc Anna Nery Rev Enferm. 2010;14(1):126-34.         [ Links ]

23. Pelucchi C, Esposito S, Galeone C, Semino M, Sabatini C, Picciolli I, et al. Knowledge of human papillomavirus infection and its prevention among adolescents and parents in the greater Milan area, Northern Italy. BMC Public Health. 2010;10(378):1-12.         [ Links ]

24. Mcpartland T, Weaver B, Lee S, Koutsky L. Men's Perceptions and Knowledge of Human Papillomavirus (HPV) Infection and Cervical Cancer. Journal of American College Health. 2005;53(5):225-230.         [ Links ]

25. Moreira ED, Oliveira BG, Ferraz FM, Costa S, Costa Filho JO, Karic G. Knowledge and attitudes about human papillomavirus, Pap smears, and cervical cancer among young women in Brazil: implications for health education and prevention. Int J Gynecol Cancer. 2006;16(2):599-603.         [ Links ]

26. Moscicki AB. Impact of HPV infection in adolescent populations. Journal of Adolescent Health. 2005;37(Supl 6):S3-S9.         [ Links ]

27. Mann J, Tarantola DJM, Netter TW, organizadores. A AIDS no mundo. Rio de Janeiro: Relume Dumará: ABIA, IMS, UERJ; 1993.         [ Links ]

28. Ayres JR, França J, Calazans GJ, Saletti F. Vulnerabilidade e prevenção em tempos de AIDS. In: Barbosa MR, Parker R, editors. Sexualidade pelo avesso: Direitos, identidades e poder. Rio de Janeiro: IMS/UERJ; São Paulo Ed. 34; 1999. Pp. 49-72.         [ Links ]

29. Ayres JRCM, França J, Calazans G, Saletti F, Heraldo C. O conceito de vulnerabilidade e as práticas de saúde: novas perspectivas e desafios. In: Czeresnia D, Freitas CM. Promoção da saúde: conceitos, reflexões, tendências. Rio de Janeiro: FIOCRUZ; 2003.         [ Links ]

30. Ayres JRCM, Paiva V, Franca I, Gravato N, Lacerda R, Negra MD, et al. Vulnerability, human rights, and comprehensive health care needs of young people living with HIV/AIDS. Am J Public Health. 2006;96(6):1001-6.         [ Links ]

31. Ministerio de la Protección Social, Fondo de Población de las Naciones Unidas (UNFPA), Universidad Nacional de Colombia. Factores de vulnerabilidad a la infección por VIH en mujeres, 2008. Bogotá (Colombia).         [ Links ]

32. Lynn MR. Determination and quantification of content validity. Nursing Research. 1986;35:382-385.         [ Links ]

33. Haynes SN, Richard DC, Kubany ES. Content Validity in Psychological Assessment: A Functional Approach to Concepts and Method. Psychological Assessment. 1995; 7(3):238-24.         [ Links ]

34. Carpenter BD, Van Haitsma K, Ruckdeschel K, Lawton MP. The psychosocial preferences of older adults: A pilot examination of content and structure. The Gerontologist. 2000;40(3):335-348.         [ Links ]

35. Trochim W. An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning. 1989;12(1):1-16.         [ Links ]

36. Bedi RP. Concept Mapping the Client's Perspective on Counseling Alliance Formation. Journal of Counseling Psychology. 2006;53(1):26-35.         [ Links ]

37. Kane M, Trochim W. Concept Mapping for planning and evaluation. Thousand Oaks, California: Sage;2007.         [ Links ]