INTRODUCTION

Adolescence comprises the period between 10 and 19 years, marked as the transition from childhood to adulthood and characterized by changes related to culture and physical, mental, emotional, and social changes ¹. Faced with these changes, adolescents tend to adopt risky behaviors, such as sexual behaviors that contribute to unplanned pregnancies and sexually transmitted infections (STIs) ².

Among the STIs, syphilis continues to cause morbidity and mortality worldwide. Although this pathology is easily identifiable and treaTable, syphilis infection rates continue to increase among vulnerable populations in high-income countries and remain at endemic levels in low- and middle-income settings ³.

Based on prevalence data from 2009 to 2016, a total of 6.3 million cases of syphilis were estimated. The estimated global prevalence of syphilis in men and women is 0.5%, with regional values ranging from 0.1 in the European region to 1.6% in the African region ⁴. Thus, the number of cases of syphilis infection is worrisome and raises the need to develop interventions that can control and prevent it. In addition, cases of syphilis in the age group of 13 to 19 years have represented considerable growth in recent years ⁵.

The global STI strategy sets the goal of reducing the incidence of syphilis by 90% and reducing the incidence of congenital syphilis to <50 cases per 100,000 live births by 2030 ⁶. To this end, technological innovations are relevant and can boost overcoming barriers, bringing the response to STIs closer. Furthermore, innovative methods are needed to address social and structural disease determinants to ensure that prevention efforts do not stall ⁷.

Using technologies and applications for mobile platforms effectively modifies and provides opportunities for the health promotion of different groups, including adolescents. In a global scenario, the proportion of people between 15 and 24 engaged with the internet is estimated to be over 70% worldwide, compared to just 48% of the total population ⁸. Therefore, using technological devices to promote youth discourses about sexuality brings a new discursive practice into everyday life ⁹.

Mobile applications are an adequate, accessible, and interesting way to educate adolescents about their sexual and reproductive health and encourage their autonomy in sexual health practices, making them protagonists in preventing health problems and promoting and maintaining health ¹⁰. In this way, the development and validation of mobile applications provide users with autonomy and present themselves as a new possibility to acquire knowledge, being, therefore, stimulated and disseminated.

In addition, the important role that nursing professionals play in health education in the most diversified care services is highlighted, considering that this is a category with a large number of professionals and that educational practices are innate to professional practice ¹¹. In this context, validation studies of nursing technologies for the prevention and control of syphilis are important as they improve these low-cost tools that can be implemented as health care interventions.

Given the above, considering that mobile applications are powerful health interventions ¹⁰, in addition to the increasing use of these devices by adolescents and their vulnerability related to sexual health, the interest in validating a mobile application for disease prevention and control is justified, with a favorable impact on the promotion of knowledge, attitudes and adequate practices of adolescents concerning syphilis. Thus, the study's objective was to validate the appearance and content of an application for the prevention and control of syphilis in adolescents.

MATERIAL AND METHOD

This is a methodological study, defined as the one aiming to investigate methods for obtaining and organizing data and developing, validating, and evaluating research tools and methods ¹². This study was conducted from January to June 2020 online.

The mobile app, called "Syphilis? I'm out!" was previously developed by the research team and published in another article ¹³. The application lists the following themes: definition of syphilis, stages of the disease, forms of transmission, forms of prevention, treatment, and specific guidelines for adolescents. The mobile application is divided into two parts: the first is similar to a booklet that adolescents can access to get information about Syphilis, and the second is a quiz game ¹³.

Nurses (judges), researchers, or professors with experience in STIs with an emphasis on adolescents' health promotion, educational technologies, or instrument validation participated in the validation stage. The search for judges was carried out on the Lattes Platform, hosted on the website of the Brazilian National Council for Scientific and Technological Development (CNPq), and on the Theses and Dissertation Database of the National Coordination for the Improvement of Higher Education Personnel (CAPES) using the following keywords: "syphilis", "sexually transmitted infections", and "adolescence". Snowball sampling was also used, given the impossibility of estimating the quantity of the target population ¹⁴.

Nurses who met two of the following criteria were included as content judges: having a master's or doctoral degree, having published a dissertation, thesis, or article on the theme of sexually transmitted infections in adolescents, and having prior clinical or teaching experience in the themes of "sexual and reproductive health" or "adolescent health" for at least one year, and expertise in sexual or reproductive health ¹⁵.

The selected judges were contacted via e-mail and received an invitation letter explaining the research objectives and procedures and requesting a response about the judge's interest in participating in the research. Potential participants interested in participating received the following materials through the Google Forms platform: the informed consent form (ICF), the evaluation instrument, a link to access the application's screenshots in PDF format, and a link to a video simulating the use of the app for usability assessment. A period of 15 days was granted to return the assessment instruments. At the time, the suggestion of other professionals who met the inclusion criteria was also requested.

As for the ideal number of judges for the validation process, we used the recommendation of 22 expert judges ¹⁶. Thus, the first 22 content judges who agreed to participate in the research and responded to the validation instrument made up the sample.

The content and appearance validation process was carried out with the participation of judges, using an instrument with variables related to the judge's characterization and variables related to the objectives, content, structure, functionality, and relevance of the application ¹⁷. The variables were measured using a Likert scale with scores from one to four.

The collected data were organized in a Microsoft Office Excel 2016® spreadsheet and analyzed in the Statistical Package for the Social Sciences® version 20.0. Central tendency and dispersion measures were used to characterize the sample of judges. The Content Validity Index (CVI) was used for each item, domain, and the technology as a whole ¹⁸. CVI values were adopted to measure the quality of each aspect or item, considering valid values equal to or greater than 0.80 ^12,18. Then, the exact binomial distribution test was used, being considered significant when p <0.05. Furthermore, Cronbach's Alpha coefficient test was applied individually and by grouping items relevant to each question, with a minimum accepTable value of 0.7 ¹⁹.

Subsequently, a descriptive evaluation of the judges' suggestions was conducted, and the team met to decide whether they would be accepted or rejected. After making the relevant adjustments, the Flesch Readability Index (FRI) was applied using Microsoft Office Word 2016. In this study, an adapted version of the FRI questionnaire was used. The instrument stratifies the rating into four levels of reading difficulty: very easy (75 - 100 points), easy (50 - 75 points), difficult (25 - 50 points), and very difficult (0 - 25 points) ²⁰.

This study was reviewed and approved by the Research Ethics Committee of the University of International Integration of Afro-Brazilian Lusophony under opinion n° 3,805,400 and CAAE n^o 23697219.8.0000.5576.

RESULTS

Eighty nurse judges from all regions of Brazil were invited to participate in the content and appearance validation stage. Of these, 35 agreed to participate in the study, but only 22 responded to the formal request sent via e-mail. Table 1 presents the profile of the participating judges.

Table 1. Characterization of content and appearance judges. 

The age ranged between 27 and 55 years, with a mean of 35.7 years. Most participants were female (n=18; 81.8%). The professional experience ranged from 2 to 39 years, with an average of 11.4 years. Regarding the occupations, the majority (n=17; 77.3%) worked in teaching jobs and were doctors (n=14; 63.6%). Many judges (n=16; 72.7%) had educational experience in syphilis control and publications on sexually transmitted infections.

Table 2 presents CVI values, the binomial test results, and Cronbach's alpha coefficients for each domain.

Table 2. Distribution of content validity index (CVI) values, binomial test results, and Cronbach's alpha coefficients. 

The judges positively evaluated domain 1 - objectives and content - since a CVI of 0.88 was obtained. The binomial test was satisfactory for all items in this domain, obtaining statistical significance. The Cronbach's Alpha of the domain was 0.848, which suggests a good internal consistency between the judges.

Concerning domain 2 - structure and functionality - we obtained a CVI of 0.86, above the accepTable cutoff. However, the items referring to the structuring in agreement and spelling of the information and the expression of the illustrations obtained a score lower than 0.80. The binomial test for this domain obtained statistical significance in most topics. However, the structuring agreement and spelling of the information and the expression of the illustrations did not obtain statistical significance. The Cronbach's alpha of the domain was 0.881, which suggests a good internal consistency between the judges. It should be noted that the screens were modified in terms of spelling and the inclusion of more representative images, as suggested by some judges.

As for domain 3 - relevance - all screens were validated with CVI 0.86. However, the item on suitability for use by any adolescent also scored greater than 0.8. The binomial test achieved statistical significance in all items with a p<0.05. The Cronbach's alpha of 0.877 also indicates good internal consistency. The judges' suggestions were analyzed and accepted to improve the final version of the application.

In the three dimensions, the average of the CVI was above 0.80, which validates the contents of the application. The global CVI of the application was 0.86, which is satisfactory and makes it possible to consider the application valid in terms of content and appearance. Concerning the total Cronbach's alpha of the application, a value of 0.94 was obtained, demonstrating an excellent homogeneity between the judges' responses.

After the quantitative evaluation with the calculation of the CVI, binomial test, and Cronbach's alpha values, a qualitative evaluation was carried out, with all the expert judges' suggestions, criticisms, and praises being analyzed. The judges suggested 112 modifications, 79 of which were accepted, aiming to improve the application, and 33 were not.

Among the rejected suggestions is the inclusion of the possibility of oral treatment, improvements on the description of the route of administration of the treatment with the addition of the term "intramuscular", the inclusion of the name of the drug used for the treatment, the inclusion of themes such as sexual rights of adolescents, an explanation of how to use male and female condoms, the change of the term "symptoms" to the term "sign" - since the sign is the clinical manifestation perceived by another person -, and of the term "exam" to the term "test" - considered less invasive.

Suggestions for changes in the application were also made by the judges, such as: adding captions to the illustrations (n=1), simplifying and re-elaborating sentences to make the language clearer (n=4), and adding some information they deemed necessary (n=3), among others.

The following suggestions given by the judges were accepted: strengthening the treatment of sexual partners, reinforcing breastfeeding even in case of reactive syphilis, inserting topics on the forms of transmission, removing the statement about local pain as an adverse reaction to the treatment, and emphasizing the importance of prenatal testing.

Regarding the application's readability assessment through the Flesch Readability Index, a value of 51.3 was obtained, classifying the material as easy to read.

DISCUSSION

The adoption of digital technology promises many potential benefits for the healthcare system, including greater efficiency in healthcare, cost reduction, and improved healthcare system governance structures, thus extending healthcare delivery beyond its conventional limits ²¹. The literature points out some examples of technologies developed for adolescents, addressing issues of contraception ²², Human Immunodeficiency Virus ^23-24, and reduction of risky behaviors ²⁵.

It is noteworthy that, in addition to developing technologies, it is essential to ensure the quality of information. Given this, the method of validating an educational technology is based on the perspective that this step is essential to ensure the authenticity, validity, and credibility of the artifact before disseminating it to the target audience ²⁶. In this way, validating the "Syphilis? I'm out!" application was essential to assess the ideas and clarity of each technology component so that it is extensible to the target population.

According to the judges' evaluation, the "Syphilis? I'm out!" illustrations needed improvement to become more expressive. It is understood that reading and understanding health information is essential for individuals' decision-making and health behaviors. Therefore, non-verbal communication has been used as a resource for health education. This communication tool should be explored in the elaboration and validation of technologies, as incorporating this resource contributes to overcoming health inequities, poverty and intervening in public health problems ²⁷.

Some judges suggested simplifying sentences, making them easier to be understood by the adolescent audience. The potential of educational technologies in health can be potentiated if researchers acknowledge the importance of using an accessible and easy-to-understand language for potential users. A prior study that built and validated an application, with the participation of 23 specialists and ten caregivers of children, showed that the creation of educational materials requires an understanding of the scenario of the target population, combined with a participatory, communicative, and collective approach, leading to effective care plans ²⁸.

An example of suitability adjustment considering the target audience is the change of the term "catheter" to "probe", as the latter is more used and well-known ²⁸. Likewise, in our study, we kept the most popular words, such as "exam" instead of "test" and "symptoms" instead of "signs". We also declined the suggestion of adopting the term "intramuscular" as a Syphillis treatment administration route for being a technical term, not easily understood by adolescents.

The inclusion of information about the oral treatment option was suggested. This proposal was rejected since benzathine benzylpenicillin, administered intramuscularly, is the drug of choice for the treatment of syphilis, being the only drug with documented efficacy during pregnancy, and there is no evidence of resistance of T. pallidum to penicillin in Brazil and the world ²⁹.

The app evaluated in this study was considered valid regarding its content and appearance. Indeed, developing and validating applications for syphilis is an important strategy to mitigate this public health problem affecting thousands of people worldwide. However, for these technologies to be used efficiently and properly, it is necessary to encourage the development of skills and abilities in health professionals to handle these tools with the same intensity as disseminating information in a globalized world ²⁷.

There are many difficulties in coping with syphilis, especially concerning cooperation and adherence of health professionals and civil society to organize a support network for the establishment of educational and preventive actions that may encompass different audiences (pregnant women, sexual partners, and others) regarding the risks of infection and its complications, as well as the lack of knowledge about the magnitude of this problem and the repercussions it can cause ³⁰. This way, the existence of a valid application with content that is easy to understand is a means of disseminating information about syphilis, contributing to the construction of knowledge and bringing adolescents closer to healthcare settings.

In this way, the application validated in our study is a resource that can be implemented in care practice to prevent syphilis. It is believed that its use can be disseminated within the scope of the Brazilian Family Health Strategy by nurses, community health agents, and other professionals who assist adolescents, aiming to bring this public closer to health services.

Among the study's limitations, the non-participation of information technology and communication professionals stand out since these professionals participated in another validation stage. In addition, there is a lack of semantic validation with the target audience. It is suggested that new studies approach other types of validation multi-professionally.

CONCLUSION

The mobile application was evaluated as valid in content and appearance, demonstrating its reliability as a technology that adolescents can use to obtain health education to prevent and control syphilis. Finally, the readability test proved that the application's content is easy to understand.