記住我
The World Health Organization (WHO) defines mobile health (mHealth) as the application of wireless technology, such as mobile phones, to the provision of health care. It is regarded as a component of eHealth, which also refers to the secure and cost-effective use of information and communication technology to assist with health system and disciplines related with health []. In fact, the WHO itself has recognized that eHealth has the potential to significantly contribute to sustainability and accessibility in the health system. In this regard, the number of mHealth interventions worldwide has been steadily increasing over the last decade.
This is even more relevant in regions such as Africa where the health challenges are much greater. In 2022, a systematic review of digital health interventions in sub-Saharan Africa identified 738 digital health interventions, highlighting multiple overlapping solutions with limited focus and scalability []. The WHO is therefore working to address these challenges, and already in 2010, it published a continent-specific digital health strategy [].
Among the diseases that seem likely to benefit from such interventions are the neglected tropical diseases (NTDs) [], a group of 20 diseases and conditions identified by the WHO. As their name implies, NTDs are diseases that have not historically received any kind of priority attention from international health organizations, despite affecting more than 1.5 billion people []. This is directly related to the fact that most of these illnesses are only found in low- and middle-income countries (LMICs) in tropical and subtropical regions, and they primarily affect women and children [].
If undetected or untreated, some of these diseases can either be fatal or become chronic and irreversible, leading not only to lifelong disabilities but also compromised mental well-being; stigmatization; social exclusion; and, in certain countries, even experiences of racism [-]. This in turn perpetuates a cycle of poverty that has a direct impact on the development and economic productivity of LMICs [].
In 2005, the WHO finally decided that a comprehensive strategy was needed to address the complexity surrounding these diseases. It therefore shifted from tackling them individually to tackling them as a single group, the NTDs []. It later decided to create subgroups depending on their management: those that are potentially preventable through large-scale chemotherapy interventions and those that require individual case management []. Within the latter category lie skin NTDs, a group of NTDs that manifest primarily as skin lesions, such as edema, patches, and ulceration, which can be detected through visual screening []. This group consists of Buruli ulcer, cutaneous leishmaniasis, deep fungal infections, post–kala-azar dermal leishmaniasis, leprosy, lymphatic filariasis, mycetoma, onchocerciasis, scabies and other ectoparasites, and yaws. The management of these diseases hinges on early detection and treatment, which demands significant resources, including skilled personnel and financial support [,]. However, early detection of these diseases is often difficult due to various factors. For instance, they are frequently painless, which can prevent them from seeking medical attention at an early stage. In addition, there is low awareness of these diseases among the population at highest risk, and, as mentioned before, stigma and discrimination associated with these diseases can also discourage people from seeking medical help [,].
Therefore, a key player that emerges in the strategic framework for successfully managing skin NTDs are the frontline health workers. This term refers to any health worker who directly provides service to a community. Although they frequently lack specialized medical training, knowledge of data collection techniques, and peer contact, they are real key players [,]. Indeed, they often are the initial point of contact for disease control for most skin NTDs, given that they can be visually identified and that clinical diagnosis is the most accessible diagnostic tool available []. Hence, it is important to enhance their education and training to facilitate their role in diagnosing, in treating, in referring patients to another level of the health system [], and even in mitigating the stigma associated with skin NTDs []. In order to achieve this last-mentioned goal, numerous training initiatives for frontline health workers have been undertaken to date with good results and have been well received by them [,-]. However, the predominance of in-person small group formats, with the program often targeting only 1 condition, limits the reach and increases the expense of these initiatives. Identifying alternatives to enhance the efficiency and expandability of these programs is crucial []. mHealth stands out as a promising, practical, and extendable approach to support frontline health worker training []. Although its adoption for this specific purpose has been limited, reviews of existing literature have highlighted its potential effectiveness []. Furthermore, specifically for skin NTDs, the WHO promotes the use of teledermatology and web-based training courses and materials whenever possible. That is the main reason why the WHO’s Department of Control of Neglected Tropical Diseases has developed the SkinNTDs app [], a mobile version of the training guide they published in 2018 []. By using an algorithm based on identifying signs and symptoms and providing more information about these diseases, this app assists frontline health workers in the diagnosis and management of skin NTDs.
However, although the third version has already been released, the app’s usability, efficacy, and effectiveness have not yet been evaluated. This is a critical step in guaranteeing quality and empowering end users not to rely entirely on popularity or “star”-rating systems, which have already been proved insufficient []. Evaluation of mHealth interventions is a key component of most digital health technology frameworks [-], even more so when app-based mHealth interventions are scarce in Africa, as most mHealth interventions in this region are SMS text messaging based [].
The intervention maturity life cycle schematic proposed by WHO [] is a practical guide explaining the goals at each stage, the number of participants required, and the measurement targets. Considering these guidelines and the stage of maturity of the SkinNTDs app (the first and second stage), it now seems appropriate to evaluate its feasibility and usability. In this stage, questions regarding how the app is used by end users, how it fits into their workflow, and how easy the learning curve of use is should be answered before moving on to the next stage (efficacy).
One reliable tool available to assess usability is the user version of the Mobile Application Rating Scale (uMARS) [], a simple tool for classifying and rating mHealth apps based on 4 objective subdomains (engagement, functionality, aesthetics, and information quality) and 1 subjective quality subdomain.
ObjectiveGiven the WHO’s commitment to mHealth as a tool to achieve a range of sustainable goals, it is essential to evaluate interventions that have the potential to be highly scalable and cost-effective. This paper summarizes the results of a cross-sectional study assessing the engagement, functionality, aesthetics, and information quality of version 3 of the SkinNTDs app for the real end user in their actual context according to a validated tool. In addition, a secondary objective was to check whether the demographic information gathered influenced the final uMARS score, given that the developers of the SkinNTD app had no plans to customize the app for different settings upon its implementation.
A cross-sectional study was conducted between December 2022 and April 2023 in Ghana and Kenya. The study design and methods are described in detail in the published protocol []. Deviations from this protocol are summarized in .
Ethical ConsiderationsThis study was conducted according to the ethical principles established by the World Medical Association in the Declaration of Helsinki of Ethical Principles for Medical Research Involving Human Subjects []. To guarantee that the protocol complied with the ethical standards of all 3 countries involved, it was approved by their respective ethics committees. The protocol was authorized by the Ethics Committee of Universitat Oberta de Catalunya (Spain) (20201127_mcarrion_NTDs), the Ethical Committee of Kwame Nkrumah University of Science and Technology (Ghana) (CHRPE/AP/576/22), and the Ethical Committee of Coast General Teaching and Referral Hospital (Kenya) (ERC-CGH/MSc/VOL.I).
Participants gave informed consent to participate in the study before taking part. To ensure confidentiality and anonymity, each participant was assigned a unique identification number (eg, P01) for coding all collected information and data. Moreover, participants did not receive any economic compensation for their participation in this study.
Participant Recruitment and Eligibility CriteriaNonprobabilistic snowball sampling was used due to the difficulty in making direct contact with frontline health care workers in these 2 countries. Snowball sampling is a recruitment technique in which chosen participants are asked to find and contact other potential participants from among their acquaintances. Although this approach is nonrandomized, it appeared to be the best strategy to locate the participants of this study, as the main researchers were based on a different continent.
A diagram representation of this snowball sampling recruitment is shown in . More detailed information can be found in the published protocol [].
Anyone who worked or had worked with skin NTDs on a regular basis in the 2 selected countries, who were or had been in charge of their diagnosis and treatment, who had a smartphone (Apple or Android), and who had downloaded and used the SkinNTDs app on at least 5 different days was eligible to take part. In addition, participants had to use WhatsApp (Meta Platforms) or email to send the informed consent. Limited understanding of the English language and refusal to sign informed consent form were factors for exclusion.
The same population was the target for both parts of the study. Before signing the informed consent form, participants were asked to read the information page.
Figure 1. Diagrammatic representation of snowball sampling recruitment. MARS: Mobile Application Rating Scale; WHO: World Health Organization. Sample Size CalculationAs already mentioned, this study followed the steps proposed in the intervention maturity life cycle schematic described by the WHO in Monitoring and Evaluating Digital Health Interventions []. As a result, sample size was determined using this guide.
The SkinNTDs app is now at the prototype stage, which, in accordance with WHO recommendations, corresponds to stages 1 and 2. The WHO suggests evaluating the intervention with a sample size between 10 and 100 people. Given that a sample of 100 participants is the maximum number advised and that this study was conducted on the web, we assumed a 50% dropout rate. These calculations resulted in a 50-person final sample size.
OutcomesDemographic VariablesParticipants were asked to complete an anonymous survey of various demographic data that included age, sex, country of residence, type of frontline health care worker, frequency of dealing with skin NTDs, experience and training in dermatology, work environment, working institution, knowledge of mobile technology, and languages spoken.
Quantitative Method: uMARS QuestionnaireThe uMARS was used to assess the quality of the SkinNTDs app []. This tool assesses 20 items clustered into 4 objective subscales (engagement, functionality, aesthetics, and information quality) and 1 subjective subscale. Participants rated each item using a Likert scale from 1 to 5; higher numbers meant a better rating. A “not applicable” option was available in case an item could not be assessed. In the extra “app-specific” category, only the relevant questions for this study were chosen. Finally, 9 self-created questions were added to complete the survey.
A total of 2 scores were obtained based on the original recommendation from the authors of this scale: app quality score and app subjective quality score. Questions rated as not applicable were excluded from the score.
In our study, we adapted the evaluation process based on the original article’s recommendations, which suggested a minimum application use time of 10 minutes before completing the uMARS. However, we considered that this time was not enough to complete a comprehensive evaluation, and we extended to 5 days.
Qualitative Methods: Semistructured Interviews and Focus GroupsDue to the different realities of the context of the 2 countries, 2 qualitative methods were used.
Semistructured interviews were conducted as a qualitative method for exploring the perspectives, perceptions, and opinions of participants, combining prepared questions with others that arose during the interview []. Each participant was asked 7 questions based on the key findings obtained from the uMARS questionnaire in the same order and in the same words as a standardization that facilitated comparison []. In addition, interviewers could ask unforeseen questions based on a participant’s answer. The questions were unbiased, open-ended, and well-written and used simple terms.
Focus groups were also used as another qualitative research method for the same purpose as before. This method involves a moderator posing targeted questions to participants in a group setting, with the advantage of capturing valuable interactions among them []. Focus group sessions also incorporated the same set of 7 questions used in the semistructured interviews. Moreover, the moderator had the same flexibility to ask unforeseen questions based on the participants’ interactions and responses.
Our study design incorporated a predetermined objective to include a minimum of 10% of the overall sample size or continue data collection until reaching a point of information saturation using both methods.
More details of the instruments can be found in the study protocol [].
Data Collection and Study ProcedureThe principal researcher delivered the consent form via email or WhatsApp when frontline health care workers contacted her. However, participants enrolled at Coast General Teaching and Referral Hospital (Kenya), due to ethical requirements, signed a paper-based consent form delivered to them by the local investigator. After it was signed, participants received a follow-up email with more details about the study and a link to the uMARS survey, which they had to answer after using the app for at least 5 days.
Participants were able to express their willingness to take part in the semistructured interviews in the same questionnaire.
After reading the detailed information sheet for this second phase of the study, participants who had agreed were required to sign a second consent form. Then, the interview was scheduled by mutual agreement.
The interviews were conducted via Google Meet (Google) and had a duration ranging from 25 to 40 minutes. The focus groups were conducted face-to-face and moderated by a local researcher for 60 minutes. All sessions were recorded regardless of the qualitative method used.
During this time, the interviewer and moderator could take notes and reiterate any questions or sentences for clarification.
Interviews and focus groups were all transcribed using Otter.ai software (Sam Liang) for analysis. The research protocol provides more information on the procedure for data collection [].
Statistical AnalysisAll quantitative data analyses were conducted using SPSS (version 25; IBM Corp) for Windows, with the statistical significance set at P≤.05. A descriptive analysis was conducted to describe the demographic variables and data obtained from the uMARS scores. Categorical variables were described in terms of frequencies and continuous variables in terms of central tendency and dispersion measurements. All data were shown in tables. Data normality was evaluated using Kolmogorov-Smirnov test (P=.05). The t test (2-tailed) was used for comparing means of bivariate variables and ANOVA for comparing means of multivariate variables. A logistic regression analysis was performed to add the covariates that could skew the main association under analysis. A CI of 95% was assumed, and P≤.05 was considered a significant difference.
Selected quotes were returned to participants for approval. Qualitative data derived from the semistructured interviews and focus groups were analyzed using ATLAS.ti (ATLAS.ti Scientific Software Development GmbH). We identified attributes, clustered them into different themes, and then coded these themes to analyze the various dimensions explored during the interviews.
In total, 60 participants took part in this study. Demographic characteristics are shown in .
Of the 60 participants, 57% (n=34) were male individuals. Participants were divided quite homogeneously in the different age groups, <36 years (n=22, 37%), 36 to 45 years (n=22, 37%), and 46 to 65 years (n=16, 27%). Participation was higher in Kenya with 36 (60%) participants, compared with the 24 (40%) from Ghana. Up to 93% (n=56) of them referred to working in a public health care institution. From all these participants, 72% (n=43) identified themselves as frontline health care workers, and 62% (n=37) mentioned working in a rural area. The vast majority of the participants were not dermatology specialists but said they had some experience with the topic (n=44, 73%). The participants were distributed similarly in terms of how frequently they dealt with skin. More than half of the participants (n=41, 68%) considered they had extensive knowledge of mobile technology, and 87% (n=52) had no additional training before using the app. Speaking English was a requirement for participation, so everyone who participated did. However, there were up to 19 more languages registered, among which Swahili stood out as the second most spoken language by the participants in Kenya (25/36, 69%) and Twi in Ghana (20/24, 83%).
Table 1. Participant demographics in the World Health Organization’s SkinNTDs app usability and user experience assessment through the user version of the Mobile Application Rating Scale survey (n=60) shown by frequencies (%).VariableParticipants, n (%)Age (y)and show the overall results from the 3 sections of the uMARS questionnaire and specific results for each subdomain. They appear separately according to the recommendations of the uMARS authors, in order to strengthen the objectivity of the final result.
The SkinNTDs app received an overall score of 4.02 (0.47) out of 5 in the app quality mean score, 3.82 (0.61) out of 5 in the subjective mean score, and 4.47 (0.56) out of 5 in the perceived impact section. Thus, the app received more than 75% of the maximum score in all 3 sections.
These overall scores can be broken down as follows: starting with the most objective scoring, the app quality mean score is the average obtained in each of the 4 uMARS domains (engagement, functionality, aesthetics, and information quality), which in turn are divided into 16 subdomains. The scores range from 3.65 to 4.20, with engagement being the lowest rated and information quality the highest rated. Moreover, SDs are between 0.47 and 0.65, indicating that the dispersion of the data tends to be medium-low. Notably, the lowest-rated subdomain was customization (mean 3.18, SD 1.02, out of 5), which refers to whether the app provides all necessary settings and preferences for app features (eg, sound, content, and notifications). In contrast, the credibility of the source subdomain was the highest rated of the 16 (mean 4.55 (SD) out of 5), referring to whether the app comes from a legitimate source.
At a subjective level, the app obtained a mean score of 3.82 (SD) out of 5. The findings in this section indicate that individuals are not very eager to pay for this app (mean 2.57, SD 1.33); however, the subvariable measuring whether they would recommend the app had a very high score (mean 4.65, SD 0.70). Finally, when the participants were asked for the overall rating, the average score was 4.08 (SD), which is extremely close to the quality mean score of 4.02 (SD). In this case, the SDs are slightly higher than in the previous section, indicating more variability in the responses.
Finally, the participants rated the perceived impact score (including awareness and knowledge) highly, with mean values of 4.40 (SD) and 4.55 (SD) out of 5 correspondingly.
shows more detailed information regarding the specific results for each subdomain. In addition, graph 1 presents the scores of all objective domains assessed in a visual way.
specifies the detailed results for each subdomain. In addition, shows the results obtained in each subdomain visually by country.
The study was completed with the comparison of the mean, which was based on the comparison between the global app quality mean score versus the main domains. The Kolmogorov-Smirnov test demonstrates a normal distribution of the data derived from the app quality mean score. Thus, a t test was used for comparing means of bivariate variables and ANOVA for comparing means of multivariate variables.
Table 2. Results of the validated user version of the Mobile Application Rating Scale domains according to the cross-sectional study.DomainMean (SD)MinimumMaximumApp quality mean score4.02 (0.47)2.894.83
Figure 2. Radar chart evaluation of each app’s objective quality domain according to the user version of the Mobile Application Rating Scale (uMARS), with results divided by country. The results () revealed that there was no statistically significant difference in any of the variables.
Similarly, linear regression showed that there was no significant association between the app quality mean score and any of the categorical variables examined above (F=0.60; P=.76), with R2=0.07.
The linear regression model analyzing the main demographic variables of the participants and their correlation with the user version of the Mobile Application Rating Scale (uMARS) app quality mean score in the cross-sectional study is as follows: The model shows an R value of 0.27, an R2 of 0.07, and an adjusted R2 of –0.05. The F test for this model (df=0.60) resulted in a P value of .76. The dependent variable for this analysis was the app quality mean score, and the predictors included age, sex, type of health worker, country, work context, frequency of dealing with skin NTDs, and dermatology experience. Meaning that there was no significant association between the app quality mean score and any of the categorical variables examined. Finally, as mentioned in the Methods section, 7 questions were added to complete the app evaluation, which are shown in .
It is worth noting that 64% (36/60) of participants used the app more than the minimum number of times required before answering the survey, and the majority believed that using the app would reduce the time it takes to diagnose skin NTDs. In terms of connectivity, while up to 67% (40/60) claimed to have a strong connection, a substantial minority group of 33% (20/60) still had limited or no access to the internet.
Almost all the participants favored the inclusion of skin NTDs surveillance, the ability to include patient records, and the availability of a computer version of the app.
Slightly more than half of the participants (32/60, 53%) considered it necessary for the app to be available in other languages, with Twi and Swahili being the first on the list.
Table 4. Comparison of means between user version of the Mobile Application Rating Scale app quality mean score versus the main demographic variables of participants in the cross-sectional study.Variables compared and statistical tests performedValueP valueApp quality mean score vs ageRegarding the second part of the study, 55% (33/60) of participants showed an interest in participating. However, only 13 men and 4 women were interviewed (5 from Ghana and 12 from Kenya). Upon interviewing 17 people, we observed that the responses started to repeat, indicating data saturation. The data collection comprised 10 semistructured interviews, with 5 participants from Ghana and 4 from Kenya, along with a single focus group conducted in Kenya involving 8 participants.
Most participants (10/17, 59%) were aged between 46 and 65 years. Approximately, 76% (13/17) of the participants were frontline health care workers, and 47% (8/17) said that they had occasional contact with skin NTDs. Demographic characteristics are shown in .
Despite being asked 7 questions, not all participants were able to provide detailed answers that allowed attributes to be identified. This was mainly due to technological and language barriers.
Both the semistructured interview and the focus group data were transcribed by 2 researchers (MC and PM) and thoroughly reviewed by 1 researcher (MC) to gain an initial understanding of the data. Subsequently, expressions aligning with the study objectives were extracted from the text and transformed into attributes (n=95) using ATLAS.ti software. These codes represented condensed versions of the professionals’ thoughts. On the basis of the content similarity, the codes were further grouped into themes (n=18), which were then associated with each question asked. At this stage, another researcher (CC) examined the coding, subcategories, and higher-level categories. Discussions were held to reach a consensus on the content and names assigned to categories.
presents the description and frequency of the identified themes and selected participant quotes.
Table 6. Participant demographics in the World Health Organization’s SkinNTDs app usability and user experience assessment through semistructured interviews and focus group (n=17).VariableParticipants, n (%)Age (y)
留言 (0)