3.2.2. Qualitative Feasibility StudyThe data collection took around 70 min on average (range 36–94). Processing the tool using think-aloud took around 31 min on average (range 10–60). Theoretical data saturation was assumed since no further new findings could be gained. By means of the qualitative content analysis, four main categories with subcategories were created: 1. Acceptance, 2. Applicability, 3. Comprehension and 4. Achievement of didactic objectives. Quotes for characteristic examples can be found in Table 2 and were labelled with “L” for laypersons and “P” for professionals and were linguistically adapted.

This category describes the satisfaction, the emotional reaction, the perceived relevance and the potential benefit, respectively, recommendation as a reaction to the methodological implementation of the tools.

Satisfaction: The participants reported that they were generally satisfied with the tool, which was mainly due to a perceived increase in knowledge or a revision of knowledge (Table 2 Q1). Dissatisfaction arose before the tool was revised due to the lack of in-depth options and the presentation of the calculation using the 2 × 2 table. Furthermore, two professionals were dissatisfied due to unfulfilled expectations. For example, one person expected a tool for calculating risk reductions from studies.Emotional reaction: The tool triggered both positive and negative emotions in the participants. Both laypeople and professionals found the mathematical content of some of the tool’s sectors to be overwhelming. This led in some cases to resignation and dispensing with assignments (Table 2 Q2). The participants named their own knowledge of mathematics as the reason for this. Nevertheless, particularly the simple introduction and repetition tasks had a motivating effect and led to an “aha effect” (Table 2 Q3).Relevance: The content and objectives of the tool were assessed as relevant by the participants due to the lack of awareness and confrontation with the issue so far, as well as the professional and everyday contact (Table 2 Q4). Some of the participants reported that the tool is relevant for personal decision-making; others, however, did not find the tool to be relevant for their own decision-making (Table 2 Q5). The participants described realistic practical applications and a low level of difficulty as factors that promoted relevance. On the other hand, some sections were considered to be less relevant if a mathematical scientific approach had been taken and the participants already thought they had understood the content.

Benefit/Recommendation: Some participants saw the application of the tool on the one hand in the training of people in the health sector, and as an offer for the general public as well as personal further training. Laypeople in particular, on the other hand, felt that the application was only useful for those interested and for a specific purpose.

2.

Applicability

This category contains statements about the practical use with regard to structure, handling and design of the tool and its contents.

Structure: It was possible to identify facilitating and hindering factors for applicability with regard to the structure. The participants found the recognizable structure of the tool and the combination of graphics and texts to be facilitating as well as the reduction in the content. Hindering factors were identified in the form of the unclear classification of texts and graphics and confusing menu displays. Due to the adaptation of the contents to the level of knowledge of laypeople, more in-depth contents, such as 2 × 2 tables or further technical terms, were presented as optionally accessible elements. This was not recognized in some cases. In addition, before the tool was revised the participants noted a greater need for opportunities to recapitulate (Table 2 Q6). Being able to choose which type of risk reduction one should begin with was not perceived by the participants as a relevant decision.Handling: Regarding the handling of the tool, the participants found using well-known interaction and navigation features such as “help” and “next” buttons to be beneficial. They also described the possibility to do calculations and corrections directly in the tool and showing the solutions as being very helpful (Table 2 Q7). Although the integrated calculator was generally seen as being positive, the fixed calculation path caused problems for participants who used a different formula or none at all. Before the revision, a major problem in the operation was the section on baseline risk (BR). The use of the interactive bar chart as well as the corresponding exercise only became clear to the participants when they received a hint.

Design: The participants considered the tool design and the form of individual components mostly as helpful for understanding the contents. The graphics were described mostly as being clear and comprehensible. By using well-known design elements, the contents could be processed easily. Furthermore, the design contributed for the most part to the transfer of knowledge. The use of animations was rated in different ways. These were partly described as facilitating the spatial imagination and imparting the contents, but some important components were not noticed and the contents thus not clear.

The design of the interaction option before revision took place led to the fact that some of the participants did not perceive symbols as interaction options, which meant that, for example, no help could be called up (Table 2 Q8).3.

Comprehensibility

The category consisted of various aspects of comprehending and understanding as a prerequisite for comprehension from the participants’ perspective, compared with the tool’s objectives. It also contains the subcategories of understanding the content, difficulty of the exercises, text difficulty and transferability.

Understanding the content: Taking the entire tool into consideration, the participants described a heightened awareness of the different ways in which risk reductions are presented (Table 2 Q9). Some of the professional participants described the contents as a deepening or repetition with no additional knowledge gain. Participants recognized that the absolute risk reduction was rather low; however, attitudes toward screening were generally very positive (Table 2 Q10). Moreover, some of the participants interpreted the contents presented before the tool was revised more as information on cancer screening than on risk reduction. The participants generally described exercises and the use of natural frequencies as conducive to understanding. Different levels of understanding of the individual sections of the tool were observed. Some of the participants said the individual sections were understandable, others said they were not. The introductory content was predominantly regarded as comprehensible, while increasing difficulties were observed with the mathematical content and calculations. Furthermore, the section on baseline risk (BS) was mostly not understood before revision took place.Difficulty of the exercises: The difficulty of the exercises and repetition tasks was described by the participants in very different ways; an increasing difficulty could be observed after the revision. The participants found the use of simple figures and the optional presentation of formulae to be useful, whereas missing graphics and unclear formulation and complexity of the exercises led to difficulties. Another factor for the difficulties were the calculations and the associated mathematical skills (Table 2 Q11).Text difficulties: The participants reported having difficulties with the text due to the complexity and quantity (Table 2 Q12). Laypeople also had problems with terminology such as “cardiovascular”, “event rate” and “intervention group”.Transferability: Both laypeople and professionals reported that they wanted to raise the awareness of the difference between relative and absolute risk reduction and their significance in everyday practice (Table 2 Q13). Some of the participants saw the retentiveness and the influence on the decision critically (Table 2 Q14).4.

Achievement of didactic objectives

This category contains descriptions that can be traced back to the didactics of the tool. In some cases, factors already described led to the fact that not all didactic objectives could be achieved to the required extent. For instance, the design of the interaction options was partly a hindrance to the self-determined adjustment of the difficulty. Nevertheless, the didactic approach and the motivational design could be identified as suitable. For example, the problem presented in the form of an everyday example led to the expected curiosity and motivation by pointing out gaps in knowledge (Table 2 Q15). 3.2.3. Revision and Finalization of the ToolThe revision of the tool was carried out on the basis of the qualitative feasibility study in an iterative process of piloting, analysis and revision. Altogether three revisions were performed: a minor revision after eight interviews with professionals, a major revision after four interviews with laypeople and at the end a second minor revision after interviews with six professionals and four laypeople. The revised and final concept of the tool can be seen in Supplementary File, Table S4. At the moment, the final version of the tool has not been published because we planned to evaluate it in a subsequent research projects, but it can, however, be accessed for scientific interest in German and English language under https://www.leitlinie-gesundheitsinformation.de/RiskTool/englisch/ (accessed on 1 September 2022).In a major revision, individual contents and the structure of the tool were adjusted. Whereas in the first version of the tool many decisions had to be made by the participants without knowing exactly what content was involved, the revised structure is both more stringent and transparent (Figure 3).The contents of the tool are now presented in a defined sequence. Animations were replaced, as they were not consistently perceived as helpful. For this purpose, pictograms were used, which were perceived as clear and beneficial for comprehension (Table 3 Q1). Due to the mentioned barriers for accessing the exercises, a mandatory exercise page for calculating the risk reduction has been added (TYK). Since some participants wanted more exercises to improve comprehension, a choice of optional exercises with increasing degrees of difficulty (E-1, E-2) was supplemented with a description of the exercises. These exercises were perceived as helpful, especially due to the increasing difficulty (Table 3 Q2). In addition, it was observed that tasks in the exercise process could be solved more easily and more often without displaying the formula. The section on baseline risk (BS) was completely revised and simplified so that the participants are now able to understand the effect of the basic risk with the help of individual calculation examples and with the same thematic orientation. These changes led to better comprehensibility ratings and participants were now more often able to solve the tasks and understand the meaning of the baseline risk (Table 3 Q3). The tool was also adjusted to suit mobile devices.

Apart from the major revision, minor revisions were carried out in two phases, which were intended to improve the acceptance, applicability, comprehension and didactic achievement of goals. Texts were shortened and sometimes replaced with pictograms. If possible, special terminology was avoided or explained with tool-tips. Option contents were reduced, and important contents highlighted. The tool design and the interaction options were amended, and the reference group highlighted by redesigning the pictograms. The menu navigation and section choices were also revised. These changes resolved the barriers that had occurred and improved clarity. Translation into English was subsequently carried out.