記住我
ORIGINAL ARTICLE
Year : 2021 | Volume
: 8
| Issue : 4 | Page : 243-248
Usability of electronic death registration system: A case study of heuristic evaluation
Fatemeh Rangraz Jeddi1
, Razieh Farrahi2
, Sara Chopannejad3
1 Health Information Management Research Center; Department of Health Information Management and Technology, School of Allied Health Professions, Kashan University of Medical Sciences, Kashan, Iran
2 Department of Health Information Management and Technology, School of Allied Health Professions, Kashan University of Medical Sciences, Kashan, Iran
3 Department of Health Information Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran
Correspondence Address:
Dr. Razieh Farrahi
Pezeshk Blvd, 5th of Qotbe Ravandi Blvd - Pardis Daneshgah, School of Allied Health Professions, Kashan University of Medical Sciences, Kashan
Iran
Source of Support: None, Conflict of Interest: None
DOI: 10.4103/iahs.iahs_86_21
Aim: This study was conducted to identify gaps in the user interface design of Iran's Electronic Death Registration System (National Registry) using the heuristic method. Materials and Methods: Five independent evaluators assessed the user interface using the heuristic evaluation. Problems were reassessed in the presence of all evaluators, and similar cases were merged, and a single list of unique problems was eventually prepared. By re-evaluating the system, the evaluators determined and categorized the severity of the problem in five domains, including no problem (zero point), a cosmetic problem (1 point), a minor problem (2 points), a major problem (3 points), and a catastrophic problem (4 points). The data were then analyzed in an excel sheet using descriptive statistics. Findings: Forty-two unique problems were identified. The “recognition rather than recall” principle with ten problems (23.8% of all cases) had the most frequent problem, while the “help and documentation” principle with one problem (2.4% of all cases) had the least frequency but the highest mean severity (4). Conclusion: The results of the heuristic evaluation of the electronic death registration system indicated that a large portion of the problems in this system was classified under the minor category, which primarily indicates the good usability of this system. However, this system suffers from several usability problems. Therefore, it could be concluded that considering the national-level use of this system, it has problems that could be resolved through meeting system design principles and standards.
Keywords: Death, death certification, evaluation, healthcare, heuristic, registry, usability, user interface
In any country, health care systems are intended to reduce the incidence of diseases and their mortality by implementing interventions and prevention programs.[1] This goal is accomplished by the collection of data, the registration of events, the calculation of indices, comparisons against standards, and information feedback through registries.[1],[2] Successful data collection through a registry requires structures compatible with modern information technologies, especially information systems.[3]
Human and software are the two essential components in the accurate recording of essential indicators such as mortality.[4],[5] In this regard, Huang suggested that healthcare softwares must undergo the most meticulous quality assurance procedures. Software evaluation is a significant factor in ensuring software products' quality and is of utmost importance for users and decision-makers.[6]
Usability is one of the most prominent criteria for evaluating effective and efficient interaction as well as user satisfaction.[7] The International Organization for Standardization defines the usability of an information system as “the extent to which a system can be used by specific users while maintaining efficiency, effectiveness, and satisfaction in achieving specific goals.”[8] Compared to other evaluation methods, expert-based evaluation methods, especially heuristic evaluation, identify more problems with less money, time, and resources.[9],[10],[11] The features of this method include the possibility of use during the evaluation of its user interface and system components without having to involve the users, which is known as the first step in the evaluation of information systems.[12] In this method, identifying major problems is more straightforward than minor problems. Moreover, experts' severity ranking of problems makes it possible to identify major problems affecting users' tasks and prioritize a resolution process without wasting workforce and resources. Once major problems are corrected, and the user interface is cleared, additional problems can be identified by performing a user-based evaluation.[12]
Various registries have been developed worldwide as information systems for the registration of cancer, death, diabetes, acquired immunodeficiency syndrome (human immunodeficiency virus), and the like.[13],[14] Data related to deaths have long been regarded as the most important health data worldwide, and particular attention has been paid to them by international organizations such as the World Health Organization.[14],[15],[16],[17] Electronic death registration systems have, therefore, been designed and are used in most countries.
The National Organization for Civil Registration is in charge of recording births and deaths in Iran. Given that accurate registration of the cause of death is a specialized matter, but the information collected by this organization could not respond to the information needs for planning and policymaking in the healthcare sector. Thus, the Ministry of Health established the initial foundation of a system for death registry in 1965 according to the type of information it required.[1] The Ministry Ministry of Health, Treatment, and Medical Education founded the system for the registry and classification of causes of death in 1997 in the Bushehr province. The use of this registry expanded across every other province in the country. The electronic system for registry and classification of causes of death was designed in 2004, requiring all medical science universities to register information in this system.[1]
Any problems in this system's user interface would affect timely and accurate registration of death-related data, the time spent by the user to register data, satisfaction, and therefore, the accuracy of the submitted reports, and the speed of preparing reports for the responsible organizations.
The results of a study conducted by Sadeqi et al. as “review of usability studies conducted on Iranian hospital information systems” showed: In studies of usability evaluation by heuristic method, the most problems with the usability of hospital information systems related to the principles of consistency and standards and match between system and the real world. Moreover, principles such as “consistency and standards”, “error prevention,” and “help and documentation” had problems with the highest severity.[18]
Therefore, it seems necessary to evaluate this registry's user interface to improve the existing software or use other countries in designing registry software, given its widespread national use for registering data as a significant health-related indicator. This system is currently in commission and used by real users, but its user interface has not yet been evaluated. Therefore, user testing with real users is not cost-effective, and it is necessary to use a sufficient number of experts who, based on knowledge in evaluating the usability and their familiarity with health systems and software, can evaluate the user interface through the heuristic method as a summative evaluation of the system. This would help better understand whether the system user interface stands up to the competition and whether a major redesign is warranted. Therefore, the present study was conducted to determine the usability problems of the user interface of Iran's electronic death registration system based on the heuristic evaluation.
The study protocol was approved by the Ethics Committee of Kashan University of Medical Sciences with ethics code IR.KAUMS.NUHEPM.REC.1396.11.
Materials and MethodsThis descriptive cross-sectional study was conducted on the national electronic death registration system at the Health Deputy of Kashan University of Medical Sciences in 2020.
A copy of the death certificate is sent from hospitals to the Health Deputy of Kashan University of Medical Sciences. Death certificate information is registered in the electronic system of death registration by the user located in the Health Deputy office to accurately provide a snapshot of death and illness in that particular geographical area and identify the common causes of death.
This system was designed and implemented by order of the Health Deputy of Minister of Health, Treatment, and Medical Education and is used by all medical sciences universities across the country to register mortality data. All mortality reports are currently obtained from this system.
Heuristic evaluation is among the most popular methods for the usability evaluation of the user interface of a system without involving users.[19] In this method, three to five evaluators examine the user interface in compliance with specific standard principles (Jakob Nielsen's 10 general principles).[20] After the problems are detected, their severity and consequences for users could be determined.[21],[22]
Since three to five evaluators could identify about 60%–75% of the problems.[23] The present study was conducted by five experts in heuristic evaluation. An expert with a Ph.D. in Health Information Management and 20 years of experience in teaching health information management, three Ph.D. candidates of Health Information Management holding master's degrees in Health Information Technology who had experience in working with different health information systems, and an M.Sc. student of Health Information Technology familiar with health information systems were among our participating experts. Moreover, evaluators had previously participated in one or more usability evaluation studies and were consequently familiar with heuristics evaluation method and death registry.
The present study was conducted over four stages:
The first stage
familiarization with the system; each evaluator examined the structure and user interface of the national electronic death registration system.
The second stage
evaluators individually evaluated the system's user interface in terms of conformity to Nielsen's usability principles and entered the problems they found in the data collection form. This form consisted of a four-column table inquiring the title, descriptions, and area of the problem as well as the usability principle is violated.
The third stage
the list of the detected problems was examined in the presence of all five evaluators, identical and similar problems were integrated, and a unified list of problems was prepared. Any disagreements on the detected problems and their association with Nielsen usability principles were discussed and resolved in this meeting.
The fourth stage
evaluators determined the severity of problems by an independent second assessment of the user interface based on the following criteria:[22]
Frequency of exposure to risk: whether the problem occurs frequently or rarelyImpact of the problem on the user experience: whether the problem is easy to overcomePersistence of the problem: whether the problem is solved on the first attempt, whether its recurrence causes any problems.The mean severity for each problem in each principle is calculated based on severity level, determined by each evaluator according to [Table 1].[22],[23],[25] Subsequently, the mean severity of the problems was classified into one of the five levels demonstrated in [Table 1]. The decimal numbers were rounded to the closest whole number to classify the problems based on the standard severity scale. When one of the principles was completely absent from the system, the principle was rated as a “catastrophic problem” only once. The data were analyzed by Microsoft office(excel)2019 using descriptive statistics.
ResultsOf the five evaluators, four were female with experience in evaluating health information systems. The evaluators detected a total of 90 problems. Forty-two unique problems were detected after the integration of the detected problems and elimination of the repeated problems (such as lack of feedback to the user while performing the tasks, incompatibility of the expressions used in the menus with user's tasks, and incapability of canceling the tasks being performed by the user). The highest frequency among the problems was attributed to the principle of “recognition rather than recall” [Table 2].
Table 2: The number of usability problems detected based on heuristic evaluation principles[Figure 2] indicates the results of heuristic evaluations. Stacked columns indicate the number of the system's usability problems in each of the heuristic principles, while the line with markers indicates the mean intensity of the problem detected in each principle. The horizontal axis in the figure demonstrates the ten usability principles. The left-side vertical axis indicates the number of detected usability problems, while the right-side vertical axis illustrates the severity of the respective problems. The problems detected in “help and documentation” and “user control and freedom” were categorized as “catastrophic and major problems” in terms of the severity scale with respective mean intensities of 4 and 3.2, while the smallest average intensity was associated with “aesthetic and minimalist design” with an average severity of one. Among the total distinct problems detected, five problems (12%) were categorized as cosmetic, 13 (31%) problems were categorized as minor, 13 problems (31%) were categorized as major, and 11 problems (26%) were categorized as catastrophic.
Figure 2: A comparison of the frequency and severity of the problems associated with the principles of heuristic evaluation Discussion A total of 42 unique problems were identified, among which 11 cases (26%) were catastrophic problems and 13 cases (31%) were major problems. The principle of “recognition rather than recall” had the highest frequency of problems with 10 cases (23.8%) and average severity of 1.5, which was categorized as a minor problem. In contrast, the principle of “help and documentation” had the least frequency of problems with one case but an average intensity of 4, which placed it in the category of catastrophic problems.
Despite having the smallest number of problems, the principle of “help and documentation” was categorized under catastrophic problems. The small number of problems associated with this principle is because the national electronic death registration system lacks a help and documentation section. The system also lacked question-and-answer interfaces, and the user would not receive any guidance in case of encountering an error or filling in the information fields. According to the research method, those mentioned above were considered as only one problem. A review of previously conducted studies[24],[25],[26] finds that the help and documentation section has been somewhat neglected in Iranian healthcare systems. Thus, a help and documentation section are suggested to be included in future versions of the death registration system, given the evaluator's opinion regarding the importance of this section in helping users and improving the quality of information systems, as well as the credibility of mortality data registration conforming to the formats approved by the Ministry of Health– which is the essential purpose of the death registration system.
Problems associated with “user control and freedom” were categorized under major problems. This principle refers to users having the option to log out of the system or cancel any given operation at their leisure. The “undo” function was not included in the system, and there was no option to cancel operations. Given that the electronic death registration system is used on a national level and users receive mortality data from various sources, they might encounter redundant data, so they must have control over the system to eliminate duplicated data and operate cause of death coding rules so that they can correct and delete information if required and undo the commands. Hence, further attention to designing these features in the next version appears imperative to prevent data redundancy in the system and ensure the accuracy of the statistics and reports obtained from the system. The impact of user interface problems on the validity and accuracy of data registered in the system is an area to be examined.
Problems associated with the “consistency and standards” principle were categorized under major problems. The unavailability of function keys in performing tasks and the use of various icons for the same task, such as data inquiry, are examples of problems associated with this principle that has made it difficult for current and prospective users to learn and work with the system. Users' confusion will increase the time required for performing tasks and reduce efficiency. On the other hand, the use of various icons for the same task results in redundancy in software design and complications for software designers. Therefore, it is recommended to hire several designers from various design teams to examine this principle and compliance with the standards.
Having the most significant number of problems, problems associated with “recognition rather than recall” were categorized under minor problems. Among the problems associated with this principle, one could mention the similarity of mandatory and optional fields, the lack of prominent visual cues to detect active windows, and the lack of proper distance between data labels and data registration. The most significant problems associated with this principle that could impact data registration from users' points of view must be detected and fixed, given the importance of mortality data registration and data quality improvement.
Taking advantage of experienced and knowledgeable evaluators, implementing unified principles (Nielson's 10 principles) to evaluate user interface, accessibility of the system for evaluation and determination of problems' severity, and generalizability of the identified problems due to the national use of the system are the strengths of the present study. Simultaneously, its weaknesses include not providing a solution by the evaluators to solve the problems and not considering users' opinions to determine the severity of the identified problems.
Hence, it is recommended to evaluate the system for compliance with other usability principles such as content and privacy. Future versions must consider taking advantage of users' opinions to determine the severity of problems and address them. The problems must be prioritized according to their impact on performing tasks and the modification costs. The results of the present evaluation could be used to define scenarios in a user-based evaluation.
ConclusionThe heuristic evaluation of the electronic death registration system indicated that many problems in this system were classified under the minor category, which primarily indicates the good usability of this system. However, this system suffers from several usability problems, specifically in the “help and documentation” and “user control and freedom” sections. Therefore, considering the national-level use of this system, it could be concluded that it has problems that could be resolved through meeting system design principles and standards and would leave negative impacts on user performance and registered data accuracy in case of nonresolution. The results of the present study will contribute to the development of a user-friendly interface, which will result in the consistency of the mortality data registration method and the quality of registered data. Therefore, it is recommended to examine the influence of usability problems on the accuracy of mortality data, statistics, and reports from authorities' and users' points of view.
Acknowledgments
The authors wish to express their gratitude to the Health Deputy of Kashan University of Medical Sciences for providing the death registration software and thus enabling our evaluation of its user interface.
Financial support and sponsorship
This study was funded by the Research Deputy of Kashan University of Medical Sciences under grant No. 9663.
Conflicts of interest
There are no conflicts of interest.
References
留言 (0)