A transformer-based deep learning model for identifying the occurrence of acute hematogenous osteomyelitis and predicting blood culture results

1 Introduction

Osteomyelitis is one of the common bone and muscle infections in children. Acute hematogenous osteomyelitis (AHO) is the most common form of this disease in pediatrics. In recent years, the incidence of osteomyelitis has been increasing annually (Walter et al., 2021). The incidence is generally higher in males compared to females, and lower limb infections are more prevalent than upper limb infections (Kremers et al., 2015; Disch et al., 2023). Typically, healthy bones have strong resistance to pathogen invasion. The occurrence of osteomyelitis mainly occurs through three mechanisms: direct inoculation, extension from adjacent lesions, and hematogenous dissemination. Additionally, in conditions of bone ischemia, trauma, or foreign bodies, pathogens are more likely to adhere to exposed bone locations, leading to bone infection. Most cases of osteomyelitis can be cured; however, a small number of affected children may experience discrepancies in limb length between the affected and unaffected sides (Liu et al., 2024).

The presentation of AHO varies, ranging from localized infections at a single epiphyseal site to multifocal infections accompanied by septic shock (Funk and Copley, 2017). Fever and pain are the most common manifestations of bone infections. Common signs of osteomyelitis include fever, pain, swelling, erythema, localized warmth, and varying degrees of functional impairment. The onset of symptoms can differ depending on the type of pathogen involved (Calvo et al., 2016). When the lower limb bones are affected, children often have difficulty bearing weight or may exhibit noticeable limping, whereas involvement of the pelvis may lead to a waddling gait. Overall, the functional impairment and the location of the infection are highly correlated (Dich et al., 1975). In fact, similar symptoms can be observed in various pediatric orthopedic conditions. For example, osteosarcoma (OSC) and Ewing’s sarcoma (EWS) are the most common primary malignant bone tumors in children and young adults, and they also present with significant pain and swelling (Wang et al., 2022). Fractures also present with localized pain, swelling, functional impairment, deformities, and abnormal movements. Considering that children may have difficulty responding accurately to physical examinations, confirming a diagnosis of osteomyelitis through simple procedures is more challenging.

Acute hematogenous osteomyelitis is often caused by pathogen infections, so identifying the type of pathogen early in the disease is crucial for treatment (Jahan et al., 2024). This not only directly guides the physician’s treatment but also provides psychological comfort to the family. Although Staphylococcus aureus is the most common pathogen in osteomyelitis (McNeil, 2020), pathogen culture is the gold standard for pathogen diagnosis, with blood culture being the most common method (Woods et al., 2021). To obtain accurate culture results, samples need to be taken before the use of antibiotics. However, in practice, it is difficult to ensure that patients have not self-medicated with antibiotics before admission. Bone biopsy or aspiration is also limited to the early stages of the disease. Thus, to achieve accurate culture results, discontinuing antibiotics is often necessary (Manz et al., 2018). For critically ill patients, discontinuing antibiotics to obtain accurate bacterial culture results is clearly impractical. Despite the stringent sampling requirements, not all children with acute osteomyelitis can successfully culture the pathogen (Section et al., 2015). Discontinuing antibiotics is often difficult for patients due to the potential risks of disease progression. Therefore, discontinuing medication to diagnose the pathogen type is a significant clinical challenge.

Artificial intelligence has been widely applied in the diagnosis and treatment of osteomyelitis. AI methods not only efficiently handle repetitive tasks and improve diagnostic efficiency but also explore complex relationships between medical information, mapping features to manifestations, and establishing quantitative relationships between medical information and clinical outcomes. A 2022 study classified acute osteomyelitis, chronic osteomyelitis, and Ewing’s sarcoma using patient X-ray images (Consalvo et al., 2022). Another 2022 study proposed a machine learning model based on clinical features and biomarkers to classify diabetic foot, necrotizing fasciitis, and osteomyelitis, trained and validated on a dataset of 1,581 samples (Kim et al., 2022). A 2024 study included 145 patients with diagnosed spinal infections undergoing metagenomic next-generation sequencing (mNGS) to differentiate pathogen types in iatrogenic vertebral osteomyelitis (IVO) and native vertebral osteomyelitis (NVO) (Gao et al., 2024). These studies demonstrate the potential of AI in the diagnosis and treatment of osteomyelitis.

This study explored the correlation between laboratory parameters and diseases commonly confused with acute hematogenous osteomyelitis. Based on this, we developed an intelligent diagnostic system based on clinical and laboratory features, which can classify patients into categories of acute hematogenous osteomyelitis, benign bone tumors, malignant bone tumors, and fractures. At the same time, we investigated the correlation between the ability to culture pathogenic bacteria and the number of hospital days, and established a deep learning model to predict whether pathogenic bacteria can be cultured. This model supports clinicians in deciding whether to discontinue antibiotics for blood culture purposes.

2 Materials and methods 2.1 Patient

This study is a retrospective analysis that includes patients under 18 years of age who were hospitalized at our institution from January 1, 2016, to June 1, 2024. Baseline characteristics, including age and gender, were collected, as shown in Figure 1.

Figure 1. Inclusion and exclusion flowchart of patients in this study.

Inclusion criteria were as follows:

1. Age under 18 years at the time of admission.

2. Diagnosis of one of the following conditions: acute hematogenous osteomyelitis, benign bone tumor, malignant bone tumor, or fracture. AHO was diagnosed based on both laboratory indicators and clinical presentation. Benign and malignant bone tumors were diagnosed through pathology, while fractures were diagnosed through imaging or clear clinical manifestations (i.e., pain, deformity, and restricted movement).

3. For patients with AHO, at least one blood-based bacterial culture must have been performed during hospitalization.

Exclusion criteria were as follows:

1. Presence of metabolic diseases.

2. Presence of other primary tumors.

3. Presence of multiple primary infections.

4. Presence of primary heart, kidney, or liver dysfunction.

2.2 Laboratory parameters and blood culture

Collect laboratory data for the included population, The parameters related to blood cell counts include white blood cell count (WBC), red blood cell count (RBC), and platelets (PLT), all of which are obtained through the electrical impedance method (Coulter principle). Hemoglobin (HGB) is obtained using the sodium dodecyl sulfate hemoglobin colorimetric method. The white blood cell classification parameters include absolute eosinophil count (EO#, Eosinophil Count) and percentage (EO%, Eosinophil Percentage), absolute basophil count (BA#, Basophil Count) and percentage (BA%, Basophil Percentage), absolute lymphocyte count (LY#, Lymphocyte Count) and percentage (LY%, Lymphocyte Percentage), absolute monocyte count (MO#, Monocyte Count) and percentage (MO%, Monocyte Percentage), and absolute neutrophil count (NE#, Neutrophil Count) and percentage (NE%, Neutrophil Percentage). EO# and EO%, BA# and BA%, LY# and LY%, MO# and MO%, NE# and NE% are all obtained through VCS counting [V represents dual motor direct current characteristics (Coulter principle); C represents radio frequency conduction characteristics; S represents laser scattering]. Biochemical indicators include the albumin/globulin ratio (A/G), alanine aminotransferase (ALT), aspartate aminotransferase (AST), where AST is obtained through the reduced coenzyme method (NADH method). Albumin (AlbG) is obtained using the bromocresol green (BCG) method, and direct bilirubin (BilD) and total bilirubin (BilT) are obtained through the vanadate oxidation method. Gamma-glutamyl transferase (GGT) is obtained using the L-γ-glutamyl-3-carboxy-4-nitroaniline substrate method, and total protein (TP) is obtained using the biuret method. RH represents the Rh blood type (Rhesus Blood Type), obtained through the gel microcolumn method. Hematocrit (HCT), platelet count (PLT), mean platelet volume (MPV), plateletcrit (PCT), platelet distribution width (PDW), mean corpuscular volume (MCV), mean corpuscular hemoglobin (MCH), mean corpuscular hemoglobin concentration (MCHC), red cell distribution width (RDW), and unconjugated bilirubin (UNBIL) are all calculated values.

2.3 Blood culture

After the patient visits, blood samples should be collected as soon as possible. Draw 10 mL of blood from each arm of the patient for both aerobic and anaerobic blood cultures. Subsequently, send the blood culture bottles to the laboratory for testing. The detection involves monitoring whether bacteria consume nutrients in the culture bottles or produce new metabolites, triggering an alarm. Generally, if no alarm is triggered by the instrument within 5 days, the result is considered negative. If an alarm is triggered by the culture bottle, it is necessary to perform Gram staining microscopy and plate inoculation with the mixed solution from the culture bottle. For positive alarms in aerobic blood culture bottles, blood agar and MacConkey agar plates are commonly used. For positive alarms in anaerobic blood culture bottles, ensure anaerobic procedures and perform incubation in an anaerobic or microaerophilic environment.

2.4 Deep learning model

We developed a deep learning model structure to process laboratory parameters. The model consists of an ANN and a transformer. The transformer is a sequence-to-sequence model based on the attention mechanism, with the core idea of using self-attention to capture contextual relationships at different positions in the input sequence (Vaswani et al., 2017). We constructed feature vectors from clinical and laboratory data. The input is encoded through an ANN (including three linear layers with RELU activation functions) and then decoded by the transformer to classify the patients. For the two tasks in this study—differentiating AHO and identifying infection bacteria through blood cultures—we used the same model architecture, only changing the final fully connected layer to suit different tasks. The research workflow of this study is shown in Figure 2. The deep learning model in this study was trained on a workstation equipped with an NVIDIA RTX 4090 GPU (24GB VRAM) and 64GB of system memory. The training was based on PyTorch 2.4.1 and utilized CUDA 11.8 for acceleration.

Figure 2. Research workflow of this study. (A) We constructed feature vectors using patients’ clinical characteristics and laboratory parameters, and utilized the deep learning architecture proposed in this study for classification tasks. (B) The structural diagram of the model in this study. Input was encoded through a standard ANN (with three linear layers and RELU activation function) and decoded through a transformer.

2.5 Statistical analysis

Statistical analysis was performed using IBM SPSS software version 26.0. First, the Kolmogorov–Smirnov test was conducted to assess the normality of all data. For normally distributed data, the mean ± standard deviation was used for description, while for non-normally distributed data, the median (interquartile range) was used. Differences in continuous variables among multiple groups were analyzed using the Kruskal–Wallis H test, while categorical variables were analyzed using the chi-square test, with Cramér’s V used to quantify the strength of association. In the analysis between two groups, after testing for normality, the Mann–Whitney U test was applied to non-normally distributed data, and the t-test was used for normally distributed data. Differences in categorical variables were analyzed using the chi-square test, and Cramér’s V was used to quantify the association between categorical variables. Correlations between continuous variables were assessed using Pearson’s test for normally distributed data and Spearman’s test for non-normally distributed data. A p-value of <0.05 was considered statistically significant. Statistical plots were generated using Python’s matplotlib.

3 Results 3.1 Baseline

A total of 634 patients were included in this study, with variables having missing values greater than 20% being removed. Table 1 shows the clinical information and laboratory parameters of the included patients.

Table 1. Baseline information of patients.

3.2 Multicategory correlation analysis of laboratory parameters for AHO, benign bone tumors, malignant bone tumors, and fractures

The expression of laboratory parameters in patients with different disease types is shown in Figure 3.

Figure 3. Expression of laboratory parameters in different disease types.

Correlation test results for disease types and clinical characteristics, as well as laboratory parameters, are presented in Tables 2, 3. We found that, among clinical characteristics, both gender and age were related to orthopedic disease types. In laboratory parameters, all parameters included in this study were related to orthopedic disease types.

Table 2. Association between categorical variables and disease classification using cramér’s V.

Table 3. Analysis of variations in continuous variables across different disease groups.

3.3 Multicategory deep learning of laboratory parameters for AHO, benign bone tumors, malignant bone tumors, and fractures

By removing columns with more than 20% missing values and imputing the missing values, a total of 21 common variables across the four diseases were used in modeling. The training, validation, and test sets were split in a 6:2:2 ratio, with baseline characteristics shown in Table 4.

Table 4. Baseline characteristics of the training, validation, and test sets for multicategory deep learning of laboratory parameters in AHO, benign bone tumors, malignant bone tumors, and fractures.

The training results are shown in Figure 4. Over 32 epochs, the model achieved optimal performance on the validation set and attained an accuracy of 1.0 on the test set.

Figure 4. Accuracy and loss curves of the multicategory deep learning model for laboratory parameters in AHO, benign bone tumors, malignant bone tumors, and fractures on the training and validation sets.

3.4 Correlation between hospitalization duration for AHO and laboratory parameters

As shown in Tables 5, 6, although AHO is more common in males, there is no correlation between the patient’s gender or age and their hospitalization duration. Among the laboratory parameters, RDW is positively correlated with the length of hospital stay, while HCT, HGB, MCH, MCV, and RBC are negatively correlated with the length of hospital stay, with a p-value of <0.05. This indicates a strong link between the patient’s hospitalization duration and their red blood cell physiological state. Although inflammation markers can assess disease severity, they are not directly related to the patient’s hospitalization duration. Liver function also shows no correlation with the hospitalization duration for AHO. The correlation between bacterial culture results and clinical information or laboratory parameters in AHO patients reveals statistical differences in HCT, HGB, RDW, A/G, and AlbG. Additionally, there is a significant difference in the length of hospital stay between the negative and positive culture sample groups. As shown in Table 7, patients with positive cultures have a longer hospital stay.

Table 5. Correlation between continuous variables and hospitalization duration.

Table 6. Analysis of variations in hospitalization duration across different categorical variables.

Table 7. Baseline information for AHO bacterial culture.

3.5 Correlation analysis of bacterial culture results in AHO

Baseline information for AHO patients is shown in Table 7. Additionally, among the patients with positive cultures, the identified strains included 51 Staphylococcus aureus, 1 Acinetobacter baumannii, 1 Staphylococcus epidermidis, 1 Bacillus subtilis, 1 Achromobacter xylosoxidans, 1 Klebsiella, 1 Malassezia furfur, and 1 Pseudomonas aeruginosa.

As shown in Tables 8, 9, there are no significant differences in gender and RH between the negative and positive culture sample groups. Only HGB, A/G, AlbG, GGT, and HCT showed significant differences between the negative and positive culture sample groups. According to Table 7, the positive culture group had lower levels of HGB, A/G, AlbG, and HCT, and higher levels of GGT. The upregulation and downregulation situations in the two groups are illustrated in Figure 5.

Table 8. Association between categorical variables and bacterial culture negative/positive groups using cramér’s V.

Table 9. Analysis of variations in continuous variables between bacterial culture negative and positive groups.

Figure 5. Upregulation and downregulation of parameters in the groups with or without detection of pathogenic bacteria in AHO patients: parameters were adjusted as follows: RDW was upregulated, while A/G, HGB, HCT, and AlbG were downregulated.

3.6 Deep learning model for detecting pathogenic bacteria in AHO

By removing columns with more than 20% missing values and filling in the missing values, 34 common variables from both bacterial culture negative and positive AHO patients were used for modeling. The baseline features are shown in Table 10.

Table 10. Baseline features of the deep learning model for pathogen detection in training, validation, and test sets.

The training/testing ratio was 6:2:2. The training results are shown in Figure 6. Over 58 epochs, the model achieved optimal performance on the validation set and an accuracy of 1.0 on the test set.

Figure 6. Accuracy and loss curves of the deep learning model for pathogen detection on training and validation sets.

4 Discussion

Acute hematogenous osteomyelitis is a challenging diagnosis in pediatric emergency departments. The condition can develop gradually over a few days but typically manifests within 2 weeks. Patients may present with localized symptoms such as redness, swelling, and fever at the infection site. They might experience dull pain, with or without movement, and sometimes systemic symptoms such as fever or chills. In subacute cases, some patients may exhibit generalized discomfort, mild pain over several weeks accompanied by slight fever or other systemic symptoms (Schmitt, 2017). The variety of symptoms and the difficulty children have in clearly describing their condition make the diagnosis and treatment of osteomyelitis quite challenging (Stephan et al., 2022). In this study, we propose a diagnostic model consisting of two deep learning models that can accurately diagnose AHO in children and predict whether the pathogenic bacteria can be identified through blood cultures. This study not only provides clinicians with a straightforward method for confirming AHO but also offers support for decisions regarding the necessity of stopping antibiotics for bacterial culture.

There are several methods for diagnosing AHO, with blood cultures and X-rays currently being strongly recommended. However, both methods have their drawbacks. Blood cultures require a lengthy period to yield results and cannot always ensure accuracy (Doern et al., 2019). X-rays, while simple, quick, and safe, have low sensitivity (Zaki and Morrison, 2024). Additionally, MRI often requires sedation for pediatric patients, and collecting samples from the affected area for bacterial culture through invasive methods faces the same issues as blood culture (Dartnell et al., 2012; Dong et al., 2019). In addition to these two types of examination methods, recent research by Paliwal et al. (2021) has revealed the diagnostic capability of ultrasound for acute hematogenous osteomyelitis (AHO). By assessing the accumulation of deep soft tissue fluid around the bones in AHO cases, rapid diagnosis can be achieved. This method is also expected to advance the diagnosis of AHO (Paliwal et al., 2021).

Additionally, Stephan et al. (2022) identified elevated C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR) as the most sensitive laboratory markers in pediatric emergency settings. In the study by Manz et al. (2020), elevated CRP was also found to be associated with poor long-term outcomes in AHO. Based on previous studies, Stephan et al. (2024) also demonstrated in subsequent experiments that elevated CRP and ESR are closely related to poor long-term prognosis in AHO. In this study, we propose a method for diagnosing AHO through laboratory parameters. By employing a multiclass model structure, it can effectively diagnose several common orthopedic conditions, significantly enhancing diagnostic efficiency. Considering that the model in this study needs to differentiate between osteomyelitis and other orthopedic diseases, and that C-reactive protein and erythrocyte sedimentation rate are not essential laboratory parameters for orthopedic diseases, they were not included in this study.

Different types of pathogenic bacteria may lead to variations in hospital length of stay. A 2023 study indicated that there is a difference in hospital stay duration for osteomyelitis caused by methicillin-resistant Staphylococcus aureus (MRSA) and methicillin-sensitive Staphylococcus aureus (MSSA) (Wen et al., 2023). Additionally, children with acute osteomyelitis who are Black, Hispanic, or of other races and ethnicities have longer hospital stays compared to White children (Campbell et al., 2023). In our study, we found that the infection level and liver function levels of patients were not associated with the length of hospital stay. Considering that in China, the length of hospital stay is highly correlated with the patient’s recovery of physical health and is less influenced by the wishes of the patients and their families, we believe that the status of the red blood cells is an important factor affecting the recovery process of AHO. This study found a positive correlation between RDW and length of hospital stay, while HCT, HGB, MCH, MCV, and RBC showed a negative correlation with length of hospital stay. This may be due to the important role of red blood cells in oxygen transport and bone tissue repair. If there is an insufficient number of red blood cells (as seen in anemia) or abnormal red blood cell function, the bone tissue may not receive enough oxygen, which can delay repair.

The Pediatric Infectious Diseases Society and the Infectious Diseases Society of America recommend performing blood cultures before the use of antibiotics (Woods et al., 2021). Despite the presence of clear symptoms, the detection rate of blood cultures remains unsatisfactory, and this rate rarely exceeds 60% in various cohorts (Russell et al., 2015; McNeil et al., 2019). Although sampling from adjacent infected sites can somewhat improve the detection rate, it does not provide an overwhelming advantage over blood cultures (Athey et al., 2019). In practical medical work, patients often have already been on oral antibiotics for some time by the time they arrive at the hospital. Once effective antibiotics are started, the detection rate of blood cultures usually drops rapidly within a few hours of exposure. Therefore, deciding whether to stop antibiotics for culture testing is a challenging decision for physicians. Previous studies have analyzed different clinical features as independent predictors of positive blood cultures (Burns et al., 2023). In this study, our model accurately predicts whether pathogens can be identified through blood cultures. This provides guidance for clinicians on whether to proceed with empirical antibiotic treatment. This study reveals that, in addition to affecting the recovery process of AHO patients, red blood cell status also influences the results of bacterial culture. This may be related to how red blood cell status affects the overall function of the immune system, thereby impacting the proliferation and distribution of bacteria within the body. Abnormal red blood cell function may weaken the immune response to infections, allowing pathogens to become localized in tissues and making them more difficult to detect in blood cultures.

Despite this, the study has some limitations. First, it is a single-center study, including both laboratory parameters and blood culture results. This suggests that the study may have potential issues with generalizability. Second, the study only established a classification method between acute hematogenous osteomyelitis and benign bone tumors, malignant bone tumors, and fractures, without exploring classifications for different subtypes and disease progressions of osteomyelitis, which is also due to a lack of data. Third, while the study predicted whether pathogens could be successfully cultured from patients, it did not further classify the cultured pathogens. Fourth, this study only included laboratory parameters from AHO patients, while the diagnosis of AHO can also rely on imaging information. Due to a lack of data, this study did not include any medical imaging information. Therefore, the next step for this research is to collect data from multiple centers, expand the cohort, and include more data modalities.

5 Conclusion

Early laboratory parameters can accurately diagnose whether pediatric patients have acute hematogenous osteomyelitis. Laboratory parameters can describe the severity of acute hematogenous osteomyelitis and are somewhat correlated with the patient’s length of hospital stay. Early laboratory parameters can predict whether a patient’s blood sample will successfully culture pathogens, thereby guiding clinical decision-making, indirectly improving clinical outcomes, and shortening the hospital stay.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Ethics Committee of The Second Hospital of Dalian Medical University. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Author contributions

YX: Formal analysis, Investigation, Methodology, Writing – original draft. QK: Data curation, Investigation, Validation, Writing – original draft. YG: Conceptualization, Data curation, Investigation, Methodology, Visualization, Writing – original draft. JS: Funding acquisition, Project administration, Supervision, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Athey, A. G., Mignemi, M. E., Gheen, W. T., Lindsay, E. A., Jo, C. H., and Copley, L. A. (2019). Validation and modification of a severity of illness score for children with acute Hematogenous osteomyelitis. J. Pediatr. Orthop. 39, 90–97. doi: 10.1097/bpo.0000000000000879

PubMed Abstract | Crossref Full Text | Google Scholar

Burns, J. D., Upasani, V. V., Bastrom, T. P., Baldwin, K. D., Schoenecker, J. G., Shore, B. J., et al. (2023). Age and CRP associated with improved tissue pathogen identification in children with blood culture negative osteomyelitis: results from the CORTICES multicenter database. J. Pediatr. Orthop. 43, e603–e607. doi: 10.1097/bpo.0000000000002448

PubMed Abstract | Crossref Full Text | Google Scholar

Calvo, C., Núñez, E., Camacho, M., Clemente, D., Fernández-Cooke, E., Alcobendas, R., et al. (2016). Epidemiology and management of acute, uncomplicated septic arthritis and osteomyelitis: Spanish multicenter study. Pediatr. Infect. Dis. J. 35, 1288–1293. doi: 10.1097/inf.0000000000001309

PubMed Abstract | Crossref Full Text | Google Scholar

Campbell, J. I., Shanahan, K. H., Bartick, M., Ali, M., Goldmann, D., Shaikh, N., et al. (2023). Racial and ethnic differences in length of stay for US children hospitalized for acute osteomyelitis. J. Pediatr. 259:113424. doi: 10.1016/j.jpeds.2023.113424

PubMed Abstract | Crossref Full Text | Google Scholar

Consalvo, S., Hinterwimmer, F., Neumann, J., Steinborn, M., Salzmann, M., Seidl, F., et al. (2022). Two-phase deep learning algorithm for detection and differentiation of Ewing sarcoma and acute osteomyelitis in paediatric radiographs. Anticancer Res. 42, 4371–4380. doi: 10.21873/anticanres.15937

PubMed Abstract | Crossref Full Text | Google Scholar

Dartnell, J., Ramachandran, M., and Katchburian, M. (2012). Haematogenous acute and subacute paediatric osteomyelitis: a systematic review of the literature. J. Bone Joint Surg. Br. 94, 584–595. doi: 10.1302/0301-620x.94b5.28523