1. IntroductionCoronavirus disease 2019 (COVID-19) [1] is a contagious disease caused by a coronavirus called SARS CoV-2. This disease quickly spread worldwide, causing a global pandemic. Symptoms of COVID-19 appear 2–4 days after exposure. People with COVID-19 may experience fever or chills, cough or shortness of breath, breathing difficulties, headache, fatigue, and loss of smell or taste. According to the World Health Organization, COVID-19 infection can be detected by testing specimens from nose or mouth swabs. Real-time reverse transcription-polymerase chain reaction (RT-PCR) is used to detect nucleic acids in secretory fluids obtained from specimens. Because coinfection with other viruses can impact RT-PCR prediction performance, repetitive testing may be recommended to prevent false negatives. The RT–PCR test has a three-day turnaround time, as RT–PCR test tools have been scarce in recent months. There has been a pressing need for additional procedures to quickly and reliably identify COVID-19 patients. Furthermore, the swabbing operation is highly susceptible to expert errors, and it must be performed repeatedly [2]. Therefore, X-rays or CT scans of the chests are suitable complements to RT-PCR because they can be gathered and processed considerably more quickly [3]. Relative to CT scans, taking chest X-ray images is less expensive, radiation-exposing, and time-consuming. In addition, CT nuclear scanning delivers larger radiation doses than traditional X-rays scanning [4]. An X-ray of the chest produces 0.1 mSv, whereas a CT produces 70 times the amount. X-ray machines are widely available and quickly provide images for diagnosis. Thus, in this work, we focus on recognizing diseases in X-ray images.Since the reintroduction of convolution neural networks (CNN) [5], deep learning has become dominant in many research fields, such as computer vision, natural language processing, and video/speech recognition. To date, deep learning has been adopted by a wide range of applications, owing to its scalability, speed, and efficiency, even outperforming humans in specific industrial processes [6]. As reviewed in [7], medical science is a relatively new field that is attempting to leverage the success of artificial intelligence and deep learning models. Developments in digital data collection, computer vision, and computation infrastructure have enabled AI applications to move into areas that were previously regarded as entirely human domains [8]. Deep learning in radiology is a game changer in terms of both quality and quantity when it comes to biomedical imaging explanation and data processing. Although machine learning and deep learning algorithms have demonstrated their ability to classify tumors and cancer progression, radiologists are still hesitant to use them [9]. One of the numerous advantages of machine learning in radiology is its capacity to automate or even replace radiologist scanning methods. Deep learning algorithms produce outcomes that are comparable to those of a top radiologist. However, situations in which resources are limited and requirements are particularly demanding, such as the COVID-19 pandemic, exemplify the need for a robust algorithm to assist medical professionals. Having witnessed the extraordinary performance of deep learning in various tasks [10,11,12,13,14,15,16,17,18,19,20,21,22], we investigated deep learning models in this paper. Inspired by the efforts and experience of healthcare professionals such as doctors and specialists during the pandemic, we propose a doctor consultation-inspired model to fuse various deep learning models to produce accurate outputs.

The novelty of this work is as follows. First, the proposed framework is motivated from the perspective of physicians. The doctor consultation-inspired method is formulated in the form of fusion models. The proposed method considers each individual deep learning model as a medical doctor. Then, a consultation is performed based on inputs from multiple individual models. In this regard, the proposed method leverages the strengths of available methods in order to boost the performance. Second, the proposed method is open in the sense that any future individual methods can be integrated into our method. Third, we evaluate the proposed method on two benchmark datasets with different consultation modes, namely early consultation and late consultation.

The remainder of this paper is organized as follows. In Section 2, we summarize related works. In Section 3, we introduces the proposed doctor consultation-inspired model. The experiments and the experimental results are presented and discussed in Section 4. Finally, Section 5 concludes the paper. 2. Related WorksMany efforts to diagnose COVID-19 and pneumonia from X-ray images have been reported in the literature. In [10], deep neural network techniques were used in conjunction with X-ray imaging to identify COVID-19 infection. The main goal of this effort was to help alleviate doctor shortages in rural areas by providing resources to fill the gap. Shibly et al. [10] used VGG-16 [11] architecture to identify COVID-19 patients from chest X-ray images. The proposed method may aid medical professionals in screening COVID-19 patients. In another work, Sethy et al. [12] sought to detect coronavirus-infected patients using X-ray images. This method involves radiographic analysis using support vector machines (SVMs) with deep features extracted from ResNet50 [13]. The efficacy of a multi-CNN in automatically detecting COVID-19 from X-ray images was examined by Abraham et al. [14], who employed naive Bayes, SVM, AdaBoost, logistic regression, and random forests before settling on the Bayes net. The best-performing method is Xception [15]. Mei et al. [16] proposed a machine learning strategy that uses diagnostic imaging and clinical studies to accurately detect COVID-19-positive patients. The authors created a DCNN to learn the initial imaging characteristics of COVID-19 patients (18-layer residual network: ResNet-18 [13]). In the next stage, random forest, SVM, and MLP classifiers were used to categorize COVID-19 patients. Multilayer perceptron (MLP) performed best on the tuning set, and a neural network model was utilized to evaluate COVID-19 status based on radiographic and clinical data. Hurt et al. [17] improved their method by just using frontal chest X-ray images. They discovered that the probabilities in their model that are matched to the quality of the imaging data are remarkably general and reliable. According to recent research, machine learning algorithms can distinguish COVID-19 from other pneumonia strains. Tuncer et al. [18] developed a technique for COVID-19 recognition using X-ray scans of the lungs. This technique is broken down into stages, which are detailed as follows. Residual example local binary pattern [19] is the name given to this method (ResExLBP). In the feature selection step, the IRF-based attribute selection method is used. Decision trees, linear classifiers, SVM, k-NN, and SD approaches are utilized in the classification step. Using 10-fold cross validation, the SVM classifier once again achieved the best performance. Recently, Hemdan et al. [20] developed a deep learning framework to aid radiologists in detecting COVID-19 in X-ray scan. They investigated many deep artificial neural networks to classify the patient’s COVID-19 status as negative or positive. Machine learning classifiers VGG19 [11] and DenseNet201 [21] achieved the best results in predicting COVID-19 using two-dimensional X-ray images. Recently, a rapid COVID-19 diagnosis technique was proposed by Ardakani et al. [22]. The authors used ten well-known pre-trained CNNs for this purpose. They trained and tested the 10 CNNs using the same dataset and compared the results to a radiologist’s classifications. For COVID-19 individuals, ResNet-101 [13] achieved the best performance. Additionally, there are many deep learning models proposed for classification [23,24].Many optimization and refinement steps have been proposed to improve the performance of classifiers. For example, data augmentation [5] enhances the size and quality of training datasets. Waheed et al. [25] proposed a GAN-based model to synthesize medical images, with the aim of increasing the number of training samples required to train a CNN-based model to detect COVID-19 from medical images. In another study, Oh et al. [26] proposed a patch-based deep neural network architecture that can be trained with a small dataset. Teixeira et al. [27] used a UNet-based lung segmentation model [28] to segment the lung first. Then, they used a CNN-based model to classify X-ray images. Similarly, Tartaglione et al. [29] adopted segmented lung images. Then, they used a feature extractor pretrained on CXR pathology datasets and fine-tuned it on COVID datasets. Balaha et al. [30] introduced a framework with a segmentation phase to segment lung regions. Then, data augmentation such as rotation, skewing, translation, and shifting was applied. Finally, a genetic algorithm was used to learn combinations of hyperparameters. Baghdadi et al. [31] presented an algorithm for COVID-19 classification using a CNN, pre-trained model, and Sparrow search algorithm on CT lung images. Perumal et al. [32] proposed a transfer learning model with Haralick features [33] to speed up the prediction process and assist medical professionals. Transfer learning alleviated the problem of the lack of COVID-19-positive data to some extent. A comparison of related works provided in Table 1. However, a review of all models used for COVID-19 detection is beyond the scope of this paper. Additional research works involving COVID-19, CNNs, and data augmentation were covered in [34,35,36]. 5. Conclusions

In this paper, we propose a doctor consultation-inspired method for recognizing disease from X-ray images. Inspired by doctor consultation practice, we explore two modes, namely late fusion and early fusion. The proposed method takes advantage of multiple state-of-the-art networks to efficiently recognize disease from an input X-ray image. The early fusion mechanism combines the deep-learned features of various models, whereas the late fusion method combines the confidence scores of all individual models. Experiments show the superiority of the proposed method over individual methods. Both fusion mechanisms outperform baselines by a large margin. In addition, the early fusion model consistently outperforms the late fusion mechanism on the two benchmark datasets. In particular, the early doctor consultation-inspired model outperforms all individual models by a large margin, i.e., 3.03 and 1.86 in terms of accuracy on the UIT COVID-19 and chest X-ray datasets, respectively.

In the future, we intend to extend our model for different diseases. Moreover, we plan to explore different kinds of medical imaging, such as CT scans or MRI. The proposed method also has the potential to integrate additional individual models to better recognize disease from an input X-ray image. The proposed method addresses the classification problem. Therefore, we intend to investigate the effectiveness of the proposed method on various tasks, such as semantic segmentation or instance segmentation in medical images.