Bladder cancer is the most common tumor disease of urinary system. Accurate segmentation of bladder tumors from medical radiographic images is of great significance for early detection, diagnosis and prognosis evaluation of bladder cancer. Due to the advantages of Deep Convolutional Neural Networks (DCNNs) in image analysis (Dai et al., 2022, Dou et al., 2016, Ross et al., 2021, Li et al., 2020a), in recent years, DCNNs have been successfully applied to analyze Magnetic Resonance (MR) images for bladder tumor detection (Cha et al., 2017, Shkolyar et al., 2019), segmentation (Ma et al., 2019, Li et al., 2020b) and auxiliary diagnosis (Garapati et al., 2017, Gosnell et al., 2018).

Deep convolutional neural networks rely on a large amount of annotated image data for model training. Directly applying deep convolutional neural networks for image segmentation is a complete data-driven process. Ge et al. (2021) and Dolz et al. (2018) utilized the dilated convolution to implement the accurate segmentation of bladder tumors. But the pure data-driven segmentation method over depends on data and lacks domain knowledge and model interpretation, which may lead to poor segmentation performances in the case of limited labeled data (Chaudhary et al., 2021).

In order to reduce the data dependency and improve the model interpretability of the DCNN-based medical image segmentation, researchers tried to incorporate the clinical priors into DCNNs to construct data-knowledge fusion segmentation methods (Yue et al., 2019, Oda et al., 2018, Kervadec et al., 2019, Mirikharaji and Hamarneh, 2018, Yan et al., 2020, Huang et al., 2021). For bladder tumor segmentation, Huang et al. (2021) formulated the clinical priors of bladder tumor size and locations with the attention mechanism and integrated the priors into DCNNs to guide the tumor segmentation. Li et al. (2020b) utilized an autoencoder network to learn the semantic features of bladder as priors and then incorporated the priors into the segmentation network. In contrast to data-driven segmentation methods, data-knowledge fusion methods incorporate clinical knowledge into the bladder tumor segmentation and are less dependent on labeled image data.

Although incorporating clinical knowledge into DCNNs is helpful to improve the medical image segmentation, the extant data-knowledge fusion methods still suffer two drawbacks. First, the knowledge are represented in the forms of parameter setting, optimization constraints, attention mechanism and probability distribution, which are difficult for knowledge acquisition and understanding by clinicians. For example, it is difficult for human doctors to provide a clinical knowledge as the optimization constraints of image segmentation model. Second, the extraction of knowledge may still require abundant labeled data, such as the priors of attentions and probability distributions learnt from massive labeled images.

To address the shortcomings of knowledge representation, we propose a novel bladder tumor segmentation method in this paper, which incorporates the logical rules of clinical knowledge into DCNNs. Logical rules provide a semantic and human-readable representation of knowledge, as a natural way of human thinking. It has been shown that symbolic logical rules can be effectively integrated into deep neural networks to improve the performances of image classification (Xie et al., 2019, Dash et al., 2021) for bladder tumor staging (Zhang et al., 2020), but the methods cannot be directly used for image segmentation tasks. To involve clinical logical rules into DCNNs for image segmentation, we first extract clinical knowledge of bladder tumors and bladder walls in the form of logical rules from the segmentation masks of ground-truth. Then we train a Graph Convolutional Network (GCN) (Kipf and Welling, 2016) to embed logical rules into the latent feature space. Afterwards, we reformulate the loss function of the segmentation network to minimize the difference between the logical rule embedding of the ground-truth segmentation mask and the predicted segmentation mask. Based on the operations above, the logical rules are incorporated into DCNNs to guide the tumor segmentation.

To the best of our knowledge, there are very limited works on incorporating logical rules of clinical knowledge into DCNN for medical image segmentation. Compared with other knowledge representations, logical rules of clinical knowledge are more semantic and human-friendly for clinical doctors. In addition, incorporating logical rules of clinical knowledge helps to reduce the data dependency of the segmentation network, and enables precise segmentation results even with limited labeled images. Experiments on bladder MR images collected from the collaborating hospital validate the effectiveness of the proposed tumor segmentation method. Our contributions are summarized as follows.

Formulate clinical logical rules of bladder tumor localization. We construct the existence and spatial propositions of bladder tumor and wall. According to the law of bladder tumor growing close to the bladder wall, we connect the propositional variables by logical connectives to form the logical rules of tumor location.

Propose a novel bladder tumor segmentation method through involving clinical logic rules into DCNNs. We use GCN to embed the clinical logic rules into the latent feature space and reconstruct the loss objective of DCNNs to harness the tumor segmentation to be consistent with both image annotation and logical rules. The proposed method is much less dependent on data and can achieve precise segmentation on limited number of images.

In addition, as the extension version of the published conference paper (Huang et al., 2022), the novel contributions of this journal version include four parts:

We give a detailed illustration of the extraction and vectorization of clinical logic rules, which play a crucial role in our approach but are not mentioned in our conference paper.

We propose a novel data augmentation method for pre-training GCN. In our method, GCN is used to determine whether the extracted vectorized logic rules from segmentation mask satisfies the ground-truth logic rules and thereby guiding the training process of the segmentation network. To pretrain the GCN, we enumerate all possible combinations of propositional variables using the truth-table to generate training data for the GCN.

We conduct ablation studies to visualize and analyze the effects of different loss terms based on logic rules.

We conduct additional comparison experiments with other state-of-the-art prior-based segmentation methods to verify the superiority of our approach.