The clinical data of patients who underwent surgery between January 2012 and May 2021 at Zhongshan Hospital (an affiliate of Fudan University) were collected consecutively and analyzed retrospectively. The inclusion criteria were: 1. Patients has received preoperative BSGI as well as US; 2. The patient has received both breast tumour resection and axillary surgery in our hospital; 3. Postoperative pathologically confirmed diagnosis of breast cancer without neoadjuvant therapy; 4. No history of other tumours. Normal breast and lymph node ultrasounds imagines were excluded. The ethical approval of this study was granted by ethics committee of Zhongshan Hospital. All methods were carried out in accordance with relevant guidelines and regulations. Zhongshan Hospital Ethics Committee waived the need of informed consent from patients since it was a retrospective study. The working flow of our study was showed in Fig. 1.

Working flow of this study

Patients underwent BSGI (Dilon 6800; Dilon Technologies) at high-resolution and a small field-of-view. Imaging was performed 10–15 min after the intravenous administration of 740 MBq Technetium-99 m Sestamibi (GMS Pharmaceutical Co. Ltd) through an antecubital vein contralateral to the suspicious breast side. Craniocaudal (CC) and mediolateral oblique (MLO) images of both breasts were obtained. A low-energy general-purpose collimator was used, with a photopeak focused at 140 keV with a symmetric 10% window. The acquisition time was approximately 6 min per image and a value of 100 000 counts per image was defined as the minimum range.

Two experienced nuclear medicine physicians analyzed the images and were blinded to the patients’ clinical information and pathological results. According to the 2010 guidelines of the Breast Imaging Reporting and Data System (BIRADS) of the Society and Nuclear Medicine and Molecular Imaging [17], lesions with homogeneous and small patchy uptake were considered to be negative, lesions with patchy uptake, mild focal uptake and definite focal uptake were considered to be positive. The tumour-to-normal lesion ratio (TNR) was calculated by dividing the maximal pixel counts of the tumor lesion by that of normal background breast tissue on both CC and MLO view. A positive axillary mass was considered to be patchy, mild focal and definite focal uptake of Technetium-99 m Sestamibi in axilla. An example of BSGI imagines analysis was showed in Fig. 2.

An example of breast-specific gamma imaging analysis. (A) showed a left-sided breast cancer without axillary lymph node metastasis, the yellow rectangle showed the uptake of Technetium-99 m Sestamibi in breast. (B) showed a left-sided breast cancer with axillary lymph node metastasis, the red circle showed a positive axillary mass

The ultrasonographic images were obtained using an HDI 5000 scanner (Philips Medical Systems) and analyzed by 2 experienced operators. According to Adler’s method [18], the degree of blood flow signal within the breast carcinomas and axillary lymph nodes was subjectively classified into 1 of 4 levels: absent (grade 0), minimal (grade 1), moderate (grade 2), or abundant (grade 3). The echogenicity was classified as cystic (no echo), hypoechoic, isoechoic, hyperechoic and mixed echoic. When a mass showed echogenicity minimally less than that of subcutaneous fat, it was defined as hypoechoic. An example of ultrasonographic imagines analysis was showed in Fig. 3.

An example of ultrasonographic lymph node imagines analysis. (A) showed an ultrasound image of the right axilla exhibiting no signs of lymph node metastasis. (B) showed a color Doppler flow imaging (CDFI) image of the right axilla, indicating the absence of lymph node metastasis. (C) showed an ultrasound image of the left axilla showcasing lymph node metastasis, featuring a 24.0*16.0 mm mass identified as a lymph node. (D) showed a CDFI ultrasound image of lymph node metastasis in the left axilla, revealing discernible colored blood flow within the affected lymph node

The collected clinical and medical information of patients included the patients’ gender, age, breast tumour location, BSGI features (tumour TNR and axillary mass status), postoperative pathological features (estrogen receptor (ER) status, proliferation index (Ki-67), progesterone receptor (PR) status, Her-2 overexpression, lymphovascular invasion (LVI), Scarff-Bloom-Richardson (SBR) grade, T stage, N stage, infiltration depth, histologic type, molecular subtype, and multifocality) and ultrasonographic parameters of tumour and axillary lymph nodes (sizes, echogenicity, margin, lymph node hilum status, color Doppler flow imaging (CDFI) grade, and resistance index (RI)).

Clinical and pathological variables associated with the risk of lymph node metastasis were assessed on the basis of their clinical importance and predictors identified in previously published articles [19, 20]. Categorical variables, such as patients’ gender, tumor location, axillary mass status (BSGI), tumor echogenicity, tumor margin, tumor CDFI, lymphatic echogenicity, absence of lymph node hilum, lymphatic CDFI, infiltration depth, histologic type, SBR grade, ER status, PR status, Ki-67, Her-2 overexpression, molecular subtype, LVI, multifocality, T stage, and N stage, were reported as integers and proportions. On the other hand, continuous variables, including patients’ age, tumor TNR (CC), tumor TNR (MLO), transverse diameter of tumor, longitudinal diameter of tumor, tumor RI, transverse diameter of lymph nodes, and longitudinal diameter of lymph nodes, were reported as means with standard deviations. The association between clinicopathological characteristics and ALN status was analyzed using X2 test or t-test as appropriate. Collinearity for all explanatory variables were assessed using correlation matrix and plausible interaction terms were also tested. To relax the assumption of a linear relationship between continuous predictors and the risk of ALNs metastasis, continuous variables (such as the patients’ age, tumor TNR (CC), tumor TNR (MLO), transverse diameter of tumor, longitudinal diameter of tumor, tumor RI, transverse diameter of lymph nodes, and longitudinal diameter of lymph nodespatient) were converted into categorical variables after valuation using restricted cubic splines (RCS) [21]. Patients were randomly sampling into the training and test sets by ratio 7 : 3. Six machine learning (ML) methods were trained in, including generalized linear model (GLM), random forest (RF), support vector machines (SVM), neural network (NNET), gradient boosting machine (GBM), extreme boosting machine (XGB) [22,23,24]. To select the strongest predictive variables, recursive feature elimination (REF) was used for each algorithm. To avoid overfitting in training, the best hyper-parameter for ML models was 10-fold cross-validation. Comparison of ML methods performance, the best classification model was select. Based on the best-performing model, we created an online calculator that can make predictor in patients easily accessible to clinicians. Finally, the receiver operating characteristic (ROC) analysis was used to assess the role of BSGI feature in this prediction model [25]. All statistical analyses were determined using the R software (version 3.6.3, http://www.r-project.org). The R packages “caret”, “rms”, “glmnet”, “randomForest”, ‘nnet’, “e1071”, “kernlab”, “pROC”, “gbm”, and “xgboost” were used. The “shiny” package was used for web application. We used t-test for continuous variables and chi-square test for classified variables. A two sided P < 0.05 was considered statistically significant.