Colorectal cancer is the third most common cancer across genders and is the second most common cause of cancer deaths [1, 2]. Incidence has almost doubled in patients aged under 55 years since the mid-1990s. The proportion of rectal cancers has also increased from 27 to 31% during the same time-period [1]. Despite this, death rates have been declining at approximately 2% per annum since 2011 with improved awareness, screening, and pre-operative imaging, alongside advancements in surgical and systemic therapies all playing important roles [1,2,3]. However there still are inadequacies apparent around diagnosis and early stratification of patients regarding their optimal treatment modality. This is particularly evident for early-stage cancers and large pre-cancerous polyps. Clinician assessment, radiological staging, and endoscopic biopsy are limited for staging such lesions, with significant risk of over staging or false negative results [4,5,6,7]. Clinicians make treatment decisions, such as whether to perform endoscopic local excision including via Transanal Minimally Invasive Surgery (TAMIS), or radical resection based on these imperfect assessments. There remains a major role to be filled around accurately characterising such colorectal lesions.

Both near infrared (NIR) imaging systems and advanced computing power, including artificial intelligence (AI) methods, are becoming more widespread and accessible to the surgical community through the development of advanced imaging stacks. In this context, dynamic imaging at a molecular level, akin to that already established in radiology [8], is a potential step advancement in intraoperative imaging and surgical management of rectal cancer and significant rectal polyps. Here, we review current and emerging intraoperative NIR dynamic fluorescence imaging strategies that can aid in the surgical management of rectal cancer with a focus on real-time digital analysis of Indocyanine Green (ICG).

Current state of the art re surgery for significant rectal polyps and tumours

Currently, rectal neoplasia is most often diagnosed on the basis of colonoscopy after either patient reported or screen detected discovery of blood in the stool. Aside from direct visualisation and some specific mucosal surface light-based spectral analysis (which is user dependent) at the time of endoscopic identification, tissue samples are needed to confirm significant lesions (i.e. those > 2 cm) as benign or malignant. Endoscopic biopsies are largely limited to the exposed surface area, and suffer sampling error in approximately 20% of cases, if not more [6, 7, 9, 10] as cancerous transformation often occurs alongside and within areas of preceding benign adenoma. Tissue biopsies are also unreliable for ruling out the presence of residual malignant cells following neoadjuvant therapy [11, 12]. The act of biopsy may also induce fibrosis, potentially frustrating subsequent local excision, particularly endoscopic submucosal dissection [13]. When it comes to treatment, full and partial thickness local excision techniques are available. While patient selection is of course key to ensure oncological outcomes are optimised, it is currently not possible to do so with perfect accuracy (additional radiological assessment is only 50% accurate in this regard). While some cancer presence may not automatically preclude local excision (many T1 cancers are curable using such techniques), the plane of excision (whether submucosal or full thickness) and indeed the provision of an intact specimen is crucial in ensuring sufficient marginal clearance has been achieved. In other cases, when local excision is performed for seemingly benign lesions or indeed even apparent T1 cancers, pathological examination of the excision specimen may only then uncover a more advanced cancer which was not obvious beforehand due to inexact staging. Any such cancers may then require radical surgery with or without preceding neoadjuvant treatment. The adverse effect of an additional surgery is clear for the patient, who must come to terms with their initial surgery being unsuccessful and may have already suffered morbidity despite the minimal access approach. There are increased costs for the healthcare system and the potential increase in complexity of the subsequent operation. “Salvage surgery” following unsuccessful local excision may also be associated with poorer outcomes compared to radical resection in the first instance [14], and is associated with higher rates of non-sphincter-preserving surgery [15]. Marginal involvement after local excision exists in up to 20% of lesions treated with local excision which can lead to regrowth of both benign and malignant lesions. For these reasons, there is much interest in providing better tissue characterisation for significant rectal polyps ideally in situ and without tissue disruption. With the knowledge that aberrant vasculature is an early, hallmark change in the transformation of benign to malignant disease, perfusion based analytical methods central to the now established field of image guided surgery (and more significantly fluorescence guided surgery, FGS) could provide the key to better endoscopic tissue discrimination especially when linked to sophisticated computational analytical methods.

Indocyanine green (ICG) – the archetypal fluorophore for FGS

Indocyanine green (ICG) is a water-soluble fluorescent dye which binds to plasma proteins, most significantly albumin. Developed by Kodak Research laboratories in 1955, it was approved for human use in 1956 and has been used for retinal angiography since the 1970’s [16, 17]. Since then, it has been adopted for various intraoperative uses across a variety of surgical specialities [18,19,20]. Across these specialties, ICG angiography is dependent on the compound in plasma absorbing NIR and back-emitting fluorescent light of distinctive wavelength (830 nm). ICG is hepatically metabolised, with a half-life of 3–4 min, depending on liver function and it has an excellent safety profile [19]. All major surgical camera manufacturers now offer the capability for near-infrared light illumination and detection.

Currently, within colorectal cancer surgery, ICG with NIR has several established uses with perfusion assessment being particularly noteworthy. Despite recent advances in technology and technique, anastomotic leakage (AL) remains a significant morbidity following the resection and re-joining (anastomosis) of segments of the large intestine, with tissue malperfusion being a significant contributor to this complication [21, 22]. Given ICG’s affinity for albumin after systemic administration, tissue perfusion is visually reflected by the presence of ICG-related fluorescence within whatever tissue is being observed with NIR [23]. The surgeon then makes a judgement based off observation of the rate of inflow into the tissue area of interest. Such evaluation may reassure the surgeon that the tissues under observation are sufficiently perfused and suitable for selection as the point of anastomosis. Alternatively, it may encourage a change in the point of resection/anastomosis to a segment of bowel which exhibits better perfusion based on the surgeon’s own interpretation of ICG flow [24, 25]. Recent meta-analyses suggest a significant reduction in AL rates when ICG is used for anastomotic assessment in this way [25, 26]. This finding has now too been borne out in a large randomised multicentre clinical trial [27] with other major studies also anticipated to report soon [28]. ICG has also been investigated as a lymph node mapping agent after endoscopic submucosal injection [29] but its clinical value in this remains investigational as the crucial concern in present day colorectal cancer surgery is detection of malignancy rather than demonstration of lymphatic physiology alone. Decisions based on ICG perfusion patterns have to date largely been dictated by qualitative assessment undertaken by the surgeon. However rapid technological advances of the 21st century have presented the possibility of quantifying fluorescent signals from ICG allowing enhancement of intraoperative decision making. This has been shown in neurosurgery and plastic surgery where ICG perfusion quantification, coupled with deep learning methods, have produced intraoperative heatmaps capable of providing intraoperative diagnoses and influencing decision making [30,31,32,33].

ICG for intraoperative cancer imaging – current state of the art

In tandem with the above clinical applications, clinicians realised that ICG given systemically also tends to be trapped in malignant deposits leading to many reports regarding its use as a cancer localisation agent for surgery. The 2009 study by Ishazawa et al., first showed nonselective staining and retention of ICG in hepatocellular carcinomas (HCC) and colorectal liver metastases (CRLM) in patients undergoing routine assessment of liver function via fluorescent imaging [34]. Subsequent to this discovery, antecedent dosing (injection before surgery) has been utilised across several specialties for tumour detection including expanded experiences in CRLM [35], HCC [36] and sarcoma [37], amongst others [38, 39]. It has also been used for margin assessment in breast surgery [40, 41].

While the exact mechanism of ICG uptake is unclear, malignant cells are generally avid consumers of substrates in their vicinity with various nonselective cellular and molecular mechanisms that predispose to the relative retention of ICG (and other substances) in comparison to adjacent normal tissue. Especially in non-hepatic tissue sites (the liver is an active ICG concentrator and its mechanisms for ICG trapping around liver lesions are different such as including the compressed zone around CRLM), the enhanced permeability and retention (EPR) effect [42] is the most commonly attributed pathway, especially when ICG is observed within tumour stroma [43,44,45]. However, alternative mechanisms likely play key roles as well. A group in Imperial College London has demonstrated the dynamic nature of ICG uptake in patients undergoing breast conserving surgery for breast cancer. ICG administered at 5 min prior to excision demonstrated a significantly superior fluorescence signal within excised specimens when compared to that administered 25 min prior to excision. One potential reason for this finding is higher capillary volume within this tumour and higher intravascular ICG concentration shortly after infusion [46]. Active, relatively nonselective cellular processes, such as clathrin-mediated endocytosis and tight-junction regulation, also play roles in tumour retention of ICG via significant malignant cell internalisation over time after initial dosing [47, 48]. As a side point, the existence of these non-selective uptake mechanisms may also account for a proportion of the uptake of proposed, selective receptor-targeted fluorophores in development.

Whilst cancer sensitivity with such an approach has been impressive, specificity issues are troublesome with antecedent dosing. ICG is prone to trap in non-malignant pathologies (such as inflammation, fibrosis, bleeding [49, 50]), and even non-pathological tissues (such as fat). The resulting false positive rates, along with the workflow impact necessary for compound administration many hours in advance of the planned surgery, limit the widespread adoption of this method. In addition, with all point in time tumour localisation, timing of ICG administration must be so ICG concentration within the region of interest is at its maximum relative to surrounding tissues and significantly different to any sites of non-malignant trapping [51]. However, exact timings of operations and, especially, the exact timing of intraoperative encounter with the site being labelled by ICG can be uncertain.

Dynamic ICG cancer perfusion patterns and curve analysis

Prior to any uptake into malignant tumours and cells, ICG must be delivered to the lesion via regional blood supply. Distorted and disrupted tissue perfusion is a hallmark characteristic of cancer in general and occurs as an early feature of malignant transformation [52,53,54]. This provides an important point of differentiation in comparison to surrounding normal tissue. Both malignant and pre-malignant colorectal lesions are also characterised by distinctive angiogenic features such as increased microvascular blood content, irregularity of the microvasculature and increased microvascular volume [55,56,57]. The progressive architectural distortion with tumour neo-vasculature consists of precocious capillary sprouting, convoluted and extensive vessel branches, and discontinuous and wider endothelial junctions between cells. These abnormalities result in irregular flow patterns, higher intravascular pressures, and extravascular leakage [58,59,60,61]. Sensitive determination of tissue microvasculature using ICG can visually discriminate these differences dynamically and so differentiate invasive from non-invasive neoplasia [62] (and more generally healthy tissue from non-healthy tissue) as each tissue type exhibits a specific vascular pattern based on its architecture (see Fig. 1). Empirical evidence for this has been seen by several groups [63,64,65]. We ourselves have previously reported on in-vivo assessment of primary colorectal cancers following intravenous administration of ICG. When observed continuously in situ for up to ten minutes, the rate of inflow in cancerous tissue is slower relative to adjacent normal tissue [66]. Active and passive nonspecific processes as detailed above likely play additional roles in tumour retention after initial perfusion [42, 47,48,49] and result in differential and in fact discriminant inflow/outflow behaviours. While such behaviours may be observable to the naked eye, ICG flow through benign tissue may appear similar making it difficult and burdensome to make an accurate prediction from visual assessment alone. Such behaviours therefore are better captured by quantitative time series extraction as illustrated in Fig. 2. The differences in the curves produced from ICG perfusion analysis of rectal cancer may be subtle. Hence, their analysis and tissue status prediction is best carried out by machine learning classification algorithms. This differs to other examples of dynamic perfusion analysis where differences are clearer to the naked eye [67,68,69]. The same underlying temporospatial patterns related to ICG flux are observable in microscopic sections taken at different time points after ICG administration when examined by near infrared microscopy alongside standard pathological staining (see Fig. 3, and note these images also illustrate how the physiological processes involved are not entirely explainable by EPR alone).

Fig. 1

Schematic showing time series changes in ICG after systemic administration in both normal tissue versus a malignant focus in rectal tissue. Among other factors, cancer angiogenesis results in distorted tissue architecture including increased capillary volume and permeability which leads to increased ICG leakage and decreased clearance versus what happens in normal and purely benign lesions. This results in retention of ICG within the malignant core of the polyp when levels have otherwise significantly decreased in normal and benign tissues

Fig. 2

Indicative Progression of ICG fluorescence in a malignant lesion (red border) and health control (green) with a corresponding time series intensity quantification taken from a representative region within the abnormal area compared to a region from the adjacent healthy tissue (interestingly this lesion was initially judged benign by the endoscopist based on clinical impression). Fluorescent appearances are quickly distinctive for malignancy which was confirmed on pathology after its excision. Image A shows the subtle area of abnormality seen endoscopically. Image B at 7 s from ICG dosage shows nuanced lack of uptake in the tumour relative to the surrounding healthy tissue. At approximately 30 s image C shows that the fluorescence of the malignant tissue has now matched or exceeded that of the healthy tissue. Image D at 3 min shows significant retention of dye relative to the washout of the healthy tissue. These same trends are more evident in the time series graph (image E) demonstrating delayed ICG entry and clearance in the cancerous tissue. Curve features for each case are extracted and serve as the data inputted into machine learning classification pipeline which produces a prediction of “cancer” or “benign”

Fig. 3

Frozen tissue samples of colorectal cancer (A) 15 min and (B & C) 2 h following systemic indocyanine green injection. Hematoxylin and Eosin stained (left) and microscopic fluorescence imaging of unstained tissue (right) demonstrate a predominance of ICG within stromal tissue and vasculature and a relative lack of ICG within neoplastic malignant glands (circled) at the early timepoint (A) Trapping of ICG within the benign: malignant tissue interphase is seen at the later time points B & C. ICG is also noted to sporadically trap within non-malignant tissues adjacent to cancer, likely as a result of distorted architecture from the nearby malignant process (similar levels of distortion can also be seen in non-malignant inflammatory tissues resulting in false positive appearances associated with single point-in-time ICG assessment). A Nikon Eclipse Ti2 Inverted Research Microscope and a LI-COR Odyssey DLx Near-Infrared Fluorescence Imaging System were used to analyse obtained samples. Specimens were mounted in OCT, flash frozen using Lamb’s freezing aerosol and cut using a cryotome to 5 micrometre thick levels

Dynamic perfusion-based imaging in radiology

Characterisation of lesions in-situ by their perfusion patterns is an established concept in radiology that has been exploited in dynamic contrast imaging for decades, particularly Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). Dynamic contrast enhanced MRI (DCE-MRI) is utilised across a wide variety of organs including breast, prostate, and liver with high levels of accuracy for cancer detection. Particularly in liver malignancies, DCE-MRI accuracy has reached levels that often preclude the need for biopsy [70, 71]. Focussing on the transfer constant (KTrans) in DCE-MRI for rectal cancers, we can see that it correlates with angiogenesis, vascular endothelial growth factor levels, tumour differentiation, tumour aggressiveness in rectal cancers and may predict response to neoadjuvant therapies [72, 73]. Recently, Muto et al. reported that KTrans levels have also shown a correlation with increased ICG retention in pituitary tumours and intracranial meningiomas [74, 75]. The cell level factors which dictate KTrans levels in DCE-MRI are the same factors responsible for ICG pooling in rectal cancers, namely permeability and angiogenesis. We have shown that intraoperative tissue characterisation is possible based on interpretable biophysical parameters found in association with these factors [76]. However, there are important differences between the curves produced by DCE-MRI and ICG fluorescence angiography. Intes et al., demonstrated a delayed inflow and outflow of ICG through breast malignancy in comparison to nearby healthy tissue in a similar fashion to that which we have observed in rectal cancers [77]. Such findings, however, are in contrast to those seen in DCE-MRI for breast and prostate cancer where, malignancies are generally seen to peak earlier and wash out more quickly than nearby healthy tissue (Type 3 curves) [68, 69]. This rapid wash-out phenomenon seen in DCE-MRI has been proposed as being secondary to arteriovenous anastomoses causing rapid outflow and therefore a reduction in contrast signal [78], although to our knowledge this has not been proven.

Intraoperative artificial intelligence (AI) decision support using ICG perfusion characterisation

The biophysical model detailed above is focussed on temporospatial fluorescent patterns and forms the basis of our methods. With this we have begun developing a computational analysis method as a means of real time tissue characterisation potentially obviating forceps biopsy and enabling margin delineation. In comparison to standard biopsies and histopathological analysis, what is exciting about ICG fluorescence signals is the production of an in-surgery assessment of the entire tumour (including some millimetres below the mucosal surface). Initially such an ICG perfusion-based decision support method could be used to direct tissue biopsy to the area of abnormality most likely to contain any cancerous component and thereafter, should AI based digital assessment outperform endoscopic biopsy in ongoing validation studies [79, 80], the need for tissue sampling could be avoided. Even just equalling the accuracy of biopsy would be a significant advancement, providing a similarly accurate prediction to patients on the day of their investigation. In a similar manner, an assessment of overall tumour size and on-screen margin guidance would be greatly beneficial for endoscopic mucosal, submucosal and full thickness local resections, encouraging optimised dissection at the index procedure and reducing involved margin status [81,82,83].

So far, our methods have produced accuracy results in the region of 90% for cancer discrimination in significant (i.e. those over 2 cm in diameter) rectal polyps and small cancers [76, 84]. To do this, ICG curve features based around parameters reflective of the wash in and wash-out phases