Artificial intelligence-enabled advanced endoscopic imaging to assess deep healing in inflammatory bowel disease

Ultra-high magnification endoscopy

Ultra-high magnification endoscopes have emerged with the concept of optical biopsy, which involves using light properties for real-time in vivo visualisation and evaluation of histological features. Currently, the two main ultra-high magnification endoscopic tools employed in clinical practice are EC (CF-H290ECI; Olympus. Japan) and pCLE (Cellvizio; Mauna Kea Technologies, Paris, France).28 35 EC uses a 520× magnification microscopic lens mounted on the tip of a flexible speculum and provides colourful images through prestaining. In contrast, pCLE employs a probe with a 1000× magnification microscopic lens inserted through a forceps hole and presents a monochromic image.36 One distinct advantage of ultra-high magnification endoscopy is the introduction of novel indicators that could not previously be observed with conventional image-enhanced endoscopy. Examples include nuclei and goblet cells with EC37 38 and fluorescein leakage, shedding of cells, crypt morphology and cellular infiltration with pCLE.28 39 These techniques enable the accurate assessment of mucosal healing and delve deeper into the evaluation of remission, advancing towards the concept of complete healing.

Endocytoscopy

Observation with EC requires prestaining with 1.0% methylene blue (±0.05% crystal violet). Several scores have been introduced since its development to standardise mucosal healing assessment. In 2011, Bessho et al proposed a pioneering endocytoscopy scoring system (ECSS) to assess mucosal healing in UC.40 The ECSS assesses the shape, the distance between gland lumens and the visibility of blood vessels, with a score ranging from 0 to 6. ECSS showed a strong correlation with histological activity according to Matts’ score.40 The clinical relapse rate was reported to be 7.4% (2/27) for patients with ECSS=0 compared with 21.6% (8/37) for patients with ECSS ≥1 during an average follow-up period of 3 years.41

Starting from the ECSS, Iacucci et al proposed their ECSS. This score predicted histological remission with a sensitivity, specificity, observed agreement and AUROC of 64%, 86%, 79.5% and 0.81, respectively, allowing us to conclude that EC is a promising tool moving closer to histology.15 Furthermore, Vitali et al developed and prospectively validated a new endocytoscopic score (ErLangen Endocytoscopy in ColiTis score) for the in vivo assessment of microscopic disease activity. This novel and intuitive score demonstrated superiority to HD-WLE, with a sensitivity of 88%, specificity of 95.2% and AUROC of 0.92, and a strong correlation to validated histopathological scoring in predicting adverse outcomes.42

As a simpler unstained EC-based indicator, microscopic vascular images obtained using NBI were proposed.43 44 Maeda et al proposed the intramucosal capillary/crypt (ICC) index to assess intramucosal vascular atypia and structural changes in the glandular lumen. During a median follow-up of 16 months in 208 patients with UC in clinical remission, they reported a relapse rate of 5.6% in patients with an ICC index ≤1, compared with 30.5% in patients with an ICC index ≥2,45 similar to stained EC observations. Furthermore, the NBI method reduced the examination time: an average of 44 s to obtain a single biopsy tissue and 130 s to take five EC images using the stained method, whereas 19 s to take five images using NBI.45

Notably, EC’s ability goes beyond the assessment of mucosal healing to the concept of barrier healing. Iacucci et al have recently shown that EC alone and combined with multispectral imaging analysis of intestinal barrier proteins can accurately predict adverse outcomes in IBD.46

Probe-based confocal laser endomicroscopy

The pCLE requires administration of an intravenous fluorescein contrast and enables the assessment of the structure and function of the intestinal barrier. A first prospective study assessed small bowel epithelial gap density by pCLE, showing significantly higher barrier damage in patients with IBD than in controls (p<0.01).47 Similarly, substantially more epithelial gaps, epithelial cell shedding and fluorescein leakage have been described in the duodenal lumen of patients with IBD compared with controls, with similar changes in CD and UC.48

Notably, Kiesslich et al proposed a score for barrier dysfunction in IBD based on the amount of cell shedding and the intensity of the luminal fluorescein signal. Three grades have been described, that is, normal function (grade I), functional defect (grade II) and structural defect (grade III). A Watson score II/III predicts significantly a flare-up throughout a 12-month follow-up (p<0.01), with a sensitivity, specificity and observed agreement of 62.5%, 91.2% and 79%, respectively.49

The Endoscopic Remission, Histological Remission and Barrier Healing for Predicting Disease Behaviour in IBD trial recently demonstrated that pCLE identifies barrier healing and is superior to endoscopic and histological remission for predicting MAO. Indeed, pCLE predicted MAO-free survival with an observed agreement of 72.7% in patients with UC and 88.7% in patients with CD, higher than ER (70.4% in UC and 54% in CD) or HR (66.7%–69.1% in UC and 56% in CD).14 These results paved the way for the new ‘third dimension’, that is, intestinal barrier healing, which could potentially be a new promising treatment target in IBD.

Delving deeper, pCLE has also been employed for molecular imaging using fluorescent antibodies, offering a potential approach to identify response to therapy in patients with IBD. Atreya et al developed a fluorescent antibody designed for the molecular imaging of membrane tumour necrosis factor (mTNF) in IBD. The topical administration of this antibody to 25 individuals who have CD facilitated the identification of mTNF-positive immune cells within the intestine via confocal laser endomicroscopy. Notably, subjects exhibiting elevated levels of mTNF-positive cells demonstrated a significantly enhanced short-term response rate (92%) to subsequent anti-TNF therapy at the 12-week mark, in contrast to those with lower levels of mTNF-positive cells (15%).50 Similarly, Rath et al51 used molecular imaging to detect α4β7 integrin expression in patients with CD, showing that responders to treatment had pericryptal α4β7⁺ cells compared with non-responders. More recently, Iacucci et al have fused pCLE with OMIC (‘endo-omic approach’), going deeper into molecular healing and predicting response to therapy in IBD. They integrated computer-assisted image analysis of pCLE with RNA transcriptomics.52 In their study, 29 patients with IBD were enrolled to assess the predictability of their response to infliximab and vedolizumab treatments. In vivo, vessel tortuosity, crypt morphology and fluorescein leakage predicted response to therapy in both CD and UC (79% area under the curve (AUC) and 80% accuracy and 93% AUC and 85% accuracy, respectively). Moreover, ex vivo analyses showed an increased baseline binding of fluorescent-labelled biologics correlated with a greater probability of treatment success, especially in UC (83% of AUC; accuracy 77%). Furthermore, a set of differentially expressed genes, including ACTN1, CXCL6, LAMA4, EMILIN1, CRIP2, CXCL13 and MAPKAPK2, demonstrated potential in forecasting responses to anti-TNF therapy (AUROC >0.7).

In conclusion, the ultra-high magnification endoscopy enables a real-time in vivo comprehensive dynamic assessment of disease activity and response to therapy, reaching the barrier and the molecular healing and having great potential in predicting outcomes and guiding management in patients with IBD. These studies are only performed in expert centres. Although image acquisition with these ultra-high magnification endoscopies, contact-type microscopy, is not technically difficult, training is required to interpret the images. Additionally, the need for fluorescein and the probe’s cost in pCLE are obstacles to clinical practice. An overview of the main studies investigating the role of advanced endoscopy in assessing inflammation and predicting clinical outcomes in IBD is shown in table 1.

Challenges and limitations of current and advanced endoscopic techniques

Current advances in endoscopic techniques allow a detailed and enhanced evaluation of mucosal and vascular architecture of intestinal mucosa, highlighting features suggestive of subtle inflammation. However, assessing features related to disease activity and their integration into currently available scoring systems is burdened by subjectivity, high inter-observer variability and lack of knowledge and training.17 Furthermore, experts use these techniques and the related scores predominantly. A learning curve is possible in advanced endoscopic diagnostics, but it is necessary to standardise training methodologies to integrate novel diagnostic approaches into regular practice.53 Moreover, the cost-effectiveness of advanced endoscopic techniques still needs to be elucidated. Finally, applying advanced endoscopic techniques is anticipated to diminish the frequency of biopsies required for inflammation evaluation. However, verifying that this reduction does not lead to an escalation in examination duration and expenses is imperative.

AI in ulcerative colitis

In 2019, Ozawa et al used a convolutional neural network (CNN) enhanced by deep learning techniques to innovate a computer-aided diagnosis (CAD) system that, benchmarked against expert diagnoses, was capable of accurately categorising 73%, 70% and 63% of the MES0, MES1 and MES2–3 images, respectively.56 Concurrently, Stidham et al engineered a CNN model providing a four-tier classification of UC activity (MES 0–3), with commendable predictive accuracy in distinguishing between MES.57 Moreover, Takenaka et al devised a deep neural network for UC assessment, which could predict endoscopic remission with 90% accuracy.58 Furthermore, their algorithm demonstrated a sensitivity of 92% and a specificity of 94% in predicting histological remission from endoscopic images.58 This result was confirmed in a multicentre prospective study, where the algorithm, updated for video analysis, exhibited an enhanced sensitivity of 98% and a specificity of 95% in predicting histological remission.59

Nonetheless, these studies relied on still images, raising concerns about selection bias. This prompted further research using videos to mitigate subjectivity and enhance the reliability of evaluations. To address this issue, Yao et al explored the application of their CAD system to unedited videos, developing an algorithm capable of autonomously generating a summary score for each colonoscopy.60 Their AI model could automatically omit unsuitable images to enhance diagnostic precision. Similarly, Takenaka et al adapted their model to video analysis, demonstrating its ability to identify endoscopic remission correctly and strongly correlate with central reader assessments.59

Recently, three systematic reviews and meta-analyses on this topic have been published.61–63 Their results revealed that the AI model demonstrated high accuracy in diagnosing endoscopic remission, as defined by both MES and UCEIS, for still images and video-based evaluations. However, these meta-analyses also showed a moderate to high heterogeneity among the targeted studies and a risk of bias. Interpretation of the results will have to await the outcomes of higher quality studies.

AI-assisted colonoscopy can also provide a more comprehensive approach to assessing the distribution of inflammation, having the ability to evaluate the patchiness of the disease and the full extent of inflammation. This approach can have significant implications for treatment choices, where understanding the full extent of inflammation is crucial. Innovations by Takabayashi et al and Fan et al have led to scoring systems considering the overall distribution of inflammation, providing a more detailed disease activity profile.64 65 The introduction of the Cumulative Disease Score by Stidham et al represents a significant leap forward. Their computer vision methods, applied to endoscopic videos from the UNIFI clinical trial, which assessed the efficacy of ustekinumab as induction and maintenance therapy in patients with UC, were able to quantify mucosal injury details better compared with MES and were more precise in discerning treatment efficacies. These results confirmed the potential of AI to significantly enhance the management of UC in both clinical trials and practice.66 The application of AI in clinical trials extends beyond image analysis, addressing time and cost implications of endoscopic disease activity scoring. For instance, Gottlieb et al used data from a phase II mirikizumab trial to train and validate a CAD system autonomously scoring endoscopic severity, demonstrating high accuracy rates for endoscopic remission and promising to standardise and expedite central reading in clinical trials.67 Furthermore, accurately defining the distribution of disease will allow for more precise IBD-related colorectal cancer screening intervals, potentially leading to a reduction in IBD-related colorectal cancer by ensuring timely detection and intervention.

AI in Crohn’s disease

In CD, AI has been mainly applied in reading capsule endoscopy (CE), a non-invasive diagnostic technique crucial for examining the small bowel.68 CE proves critical for evaluating disease activity, discerning mucosal inflammation and identifying complications such as strictures and fistulas. However, interpreting CE images is time-consuming and subjective, relying on physicians’ expertise.

Recent years have witnessed substantial progress in the fusion of AI with CE, now capable of analysing CE images to identify mucosal lesions indicative of IBD, including ulcerations, erosions and strictures. Klang et al developed an AI algorithm to recognise ulcers and strictures with an AUC of 0.94 and 0.99, respectively.69 70 Furthermore, Majtner et al showed that their AI system could categorise lesions into four types—normal mucosa, aphthous ulcerations, ulcers and large ulcers/fissures—with a substantial degree of agreement with experienced gastroenterologists (κ=0.72).71 Such automated lesion detection systems can facilitate early diagnosis and monitoring of disease progression. Moreover, AI-enhanced CE holds promise for prognostic prediction, as demonstrated by Kellerman et al, who showed that their algorithm could predict the need for biological therapy within 6 months for patients newly diagnosed with CD, achieving an accuracy rate of 81% and an AUC of 0.86, outperforming human assessment and faecal calprotectin levels.72

Finally, Brodersen et al undertook a multicentre investigation employing AXARO for pan-enteric CE analysis in patients with suspected CD.73 Their findings indicated that observers attained a 92%–96% sensitivity and 90%–93% specificity in detecting CD, alongside a 97% sensitivity and 90%–91% specificity for IBD detection overall. The use of AXARO notably reduced the initial review duration to a median of 3.2 min.

AI-aided endoscopy enables objective and accurate assessment of mucosal healing in both UC and CD, showing potential to standardise and expedite patients’ management in clinical trials and clinical practice. It holds promising abilities in disease monitoring, stratifying future relapse risk and predicting response to therapy. Table 2 summarises key studies exploring the role of AI-enabled endoscopy in assessing endoscopic healing in IBD.