Uncertainty-Aware Multimodal Deep Learning for Morphologically-Ambiguous Biological Boundaries
Code: BBSRC-DFA_2026_01
Primary Supervisor: Oscar Maiques
Email: o.m.carlos@qmul.ac.uk
Institute: Barts Cancer Institute
Secondary Supervisor: Ziquan Liu
Email: ziquan.liu@qmul.ac.uk
Institute: School of Electronic Engineering and Computer Science
Abstract:
Biological boundaries where tissue states transition gradually - precancerous to neoplastic, developmental to mature, normal to pathological - pose fundamental computational challenges for image analysis. These morphologically ambiguous regions exhibit high inter-observer variability, making them ideal testbeds for uncertainty-aware AI methods.
Current deep learning approaches provide point predictions without quantifying when models are extrapolating beyond training distributions or encountering genuinely ambiguous biological states. This research gap leads to unreliable application of AI in spatial biology and potentially erodes trust in advanced AI models once there are severe outcomes caused by AI.
This project develops novel uncertainty-aware multimodal deep learning architectures that:
(1) separate aleatoric uncertainty (irreducible biological ambiguity from data) from epistemic uncertainty (model limitations resulted from the lack of data);
(2) dynamically weight heterogeneous data modalities based on local confidence; and
(3) generalize across distribution shifts through uncertainty-guided adaptation.
We address three algorithmic gaps: evidential deep learning for multimodal fusion, attention-based cross-modal calibration under uncertainty, and test-time adaptation strategies for tissue imaging.
Using skin epithelial transition (precancerous lesions to invasive melanoma) as initial validation with morphologically ambiguous boundaries, we integrate three complementary modalities: H&E whole-slide images, dermoscopy, and unstructured pathology reports.
Critically, the computational framework is designed for generalization beyond a static environment: we validate across multiple independent multimodal biomedical datasets sharing similar data structures, including pancreatic neoplasia (endoscopy + histology + reports), to demonstrate cross-anatomy transferability.
Deliverables include open-source software for uncertainty quantification in multimodal biomedical imaging, benchmark datasets for evaluating calibration under distribution shift, and validated methods applicable to organoid characterization, high-content screening, and any tissue imaging context where expert annotations show high disagreement.
This establishes generalizable principles for AI-driven analysis of complex biological transitions across BBSRC-relevant domains.
Lay Summary:
When biologists examine tissue images, some regions are clearly one cell type while others represent gradual transitions - developing embryos shifting between developmental stages, stem cells differentiating into specialized types, or early disease emerging from normal tissue. These boundary regions are difficult even for experts to classify consistently, yet understanding them is crucial for developmental biology, regenerative medicine, and disease detection.
Artificial intelligence could help by analysing multiple types of data simultaneously: microscope images showing cell appearance, photographs showing tissue structure, molecular measurements showing gene activity, and written descriptions from experts. However, current AI systems make predictions without a reliable confidence level. If an AI is uncertain because it has never seen a similar example before (model limitation) versus uncertain because the boundary is genuinely gradual (biological reality), scientists need to know the difference - but existing methods cannot tell them apart.
This PhD project develops new AI methods that explicitly measure and communicate different types of uncertainty when analysing biological images. The system will:
(1) identify which data sources are reliable for specific regions;
(2) weight information sources based on confidence; and
(3) work across different imaging equipment, tissue types, and experimental protocols.
We test these methods on skin tissue transitions from precancerous moles to early cancer and pancreatic lesion grading, where pathologists frequently disagree - but the computational approaches we develop will apply broadly to developmental biology (tracking cell fate decisions), organoid quality control (identifying abnormal regions), and high-content drug screening (detecting subtle phenotypic changes).
The goal is generalizable uncertainty-aware AI for any biological image analysis problem.
Aims and Objectives:
This research has three integrated aims spanning method development, adaptive fusion, and cross-dataset validation.
Aim 1: Develop uncertainty-aware multimodal fusion models.
We will build multimodal classifiers that output calibrated uncertainty, decomposed into biologically meaningful components. Specifically, we will replace softmax heads with evidential (Dirichlet) layers to estimate (i) aleatoric uncertainty, reflecting irreducible ambiguity at true biological transition boundaries, and (ii) epistemic uncertainty, reflecting model knowledge gaps that can be reduced with additional data. Each modality, H&E histopathology, dermoscopy or gross/endoscopic imaging, and clinical text, will produce its own evidential output prior to fusion.
We will evaluate calibration against standard baselines (temperature scaling, Monte Carlo dropout, deep ensembles) using Expected Calibration Error, reliability diagrams, and Brier scores.
Aim 2: Create uncertainty-guided, attention-based fusion.
We will design gated attention mechanisms that dynamically weight each modality using local uncertainty estimates. Modalities with high epistemic uncertainty will be down-weighted to reduce overconfident errors and overfitting, while modalities with low epistemic but high aleatoric uncertainty will retain influence to represent genuine biological indeterminacy.
We will propagate uncertainty through attention layers to generate spatial uncertainty maps that localize ambiguous regions and identify where specific modalities fail to provide reliable signal. Ablation studies will test when dynamic weighting outperforms fixed fusion, define thresholds for flagging cases for expert review, and assess how uncertainty correlates with inter-observer disagreement among pathologists.
Aim 3: Validate generalization under distribution shift across datasets.
We will evaluate on two independent multimodal cohorts: a primary melanoma cohort from HUAV (Spain; ~300 patients with dermoscopy, H&E whole-slide images, reports, and long-term outcomes) and a secondary pancreatic neoplasia cohort from Fudan University (Shanghai; endoscopic imaging, H&E biopsy histology, and reports for IPMN/PanIN).
We will quantify uncertainty under shifts in scanners, demographics, and agreement levels, and test uncertainty-triggered test-time recalibration. Cross-domain transfer (melanoma, pancreas) should show low accuracy but appropriately high epistemic uncertainty.