Radiology scans and pathology slides each capture something different about a tumor. Reading them together can provide a more comprehensive picture than either alone.
This study introduces MuTriM, a deep learning model that combines longitudinal DCE-MRI scans with whole-slide pathology images of tumor cell structure. It uses an "attention-based" design that learns which features matter most across both data types and across time. MuTriM predicted which patients were likely to have their cancer return and which were likely to benefit from radiation after surgery, outperforming models built on imaging or pathology alone.
MuTriM was prognostic of recurrence-free survival subtype-agnostically (HR=5.26, 95% CI 1.69-16.4, p=0.004; C-index 0.75), beating DCE-MRI-only (0.65) and pathology-only (0.70) models, and predicted radiation benefit (interaction p=0.04).
Transcriptomic analysis linked high-risk tumors to suppressed cytotoxic T-cell activity and ECM remodeling, giving the fused radiology-pathology framework an interpretable biological grounding for prognosis and treatment guidance.
Read the full paper HERE.
