Immunotherapy

Optimizing Targeted Immunotherapy and Response by Correlating TMB Estimation and Spatial TILs Mapping 

Conventional methods for estimating Tumor Mutational Burden (TMB) and neoantigen load, such as Whole Exome Sequencing (WES) and other genomic techniques, provide a comprehensive analysis of the mutations present in tumor DNA. These methods are crucial for calculating TMB and identifying neoantigens, which are new antigens formed due to these mutations. However, integrating H&E TILs mapping and spatial transcriptomics can complement these genomic techniques by providing spatial context. H&E TILs mapping offers a visual representation of TIL distribution within the tumor tissue, while spatial transcriptomics links gene expression data to specific anatomical locations. This combined approach enhances the accuracy of TMB estimation and provides a more holistic view of the tumor's immunogenicity.


The importance of H&E TILs mapping and spatial omics in understanding TIL activities cannot be overstated. Mapping TILs in the tumor microenvironment (TME) using H&E histopathology provides insights into the immune response within the tumor. When combined with spatial transcriptomics, this approach reveals how TILs interact with other cells and structures within the tumor, identifying potential targets for enhancing TIL function. This integrated strategy not only improves the estimation of TMB and neoantigen load but also informs the development of more effective TIL therapies. By leveraging the strengths of both H&E TILs mapping and spatial omics, researchers can gain a deeper understanding of the TME, uncover new biomarkers, and ultimately enhance the efficacy of immunotherapy.


We are developing (underway) and testing models on multi-modal imaging data, including Large Language Models (LLMs), to understand cellular and molecular-level communication of TILs in the TME. This data is correlated with WES genomics and proteomics data, providing a comprehensive understanding of TMB and neoantigen load. This integration enhances the development of TIL therapies and improves response rates by identifying key factors that influence the immune response and tumor progression. 

Understanding the Role of TAM Macrophages for Targeted Immunotherapy Using LLMs and Multimodal Data 

The Tumor Microenvironment (TME) in colorectal cancer (CRC) plays a crucial role in tumorigenesis, progression, and therapeutic response. The TME consists of various cellular and non-cellular components, including immune cells, stromal cells, blood vessels, and extracellular matrix. These components interact with tumor cells, influencing their behavior and response to treatment. Understanding TME is essential for developing effective therapies and improving patient outcomes in CRC. Tumor-associated macrophages (TAMs) are a significant component of the TME and play a vital role in tumor progression and metastasis. TAMs can exhibit both pro-tumor and anti-tumor functions, depending on their polarization state. Targeting TAMs has emerged as a promising therapeutic strategy to modulate the TME and enhance the efficacy of cancer treatments.


Imaging biomarkers serve as surrogate endpoints in clinical trials and can be used to predict the response or resistance in immune checkpoint inhibitor (ICI) based therapy. GenAI models predict patient responses in ICI or TCR-based therapies by analyzing genetic and molecular profiles, enabling personalized treatment plans that ensure patients receive the most appropriate and effective therapies. Generative AI (GenAI) designs and optimizes TCRs with high specificity and affinity for tumor antigens, reducing off-target effects and enhancing therapeutic efficacy of different TCR-antigen interactions.


Multimodal (MM) spatial omics data integration plays a pivotal role in immunotherapy by providing a comprehensive understanding of cellular functions and interactions within the immune system. Mapping the spatial distribution of immune cells using LLMs is crucial for identifying therapeutic targets and developing precise, effective treatments in ICI based approaches. LLMs play a significant role in MM data integration, feature extraction and processing. They can identify and extract key features from single-cell data, such as gene expression profiles, cell(sub-cell) types, and cell states, as well as analyze cell-cell interactions. In spatial transcriptomics and CODEX proteomics imaging data, LLMs help map the spatial context of TAM morphological features, interactive zones, and protein expression pathways (workflow illustrated in figure). By integrating these diverse datasets, LLMs provide detailed insights into the tumor microenvironment. Fine-tuning of LLMs based on TCR and ICI design in immunotherapy further enhances their predictive capabilities and therapeutic efficacy.

This is a proof-of-concept model under development on mapping macrophages, dendritic cells, and other key players in TME of CRC for targeted immunotherapy using spatial omics MM data with GenAI and LLMs. 

A Multimodal Spatial Omics Data Integration Strategy Using LLMs for Characterizing TAM Macrophages in the Tumor TME for Predicting Immunotherapy Response 

Tumor-associated macrophages (TAMs) are pivotal components of the tumor microenvironment (TME), orchestrating cancer progression through diverse mechanisms, including angiogenesis, metastasis, and the development of therapeutic resistance. These versatile immune cells, encompassing subtypes like M1 and M2 macrophages with distinct functional profiles, have become a focal point for targeted immunotherapy. Strategies aimed at modulating TAM behavior, such as suppressing recruitment and promoting repolarization, are actively being explored in preclinical and clinical settings. Furthermore, the integration of artificial intelligence (AI) with large datasets holds promise for predicting patient responses to immunotherapies, including immune checkpoint inhibitors (ICIs), for which PD-L1 expression and tumor mutational burden (TMB) are established biomarkers.


A comprehensive understanding of TAMs in ICI therapy and response/resistance necessitates a multimodal data integration strategy. This approach leverages advanced technologies such as spatial omics, which elucidates the spatial organization and interactions of TAMs within the TME. Simultaneously, single-cell RNA sequencing (scRNA-seq) dissects TAM heterogeneity, revealing distinct subtypes and their unique transcriptomic signatures. CITE-seq further enhances this characterization by simultaneously measuring gene and protein expression within individual cells, providing a detailed molecular fingerprint of TAMs. Integrating these diverse data sources enables researchers to connect spatial context with high-resolution molecular information, offering unprecedented insights into TAM function.


The accompanying figure visually encapsulates this multimodal integration strategy. Spatial omics data illustrates the distribution of TAMs and other immune cell populations within the tumor. A UMAP plot, derived from scRNA-seq data, showcases the diversity of macrophage subtypes and their associated gene expression patterns. CITE-seq, depicted with a schematic of RNA and ADT capture, highlights the simultaneous measurement of gene and protein expression. The central graphic emphasizes the convergence of these data streams, along with information derived from ligand-receptor interaction analysis and potentially multi-modal data feature extractions by large language models (LLMs), to provide a holistic understanding of TAM behavior. This integrated analysis aims to improve prediction of patient response and resistance to ICI and cell therapies, ultimately facilitating personalized cancer treatment strategies. 

TIL-Related Biomarkers in Colorectal Cancer (CRC)   

CD3

Marker for total T-cell infiltration, linked to improved prognosis in CRC.

✅ Yes, high levels indicate better survival.(Predictive Biomarker)

❌ No direct targeting.(Therapeutic Target)

T-cell activation pathways (Pathway Analysis)


CD4

Helper T-cell marker, supports immune responses and promotes antitumor activity.

✅ Yes, contributes to immune response.

❌ No direct targeting.

TCR signaling, cytokine production.

CD8

Cytotoxic T-cell marker; strong infiltration correlates with better prognosis and therapy response.

✅ Yes, associated with better immunotherapy response.

❌ No direct targeting, but enhanced in therapies.

Cytotoxic T-cell activation, antigen presentation.

FOXP3

Marker for regulatory T cells (Tregs), which suppress immune responses and may promote tumor progression.

✅ Yes, high levels may indicate immune suppression.

✅ Potential target for reducing immune evasion.

Treg-mediated immune suppression pathways.

PD-1

Immune checkpoint receptor that inhibits T-cell activity, leading to immune evasion.

✅ Yes, predicts response to checkpoint inhibitors (especially in MSI-high CRC).

✅ Targeted by anti-PD-1 therapies (e.g., pembrolizumab, nivolumab).

Immune checkpoint pathways, exhaustion mechanisms.

PD-L1

Ligand for PD-1; its expression on tumor or immune cells can predict response to checkpoint inhibitors.

✅ Yes, high expression predicts response to PD-1/PD-L1 inhibitors.

✅ Targeted by anti-PD-L1 therapies (e.g., atezolizumab, durvalumab).

PD-1/PD-L1 immune evasion pathway.

LAG-3

Checkpoint receptor involved in T-cell exhaustion, contributing to immune evasion.

✅ Emerging biomarker for immune exhaustion.

✅ Targeted by LAG-3 inhibitors (e.g., relatlimab).

T-cell exhaustion pathways.

TIM-3

Immune checkpoint protein linked to dysfunctional or exhausted T cells in the tumor microenvironment.

✅ Emerging biomarker for T-cell exhaustion.

✅ Targeted by TIM-3 inhibitors (under clinical trials).

Immune tolerance and exhaustion pathways.

Granzyme B

Cytotoxic enzyme produced by CD8+ T cells and NK cells, indicating active antitumor immune response.

✅ Yes, high levels indicate strong immune attack on cancer cells.

❌ No direct targeting, but enhanced in immunotherapies.

Cytotoxic T-cell and NK cell pathways.

IFN-γ

Cytokine produced by T cells and NK cells that enhances antigen presentation and immune activation.

✅ Yes, associated with response to immunotherapies.

❌ No direct targeting, but a key immune modulator.

JAK/STAT immune signaling pathway.

CXCL9/CXCL10

Chemokines that recruit T cells to the tumor microenvironment, enhancing immune response.

✅ Yes, associated with increased TIL infiltration and improved prognosis.

❌ No direct targeting, but used in immune modulation strategies.

Chemokine signaling and T-cell recruitment pathways.


TILs in CRC: