Jinfeng Bio's Bioinformatics Team Unveils Deep Learning Model TABR-BERT to Advance Cellular Immunotherapy

DISCOVERY TO CURE Bioinformatics Team

Recently, the team published an article titled "Accurate TCR-pMHC Interaction Prediction Using a BERT-based Transfer Learning Method" in Briefings in Bioinformatics (Impact Factor=9.5).TABR-BERT). TABR-BERT adopts two pre-trained models based on the BERT (Bidirectional Encoder Representations from Transformers) framework to generate embedding matrices for TCR (T cell receptor), pMHC (peptide-MHC), and predict TCR-pMHC binding.

Numerous TCR-based therapies have shown great potential in clinical applications.TCR-TTCR sequences that can specifically bind to target antigens will be screened and identified, and then transferred into patient-derived peripheral blood T cells (or allogeneic T cells) using genetic engineering methods. The modified T cells will be reinfused into the patient's body to specifically recognize and kill tumor cells expressing the antigen.ImmTACs(Immune Mobilizing Monoclonal TCRs Against Cancer) is a novel bispecific macromolecule, composed of an engineered TCR and an anti-CD3 scFv (single chain antibody fragment). The engineered TCR can specifically recognize and bind to pMHC on the surface of tumor cells with significantly enhanced affinity (9 times higher than antigen-antibody affinity), while the anti-CD3 scFv attracts and recruits T cells to the vicinity of tumor cells, activating them to exert tumor-killing effects.TCRm Ab(TCR mimic antibody, TCR-mimic antibody) activates T cells and initiates an immune response by binding to the TCR on the surface of T cells, mimicking the natural TCR-antigen interaction.

However, the prediction of TCR-pMHC binding has always been a challenge in immunotherapy. The contradiction between the diversity of TCRs and neoantigens and the scarcity of experimentally determined TCR-pMHC binding data indicates that predicting TCR-pMHC through general machine learning or deep learning methods is quite difficult. Moreover, most neoantigens encountered in real-world scenarios are "novel" ones without corresponding experimental data. This means that predicting the binding specificity of unseen epitopes (neoantigens not present in the training set) with their corresponding TCRs is a more practically significant problem.

TABR-BERT is based on the BERT architecture and leverages large-scale TCR sequence data as well as peptide-MHC pair data using a pretraining-finetuning approach. During the pretraining phase, TCR-BERT and pMHC-BERT learn the distribution characteristics of amino acid sequences through the MLM (Masked Language Modeling) task and the NSP (Next Sentence Prediction) task. The attention mechanism allows the two embedding models to focus more on important residues in the amino acid sequences, thereby enhancing the model's generalization ability and improving the prediction performance for TCR-pMHC binding.

BERT is a natural language processing model based on deep learning, which learns rich information within the data through unsupervised training on large-scale datasets, and then generates an embedding matrix for specific downstream tasks. The TABR-BERT model consists of three sub-models, among whichTCR-BERT（TCR embedding model）、pMHC-BERT（pMHC embedding model) Pre-trained with over 110 million TCR cdr3β sequences and over 4 million pMHC sequences to learn the "semantic" information within amino acid sequences, generating embedding matrices for predicting TCR-pMHC binding.TCR-pMHC prediction modelUsing the two embedding matrices mentioned above, a TCR-pMHC binding prediction is performed through an MLP (Multi-Layer Perceptron).

TP53It is an important tumor suppressor gene, with mutation rates as high as 38%-50% in ovarian, esophageal, colorectal, head and neck, and lung cancers. According to records from a dataset covering 25% of the U.S. population (SEER epidemiological data), between 2000 and 2017, the number of malignant tumor cases in the United States exceeded 7 million, among which...TP53Mutation accounts for up to 35%, involving more than 2 million cases.

Using TABR-BERT across eight cohorts covering 3.717% TP53 mutation patientsTP53Tested on hotspot mutations, TABR-BERT's AUC-ROC reached0.934，Better than TEIM (Tsinghua University-Institute for Interdisciplinary Information Sciences), PanPep (Tongji University -School of Life Sciences and Technology）、DLpTCR（Harbin Institute of Technology - School of Life Science and Technology) and other models.

Different models inTP53ROC Curve on Hotspot Mutation Test Set

TABR-BERT's excellent performance on different test sets demonstrates that the BERT-based pre-training approach can generate meaningful embedding matrices. These embedding matrices can not only be used for predicting TCR-pMHC binding specificity but also applied to a series of downstream tasks such as peptide-MHC binding specificity prediction. This provides new analytical tools for adaptive immunity research and broadens the pathways to understanding adaptive immunity.

In summary, TABR-BERT provides a new method for accurately predicting TCR-pMHC binding specificity, offering more possibilities for T cell screening in cellular immunotherapy.