BioMap Unveils xTrimoPGLM: The World's First Open-Source 100-Billion-Parameter Protein Language Model

https://github.com/biomap-research/xTrimoPGLM

The field of protein is the most fruitful and prominent pearl in the application of AI in life sciences.

As an important molecule in the life system, proteins participate in almost all life processes, including metabolism, immunity, conduction, cell differentiation, and signal transduction. The complexity of their structure, function, interactions, and regulatory mechanisms has always been a key focus for scientists to explore.

From the debut of AlphaFold2 in 2020 to winning the 2024 Nobel Prize in Chemistry, AI has helped humanity decipher the "protein code" and moved from the laboratory to industrial applications, coveringNew Drug Development, Disease Diagnosis, Synthetic BiologyIn multiple fields, showing huge market potential.

Because of this,Protein models have gained far more attention and popularity in the AI + life sciences field than others., which is often the first battle for enterprises or research teams to prove their strength and gain industry recognition. Meta and DeepMind, both highly influential in the industry, have launched protein-related models.

As protein computing pioneer David Baker said:"Proteins are the machinery of life, and understanding their language will unlock the secrets of biology."

As one of the earliest companies globally to dedicate itself to the research and development of large models in the life sciences, BioMap's first academic open-source project, HelixFold-Single, focuses on the field of protein structure prediction and has been featured on the cover of a Nature sub-journal.

HelixFold-Single Model Framework Diagram

This model is globallyThe First High-Speed Protein Structure Prediction Model Not Dependent on MSA, achieving a breakthrough in "Folding with Large-scale Protein Language Model," with a speed increase of over a hundred times in evaluation tasks, bringing a new leap to the field of protein structure prediction.

In 2023, the company's protein language modelxTrimoPGLMA more successful pre-training method that integrates two different types of tasks: protein understanding and protein generation.Achieved SOTA in 15 out of 18 tasks, with overall performance surpassing the original SOTA task model as well as the pre-trained Meta ESM-2.

In the same year, BioMap's "ChatGPT Capable of Generating Proteins"has also been born. This AIGP (AI Generated Protein) platform, driven by xTrimo, is able to generate proteins in a targeted manner or design proteins through generative methods.

After years of沉淀, BioMap's large protein model has completed several rounds of self-evolution and improvement, with various data feeding back into the AI platform's training through an ecological循环, further enhancing the model's capabilities.

The First Large Protein Model with a Trillion Parameters:

Big is Powerful, Bridging Understanding and Generation

Firmly choosing the large model direction, BioMap has expanded the boundaries of biological computing with its profound technical practice.

In the field of natural language,Scaling LawHas become the golden rule, the most well-known principle in the artificial intelligence industry, and is regarded by Microsoft CEO Nadella as the true driving force behind the AI revolution.

The reason lies in the Scaling Law, which reveals the key to global AI competition —The Intrinsic Relationship Between Model Performance and Data Scale: Model performance should grow linearly as model parameters, data volume, and computation increase proportionally in an exponential manner.

Scaling Law has become the cornerstone of a host of large models like GPT, prompting all parties in the industry to build moats around data and computing power, and serving as the underlying belief for many elites who are convinced that AI can change the world.

Furthermore, the research results of xTrimoPGLM validate the Scaling Law:With the exponential growth of computational power in protein language models, the performance on downstream tasks will also increase linearly.。

This breakthrough achievement demonstrates the necessity of large models in handling complex biological tasks, providing theoretical support for the development of biological large models.

Under the rule of "bigger is stronger," BioMap, with the largest parameter scale in the industry, has secured a leading position.

Of course,BioMap's technical advantages are not only reflected in the breakthrough of model scale, but also in its deep understanding and precise grasp of the complex systems in life sciences.

Traditional protein language models are often limited by a single pre-training objective, either excelling in understanding tasks (such as the ESM series models, mainly used for protein structure prediction) or focusing on generation tasks (such as ProGen, which emphasizes protein generation), revealing shortcomings in task adaptability and generalization capabilities.

Based on a deep understanding of protein data, BioMap researchers innovated the xTrimoPGLM pre-training framework, successfully optimizing both understanding and generation tasks by combining the advantages of GLM (General Language Model) and MLM (Masked Language Model).

This unified framework enables xTrimoPGLM toUnderstand the task of providing precise amino acid and sequence-level representations, while in generation tasks, being able to produce novel protein sequences that are structurally similar to natural proteins.

By constructing an unprecedentedly large protein language training dataset and combining it with an innovative algorithm architecture to fully explore the potential value of massive parameters, BioMap's xTrimoPGLM demonstrates outstanding performance.

In protein understanding tasks, xTrimoPGLM excels in various evaluations, covering areas such as protein structure, function, interaction, and developability.15 out of 18 tasks surpass the previous SOTA model。

In addition, xTrimoPGLM has also demonstrated outstandingDe novo design of protein sequencesThe performance, which can generate structurally similar but sequence-divergent proteins, offers more possibilities for drug design and protein engineering.

By customizing specific structures and biophysical properties through supervised fine-tuning, the "super alignment" capability of xTrimoPGLM will further unleash its potential as a programmable model in exploring and synthesizing the vast protein space.

After continuous technological iteration and optimization, xTrimoPGLM has reached an internationally leading level in terms of model scale and performance metrics, establishing its benchmark status in the field of biological computing.

There is no doubt that the open-source release of xTrimoPGLM will provide strong momentum for both academia and industry. This choice echoes the practice of DeepSeek, promoting the widespread application of AI in life sciences and accelerating global research progress.

Recommended Reading