Benchmarking Standards and Performance Evaluation of Tumor NGS Liquid Biopsy Technologies: Insights from the SEQC2 Consortium

In recent years, with the continuous advancement of technology, liquid biopsy techniques based on circulating tumor DNA (ctDNA) have provided precise and convenient methods for the detection of driver genes and the tracking of minimal residual disease (MRD). These techniques have become an emerging tool for prognostic stratification and recurrence risk assessment in early-stage cancer patients, an effective alternative for cancer patients unable to obtain tissue samples, and a valuable aid for the comprehensive management of patients throughout their treatment course, demonstrating significant clinical value and promising application prospects.

SEQC2 Study Validates the Core Clinical Practice Metrics of Liquid Biopsy Products Through Single-Blind, Multi-Site, and Cross-Platform Performance Analysis, Aiming to Establish Industry Standards and Provide Technical Guidelines. The study included five globally leading NGS-based tumor-targeted testing kit manufacturers: Roche Sequencing Solutions (referred to as ROC), Illumina, IDT (Integrated DNA Technologies), Thermo Fisher Scientific (referred to as TFS), and Burning Rock Biotech Ltd (BRP). By using simulated sequencing data and artificially synthesized human ctDNA reference materials, the study rigorously assessed the result validity of five leading ctDNA analysis methods based on the impact of variables at each step of the ctDNA sequencing workflow. The results showed that all detection methods reliably detected ctDNA with a VAF ≥ 0.5% (with high sensitivity and repeatability), but accuracy decreased when ctDNA VAF < 0.5%, with differences observed among the various detection methods.

1. SEQC2 Research Content and Results: ctDNA detection is identified from the background of cell-free DNA (cfDNA) containing non-tumor cells. To eliminate confounding experimental variables, simulated NGS libraries were first generated. Targeted analysis of cell-free DNA was simulated through hybrid capture sequencing, and then synthetic DNA sequences that mimic natural human genes and mutations were used to evaluate ctDNA analysis performance. Variations in ctDNA abundance in plasma may be associated with tumor progression, sensitivity or resistance to treatment, etc. Therefore, while qualitatively identifying mutations, it is also necessary to accurately detect ctDNA mutation frequency. The study showed that at the maximum fragment depth, a variant allele frequency (VAF) of 0.8% can be reliably resolved, but VAF analysis below this level is inaccurate and is affected by sequencing depth and coverage. This demonstrates the difficulty of accurately identifying low-frequency ctDNA mutations in patients early on or during treatment.

Figure 1Performance Evaluation Process of Tumor ctDNA High-Throughput Sequencing

2. Sensitivity Assessment - More Than Just Sufficient Sequencing Depth: ctDNA contains a large number of low-abundance genetic variations, and the accurate detection of low-abundance mutations has always been the biggest bottleneck in the clinical application of liquid biopsy. It is a widely accepted consensus among clinical experts that increasing sequencing depth helps improve detection sensitivity. However, the results of the SEQC2 project show that when library construction technology capabilities are limited, simply increasing sequencing depth can no longer effectively improve the detection rate. Only when the library preparation technology raises the ceiling of sensitivity can increased sequencing depth lead to more accurate results. In addition, sequencing coverage and uniformity of coverage are also very important. The results of the SEQC2 project reveal the capability differences between different library construction technologies.

(1) Coverage Depth and Effective Fragment Recovery Rate: The effective fragment recovery rate represents the cell-free DNA (cfDNA) fragments effectively retained from the same blood volume after layers of signal collection and processing, which directly determines the possibility of ctDNA detection. In the SEQC2 project, the median coverage depth of single fragments sequenced ranged between 1,235X-4,698X, with ROC (4,698X) > BRP (4,528X) > TFS (3,280X) > IDT (2,476X) > LIM (1,235X), and the effective fragment recovery rates were 63% > 60% > 44% > 33% > 17%, indicating that ROC (Roche) and BRP (Burning Rock Biotech Ltd) performed best in ctDNA library construction technology.

(2) Panel coverage uniformity: In the case of the same DNA input amount of 25 ng (equivalent to 7,500 genome copies), SEQC2 compared the differences in target design region coverage at varying sequencing depths for different panels. Better coverage uniformity ensures that even difficult regions are well-covered, thereby improving detection sensitivity. The best coverage uniformity was observed with BRP (Burning Rock Biotech Ltd) and IDT. Additionally, under similar effective fragment recovery rates between Burning Rock’s Langqing panel and the ROC panel, Burning Rock’s Langqing demonstrated better coverage uniformity. Based on these parameters, even though Burning Rock did not have the highest raw sequencing depth, its sensitivity remained exceptionally strong due to sufficient assurance of effective fragment recovery rate and coverage uniformity.

High-sensitivity detection not only aids in more precise late-stage medication guidance but also expands the clinical application scenarios of liquid biopsy. The rapid advancement of MRD (minimal residual disease) detection technology during the perioperative period has brought benefits to patients, which is closely related to the improvement in detection sensitivity. With the development and refinement of technology, the increase in sensitivity is expected to further expand the number of patients benefiting from liquid biopsy, allowing more patients to benefit.

3. Ensuring the Accuracy of Liquid Biopsy: High Sensitivity and Low False Positive Rate Are Both Essential: In addition to sensitivity, another key performance metric for liquid biopsy panels that clinicians focus on is the false positive rate, which indicates detection specificity. High sensitivity ensures accurate identification of low-frequency mutations without missed detections, while high specificity guarantees precise discernment of tumor-derived mutations, capturing truly beneficial treatment opportunities for patients and avoiding delays in optimal treatment timing and financial waste. Technically, it is often difficult to achieve both high sensitivity and high specificity, making the balance between them another challenge for ensuring the accuracy of liquid biopsy technology.

SEQC2 evaluates the sensitivity of ctDNA detection based on hybrid capture methods by analyzing the detection results of known variants in the target regions of each library. In high-frequency mutations, all panels demonstrated extremely high sensitivity, reaffirming the significant impact of ctDNA content in samples on test outcomes. However, when the VAF drops below 0.5% into the "challenging zone," the performance of different panels begins to show notable differences. Among them, Burning Rock's Langqing and IDT panel emerged as the most sensitive NGS liquid biopsy panels for low VAF mutations in this study: at a VAF of 0.3-0.5%, sensitivity exceeded 90%.

In addition, it was also verified that a单纯 high coverage depth does not necessarily mean high sensitivity. Although the single fragment coverage depth of IDT is relatively low, its sensitivity is higher than that of ROC, which may be related to the high coverage uniformity of the IDT panel.

To ensure high sensitivity and a low false positive rate, all panel analyses utilized Unique Molecular Identifiers (UMIs) to correct sequencing errors, resulting in a relatively low false positive rate. At a DNA input of 25 ng, the lowest false positive rates were observed for BRP (Burning Rock Biotech Ltd) and ILM (Illumina), at 0.030 FP/kb and 0.039 FP/kb, respectively. Based on an overall evaluation of sensitivity and specificity, the Burning Rock Medicine's Lang Qing panel demonstrated the highest accuracy among the four NGS liquid biopsy panels.

4. Reproducibility of ctDNA Test Results: SEQC2 evaluated the reproducibility of test results for the same samples within and between laboratories. The results showed that the reproducibility of the same panel across different batches within the same laboratory and across different laboratories was similar. This indicates that the testing performance and bioinformatics analysis of the NGS panel kits involved in the SEQC2 study have strong stability, and the fluctuations in results stem from random rather than systematic inter-laboratory differences.

It is similar to sensitivity.When ctDNA VAF≥0.5%, reproducibility is generally high, but between 0.1%~0.5%, reproducibility is relatively low, with sensitivity being the determining factor for measuring reproducibility; despite significant variations among samples and groups, researchers observed no substantial differences in reproducibility during intra- and inter-laboratory comparisons, indicating that all participating ctDNA panel testing products and bioinformatics analyses are stable, and result fluctuations stem from random rather than systematic inter-laboratory differences. This suggests that the evaluated liquid biopsy products exhibit strong stability, ensuring consistency of test results across different patient channels, helping patients obtain reliable and effective guidance.

5. Detection of "difficult" samples with low cfDNA total amount poses higher requirements for liquid biopsy technology: In clinical practice, it is common to encounter patient blood samples with extremely low cfDNA content, especially in early-stage cancer patients. Therefore, the performance of liquid biopsy at low cfDNA input levels deserves special attention. The SEQC2 project evaluated the analytical performance of various panels at high (50 ng), medium (25 ng), and low (10 ng) DNA input levels. Sequencing depth correlates linearly with input amount but varies significantly across different analyses. Compared to 10 ng, a 25 ng input provides increased fragment depth, leading to significant improvements in sensitivity, reproducibility, and overall diagnostic performance across all analyses, particularly for low-frequency variants. At the lowest DNA input level of 10 ng, BRP (Burning Rock Biotech Ltd.) demonstrated the best stability.

The development of liquid biopsy technology will gradually extend the benefits of precision medicine from late-stage patients to early-stage ones, significantly expanding clinical application scenarios while assisting doctors in managing the entire treatment process for patients. However, reliably detecting trace amounts of fragmented ctDNA from blood remains a major technical challenge. The SEQC2 project evaluates key metrics such as sensitivity, false positive rate, and reproducibility, providing clinicians with an intuitive, objective, comprehensive, and quantitative understanding of liquid biopsy technology. It establishes judgment criteria for NGS-based liquid biopsy products and offers best practice guidelines for the comprehensive evaluation of ctDNA analysis performance.