Updated benchmarking of variant effect predictors using deep mutational scanning

to related objects

Abstract: The assessment of variant effect predictor (VEP) performance is fraught with biases introduced by benchmarking against clinical observations. In this study, building on our previous work, we use independently generated measurements of protein function from deep mutational scanning (DMS) experiments for 26 human proteins to benchmark 55 different VEPs, while introducing minimal data circularity. Many top‐performing VEPs are unsupervised methods including EVE, DeepSequence and ESM‐1v, a protein language model that ranked first overall. However, the strong performance of recent supervised VEPs, in particular VARITY, shows that developers are taking data circularity and bias issues seriously. We also assess the performance of DMS and unsupervised VEPs for discriminating between known pathogenic and putatively benign missense variants. Our findings are mixed, demonstrating that some DMS datasets perform exceptionally at variant classification, while others are poor. Notably, we observe a striking correlation between VEP agreement with DMS data and performance in identifying clinically relevant variants, strongly supporting the validity of our rankings and the utility of DMS for independent benchmarking.

Location: Deutsche Nationalbibliothek Frankfurt am Main

Extent: Online-Ressource

Language: Englisch

Bibliographic citation: Updated benchmarking of variant effect predictors using deep mutational scanning ; day:13 ; month:06 ; year:2023 ; extent:15
Molecular systems biology ; (13.06.2023) (gesamt 15)

Creator: Livesey, Benjamin J.
Marsh, Joseph A.

DOI: 10.15252/msb.202211474

URN: urn:nbn:de:101:1-2023061415350252516896

Rights: Open Access; Der Zugriff auf das Objekt ist unbeschränkt möglich.

Last update: 14.08.2025, 11:01 AM CEST

Data provider

This object is provided by:
Deutsche Nationalbibliothek. If you have any questions about the object, please contact the data provider.

Show original at data provider

Associated

Livesey, Benjamin J.
Marsh, Joseph A.

Other Objects (12)

Variant effect predictions capture some aspects of deep mutational scanning experiments

Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations

Mutational scanning of CLOCK exon 19 transactivation domain

A statistical framework for analyzing deep mutational scanning data

Benchmarking variant identification tools for plant diversity discovery

Optimization of the antimicrobial peptide Bac7 by deep mutational scanning

Correction to: A statistical framework for analyzing deep mutational scanning data

Comprehensive benchmarking and guidelines of mosaic variant calling strategies

Jointly benchmarking small and structural variant calls with vcfdist

Benchmarking UMI-aware and standard variant callers for low frequency ctDNA variant detection

Comprehensive mutational scanning of EGFR reveals TKI sensitivities of extracellular domain mutants

Mutational scanning pinpoints distinct binding sites of key ATGL regulators in lipolysis

Variant effect predictions capture some aspects of deep mutational scanning experiments

Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations

Mutational scanning of CLOCK exon 19 transactivation domain

A statistical framework for analyzing deep mutational scanning data

Benchmarking variant identification tools for plant diversity discovery

Optimization of the antimicrobial peptide Bac7 by deep mutational scanning

Correction to: A statistical framework for analyzing deep mutational scanning data

Comprehensive benchmarking and guidelines of mosaic variant calling strategies

Jointly benchmarking small and structural variant calls with vcfdist

Benchmarking UMI-aware and standard variant callers for low frequency ctDNA variant detection

Comprehensive mutational scanning of EGFR reveals TKI sensitivities of extracellular domain mutants

Mutational scanning pinpoints distinct binding sites of key ATGL regulators in lipolysis

Variant effect predictions capture some aspects of deep mutational scanning experiments

Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations

Mutational scanning of CLOCK exon 19 transactivation domain

A statistical framework for analyzing deep mutational scanning data

Benchmarking variant identification tools for plant diversity discovery

Optimization of the antimicrobial peptide Bac7 by deep mutational scanning

Correction to: A statistical framework for analyzing deep mutational scanning data

Comprehensive benchmarking and guidelines of mosaic variant calling strategies

Jointly benchmarking small and structural variant calls with vcfdist

Benchmarking UMI-aware and standard variant callers for low frequency ctDNA variant detection

Comprehensive mutational scanning of EGFR reveals TKI sensitivities of extracellular domain mutants

Mutational scanning pinpoints distinct binding sites of key ATGL regulators in lipolysis

Cultural heritage institutions wishing to register will find more information here.

Fields marked * need to be filled in.

Username*

Please enter your username

Email*

Please enter your email address

Please do not fill this field

First name

Last name

Password*

Please enter your password

Confirm password*

Please enter the same password

I have read the terms of use and the privacy policy for the collection of personal data and accept them. *

This field is required.

I would like to subscribe to the newsletter of the Deutsche Digitale Bibliothek. See newsletter subscription info.

Account created

Your "My DDB" account has been successfully created. Before you can log in to your account, you must click the confirmation link in the message we just sent to the email address you provided.

Updated benchmarking of variant effect predictors using deep mutational scanning

Object Details

References and Relationships

Contributors, Places and Time

Further information

Data provider

Associated

Other Objects (12)

Variant effect predictions capture some aspects of deep mutational scanning experiments

Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations

Mutational scanning of CLOCK exon 19 transactivation domain

A statistical framework for analyzing deep mutational scanning data

Benchmarking variant identification tools for plant diversity discovery

Optimization of the antimicrobial peptide Bac7 by deep mutational scanning

Correction to: A statistical framework for analyzing deep mutational scanning data

Comprehensive benchmarking and guidelines of mosaic variant calling strategies

Jointly benchmarking small and structural variant calls with vcfdist

Benchmarking UMI-aware and standard variant callers for low frequency ctDNA variant detection

Comprehensive mutational scanning of EGFR reveals TKI sensitivities of extracellular domain mutants

Mutational scanning pinpoints distinct binding sites of key ATGL regulators in lipolysis

Variant effect predictions capture some aspects of deep mutational scanning experiments

Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations

Mutational scanning of CLOCK exon 19 transactivation domain

A statistical framework for analyzing deep mutational scanning data

Benchmarking variant identification tools for plant diversity discovery

Optimization of the antimicrobial peptide Bac7 by deep mutational scanning

Correction to: A statistical framework for analyzing deep mutational scanning data

Comprehensive benchmarking and guidelines of mosaic variant calling strategies

Jointly benchmarking small and structural variant calls with vcfdist

Benchmarking UMI-aware and standard variant callers for low frequency ctDNA variant detection

Comprehensive mutational scanning of EGFR reveals TKI sensitivities of extracellular domain mutants

Mutational scanning pinpoints distinct binding sites of key ATGL regulators in lipolysis

Variant effect predictions capture some aspects of deep mutational scanning experiments

Using deep mutational scanning to benchmark variant effect predictors and identify disease mutations

Mutational scanning of CLOCK exon 19 transactivation domain

A statistical framework for analyzing deep mutational scanning data

Benchmarking variant identification tools for plant diversity discovery

Optimization of the antimicrobial peptide Bac7 by deep mutational scanning

Correction to: A statistical framework for analyzing deep mutational scanning data

Comprehensive benchmarking and guidelines of mosaic variant calling strategies

Jointly benchmarking small and structural variant calls with vcfdist

Benchmarking UMI-aware and standard variant callers for low frequency ctDNA variant detection

Comprehensive mutational scanning of EGFR reveals TKI sensitivities of extracellular domain mutants

Mutational scanning pinpoints distinct binding sites of key ATGL regulators in lipolysis

Related objects

Reset password