Shift Bioscience (Shift), a biotechnology company uncovering the biology of cell rejuvenation to develop new therapies for age driven diseases, today announced the results of a new study detailing an improved approach to ranking virtual cell models for gene discovery. The research describes the introduction of new metrics and baselines to assess model performance, providing an improved framework for development of more powerful virtual cell models. The new ranking system will also enable Shift to accelerate its rejuvenation target discovery pipeline.
Virtual cell models trained using single-cell RNA (scRNA) datasets offer a powerful solution for large-scale screening of phenotypic changes brought on by various perturbations, including up- and down-regulation of genes. They offer a unique opportunity to expand target discovery programs by enabling researchers to compress centuries of real-world experiments into months of virtual experiments, and therefore identify the most promising gene targets ahead of progressing to resource-intensive wet lab research. Despite this, benchmarking studies using common performance metrics have reported that the most prominent virtual cell models are outperformed by the simple dataset mean – a prediction of the average result of all cells within a dataset.
The new study, which was led by Shift’s Head of Machine Learning, Lucas Paulo de Lima Camillo, used both virtual cell and real-world data to reveal that experimental factors such as control bias and weak perturbations misrepresent true model performance when using commonly used metrics. Based on this analysis, the team developed a series of steps that can be employed to better rank models and drive focus toward more biologically meaningful changes. These pre-processing steps include differentially expressed gene (DEG)-weighted score metrics, negative and positive baseline calibrations, and DEG-aware optimization objectives. Incorporation of this new approach into perturbation modelling better highlights models with poor performance, ensuring that only effective models are leveraged for target identification programs.
Lucas Paulo de Lima Camillo, Head of Machine Learning, Shift Bioscience, commented: “In this research, our team has shown that by focusing on the development of new metrics and baselines, we can more easily identify models that demonstrate strong predictability. The paper provides foundational data which will enable us to develop more powerful, biologically-useful perturbation models, ultimately accelerating our therapeutic pipeline and helping us to uncover new targets for rejuvenation therapeutics.”
