Tool Audit

[easlinger]

Tools

From Heumos, L., Schaar, A. C., Lance, C., Litinetskaya, A., Drost, F., Zappia, L., ... & Theis, F. J. (2023). Best practices for single-cell analysis across modalities. Nature Reviews Genetics:

One area of perturbation modelling tries to differentiate successfully from unsuccessfully targeted cells for experimental set-ups in which this assignment is unknown and to assess the perturbation effect. Mixscape116 and MUSIC120 first remove confounding sources of variation, then dissect successfully from unsuccessfully perturbed cells, to finally visualize and score perturbation effects. Augur121,122 and MELD123 cover only the third step and rank cell types according to the degree of perturbation response to identify cell populations that were most affected by a perturbation.

A second area of perturbation modelling concerns perturbations that are not experimentally measured. Latent space learning models such as scGen42, CPA124 and CellBox125 aim to predict responses for unseen perturbations, combinations or drug doses. Such models generally work well for highly expressed genes but may struggle with lowly expressed genes owing to a lack of variance.

Pertpy (Python)

Pertpy seems to be the CRISPR analysis tool with the broadest coverage (among those with adequate documentation).

Documentation

Source Code
Native Datasets
General Usage
Tutorials (Stable Repository)

Functionality

• Pooled CRISPR screens

  • Mixscape: Python implementation of Paplexi et al. Mixscape method. Embeds cells into perturbation space (signature) to remove confounding sources of variation (e.g., cell cycle, replicates). Mixture models identify which targeted cells were perturbed or not. Creates plots to visualize perturbation condition differences in gene expression.

• Compositional analysis

  • Milo: "[D]ifferential abundance testing on KNN graphs, to ease interoperability with scverse pipelines for single-cell analysis" (see here for statistical details) Click here to download Milo paper.
  • scCODA and tascCODA: "[I]dentification of compositional changes in high-throughput sequencing count data and tascCODA for sparse, tree-aggregated modeling of high-throughput sequencing data." (Statistics and benchmarking: scCODA is a Bayesian model for compositional single-cell data analysis and tascCODA: Bayesian Tree-Aggregated Analysis of Compositional Amplicon and Single-Cell Data. scCoda GitHub and tutorials.
    • Documentation:

      When analyzing biological processes via single-cell RNA-sequencing experiments, it is often of interest to assess how cell populations change under one or more conditions. This task, however, is non-trivial, as there are several limitations that have to be addressed:

      • scRNA-seq population data is compositional. This must be considered to avoid an inflation of false-positive results.
      • Most datasets consist only of very few samples, making frequentist tests inaccurate.
      • A condition usually only effects a fraction of cell types. Therefore, sparse effects are preferable.

      The scCODA model overcomes all these limitations in a fully Bayesian model, that outperforms other compositional and non-compositional methods.

      scCODA is fully integrable with scanpy, but provides its own data structure for aggregating, plotting and analyzing compositional data from scRNA-seq. Additionally to the scCODA model, the package also features a variety of implementations of other statistical models that can be used as comparisons.

• Multi-cellular or gene programs

  • DIALOGUE

From DIALOGUE repository

• Distances and Permutation Tests: Reimplementation of scperturb (Python and R) functions, which in their preprint, they say provide "uniform pre-processing and quality control pipelines," "harmonize feature annotations," and "[facilitates] direct comparison and integration across datasets" as well as use "E-statistics for perturbation effect quantification and significance testing...for quantifying perturbation similarity and efficacy."

• MetaData

  • "MetaData provides tooling to fetch and add more metadata to perturbations by querying a couple of databases. We are currently implementing several sources with more to come."
  • "CellLineMetaData aims to retrieve various types of information related to cell lines, including cell line annotation, bulk RNA and protein expression data."

• Response Prediction

  • scGen: "[P]erturbation response prediction of scRNA-seq data in Jax. See scGen predicts single-cell perturbation responses for more details."
  • Augur: Python implementation of Augur R package
    • Skinnider, M.A., Squair, J.W., Kathe, C. et al. Cell type prioritization in single-cell data. Nat Biotechnol 39, 30–34 (2021).

      Augur aims to rank or prioritize cell types according to their response to experimental perturbations given high dimensional single-cell sequencing data. The basic idea is that in the space of molecular measurements cells reacting heavily to induced perturbations are more easily separated into perturbed and unperturbed than cell types with little or no response. This separability is quantified by measuring how well experimental labels (eg. treatment and control) can be predicted within each cell type. Augur trains a machine learning model predicting experimental labels for each cell type in multiple cross validation runs and then prioritizes cell type response according to metric scores measuring the accuracy of the model. For categorical data the area under the curve is the default metric and for numerical data the concordance correlation coefficient is used as a proxy for how accurate the model is which in turn approximates perturbation response.

      We reasoned that cell types most responsive to a perturbation should be more separable, within the multidimensional space of single-cell measurements, than less affected ones, and that the relative difficulty of this separation would provide a quantitative basis for cell type prioritization. We formalized this difficulty as a classification task, asking how accurately disease or perturbation state could be predicted from highly multidimensional single-cell measurements. For each cell type, Augur withholds a proportion of sample labels, and trains a classifier on the labeled subset. The classifier predictions are compared with the experimental labels, and cell types are prioritized based on the area under the receiver operating characteristic curve (AUC) of these predictions in cross-validation. (Cell type prioritization in single-cell data. Nat Biotechnol 39, 30–34 (2021).)

• Perturbation Space: "[C]alculating and evaluating perturbation spaces."

---

scGen (Python)

s41592-019-0494-8.pdf

scGen is a generative model to predict single-cell perturbation response across cell types, studies and species (Nature Methods, 2019). scGen is implemented using the scvi-tools framework.

Documentation

Documentation
Tutorials (Stable Repository)
Source Code

Quotes from their repository README

Functionality

• Train on multi-cell-type, multi-condition dataset to predict what perturbation effect would be for a cell type for which you have data in only one condition

• Predict from one dataset with two conditions (e.g. control and perturbed) to another (must have similar gene set)

• Handle batch effects (can handle non-overlap of batch cell types) in annotated data

• Recommends use of normalized data, e.g.,

import scanpy as sc

adata = sc.read(data)
sc.pp.normalize_total(adata)
sc.pp.log1p(adata)

---

Sceptre (R)

sceptre is a method for single-cell CRISPR screen data analysis. sceptre tests for association between CRISPR perturbations and the expressions of genes and also estimates the effect size of perturbations on changes in gene expression. sceptre achieves state-of-the-art calibration and power by leveraging several theoretical advances in assumption-lean and computationally efficient differential expression analysis.

Documentation

• Documentation

• Source Code

• Tutorials

  • Tutorial (High MOI scCRISPR Screen)
  • Tutorial (Low MOI)
  • Tutorial (Low MOI QC)

Functionality

From their repository:

sceptre includes separate modules for low multiplicity-of-infection (MOI) and high MOI single-cell CRISPR screen analysis.

Low MOI: If you are working with low MOI data (1 gRNA per cell), see the low MOI tutorial.

High MOI: If you are working with high MOI data (2 gRNAs per cell), see the high MOI tutorial.

Large-scale analysis: If you are running a large-scale analysis (i.e., the data do not easily fit into memory or you are using a high-performance cluster or cloud), open a Github issue to discuss use of the beta sceptre Nextflow pipeline.

From their low MOI tutorial:

All sceptre analyses follow this general template.

• Setup

Setup: We begin by setting up the analysis. This entails (1) loading the data; (2) constructing the set of CRISPR perturbation-gene pairs that we seek to test for association; and (3) defining a formula object that specifies how sceptre is to adjust for the cell-specific covariates (e.g., sequencing depth, batch, etc.).

• Calibration

We then carry out the “calibration check,” which involves applying sceptre to a set of automatically-generated negative control CRISPR perturbation-gene pairs (i.e., perturbation-gene pairs for which we know there is no regulatory relationship). The calibration check enables us to verify that sceptre controls the number of false discoveries on the dataset under analysis, which ultimately ensures that the discovery set that sceptre returns is high-quality. The calibration check step entails (4) carrying out the calibration check itself and (5) visualizing the calibration check results.

• Discovery

We finally conduct the “discovery analysis,” which consists of applying sceptre to analyze the set of perturbation-gene pairs whose regulatory relationships we seek to learn. The discovery analysis step entails (6) carrying out the discovery analysis itself, (7) visualizing the discovery analysis results, and (8) obtaining the final discovery set.

---

[easlinger]

Tools

Pertpy (Python)

• Pooled CRISPR screens, i.e., Mixscape

  • Yep

• Compositional analysis

  • Milo
    • Doesn't seem to offer much incrementally (Chris & I agree)
  • scCODA and tascCODA
    • scCODA "higher sensitivity for lowly abundant cell types" (important for senescence study)
    • scCODA has UC example/validation
    • scCODA helps distinguish cell type more abundant b/c other went down and can only measure so many cells (proportionality)
    • scCODA very credible methods
    • In the yes pile, but stage II/advanced module, not core/basic (C & E agree)

• Multi-cellular or gene programs, i.e., DIALOGUE

  • "Stretch goal"
  • Interesting, though, could be super important for senescence project

• Distances and Permutation Tests: Reimplementation of scperturb (Python and R) functions (preprint)

• MetaData

  • N/A

• Response Prediction

  • scGen
  • Augur

• Perturbation Space: "[C]alculating and evaluating perturbation spaces."

---

scGen (Python)

See above subsection (reimplementation)

---

Sceptre (R)

---

[easlinger]

Preliminary Verdict

Steps

1. Pre-processing (Scanpy)
2. Remove confounding sources of variation, e.g., normalization, batch effect correction (Mixscape, Harmony?)
3. Perturbation classification (perturbed or not), scoring, and visualization (Mixscape and Augur)
4. Response prediction with Augur, and maybe scGen (for unseen conditions)
5. Composition analysis (scCoda)

Outstanding Questions

• Use native packages or Pertpy re-implementations?
  • Benefits of Pertpy: Stay in Python/Scanpy, cohesion, dependency management ease
• Use Mixscape for confound correction? Harmony?
• Do we need both Mixscape and Augur?

[easlinger]

Key Resources

• Best Practices for Single-Cell Analysis across Modalities
• Pertpy
  • GitHub
  • Paper
  • Tutorials
    • Website
    • Repository
  • General Usage
• Mixscape
  • GitHub
  • Paper
  • Tutorials
    • Seurat (R)
    • Pertpy (Python)
• Augur
  • GitHub
  • Paper
• MELD and VFC
  • GitHub
  • Paper
• scGen
  • GitHub
  • Paper
  • Tutorial (3rd Party)
• scCODA
  • GitHub
  • Paper
• DIALOGUE
  • GitHub
    • Original (R)
    • Pertpy Implementation (Python)
  • Tutorial

[easlinger]

Per meeting with Hua Jun (08/24/2023), don't need Harmony for his CRISPR screen because normalize to control sample processed with perturbed

He has two guides targeting one gene
Public data we were looking at with "combinatorial" in name have two guides targeting two genes