ShortCake🍰

A docker image for single-cell analysis. It's on docker-hub and GitHub. This repository is an update of singlecell_jupyter.

0. Changelog

See Changelog

1. Included tools (latest)

(The tools that cannot be installed due to unresolved errors are crossed out.)

• Pipeline: Seurat (and Seurat wrappers), scater, scran, scranPY, Scanpy, scvi-tools (previous scVI), Pagoda2, kallisto-bustools, rapids_singlecell

• Quality check: DropletQC

• Doublet finding: Scrublet, DoubletFinder

• Batch correction and data integration: Harmony, scmap, scBio, SingleCellNet

• Clustering: SC3, metacell, SCCAF, Constclust, bigSCale2, scTriangulate, miloR

• Cell-type annotation: RCA, garnett, scCatch, SingleR, CellTypist, ikarus

• Trajectory (pseudo-time) analysis: Monocle3, slingshot, Palantir, FROWMAP

• RNA velocity: velocyto, scVelo, CellRank, Dynamo, MultiVelo, UniTVelo

• Trajectory alignment: cellAlign, Genes2Genes

• Spatial transcriptome: STELLAR, BANKSY

• Cell-cycle prediction: tricycle

• Gene network: WGCNA, SCENIC (pySCENIC), CellOracle, EEISP

• Cell-cell interaction: CellPhoneDB, SingleCellSignalR, scTensor, cell2cell, CellChat, Scriabin

• Data imputation: MAGIC, SAVER, scImpute, SCRABBLE, RECODE

• Multi-modal analysis: LIGER, scAI, MOFA2, scMoMaT, Mowgli, MARIO, SATURN, Moscot, SCOT

• Bulk deconvolution: SCDC, MuSiC, BayesPrism, InstaPrism, AutoGeneS

• Gene perturbation prediction: GEARS

• Simulation: Splatter, dyngen, scGen, scReadSim, scDesign3

• Others: Sleepwalk, singleCellHaystack, SignatuR, ComplexHeatmap

• scATAC-seq: Cicero, chromVAR, ArchR, Signac, cisTopic, EpiScanpy, SCENT

• Database (genome): BSgenome.Hsapiens.UCSC.hg19, BSgenome.Hsapiens.UCSC.hg38, BSgenome.Mmusculus.UCSC.mm10, BSgenome.Scerevisiae.UCSC.sacCer3, BSgenome.Dmelanogaster.UCSC.dm6

• Database (gene): EnsDb.Hsapiens.v75, EnsDb.Hsapiens.v79, EnsDb.Hsapiens.v86, EnsDb.Mmusculus.v79

• Database (motif): JASPAR2016, JASPAR2018, JASPAR2020, JASPAR2022

• SeuratData: ifnb_3.1.0, panc8_3.0.2, pbmcsca_3.0.0, pbmc3k_3.1.4, celegans.embryo_0.1.0, cbmc_3.1.4, hcabm40k_3.0.0, thp1.eccite_3.1.5, stxBrain_0.1.1, stxKidney_0.1.0, bmcite_0.3.0, pbmcMultiome_0.1.2, ssHippo_3.1.4

2. (new!) Flavors of ShortCake

Since ShortCake version 3, we have created several flavors to reduce the image size and make it easier to use, as shown below.

• shortcake_seurat: Contains only Seurat and its related packages.
• shortcake_r: Contains additional R packages installed on top of shortcake_seurat. Jupyter notebook is available, but Python tools are not installed.
• shortcake_light: Installs the shortcake_default environment on top of shortcake_r. This flavor includes Seurat, Scanpy, Monocle3, and scVelo, and is sufficient for most users.
• shortcake: Installs almost all Python virtual environments on top of shortcake_light. The only exceptions are the scVI and rapids_singlecell environments.
• rnakato/shortcake_scvi: Installs the scVI environment on top of shortcake_light.
• rnakato/shortcake_rapidsc: Installs the shortcake_rapidsc environment on top of shortcake_light.
• shortcake_full: The full image with all tools installed.

For example, you can use shortcake_light version 3.0.0 with this command:

docker run --rm -it rnakato/shortcake_light:3.0.0 jupyternotebook.sh

3. Run

3.1 Docker

For Docker:

You can pull (download) the ShortCake docker image with this command:

docker pull rnakato/shortcake:<version>

Note: The size of the ShortCake docker image is large (about 100GB). It requires a large amount of space on the docker cache directory.

Then you can run ShortCake with the command:

# Container login
docker run [--gpus all] --rm -it rnakato/shortcake /bin/bash

# Execute jupyter notebook (see 'mnt/' directory in the notebook )
docker run [--gpus all] --rm -p 8888:8888 -v (your directory):/work/mnt rnakato/shortcake jupyternotebook.sh

The --gpus all option is needed if you use a GPU (e.g., scvi-tools).

3.2. Singularity

You can build the singularity file (.sif) of ShortCake with this command:

singularity build -F shortcake.sif docker://rnakato/shortcake    

Instead, you can download the singularity image of ShortCake from our Dropbox (We use singularity version 3.8.5).

Then you can run ShortCake with the command:

# Execute jupyter notebook (Python and R)
singularity exec [--nv] shortcake.sif jupyternotebook.sh

# Execute R directory
singularity exec [--nv] shortcake.sif R

The --nv option is needed if you use a GPU (e.g., scvi-tools).

4. Usage

4.1 Virtual environments for Python

To avoid version conflicts between tools, we created several Python environments with micromamba. You can see the list of environments installed in the image with micromamba env list command as follows:

$ docker run -it --rm rnakato/shortcake:3.0.0 micromamba env list                                                                                                                                  (base)
      Name                 Active  Path
    ─────────────────────────────────────────────────────────────────────────
      base                 *       /opt/micromamba
      cell2cell-screadsim          /opt/micromamba/envs/cell2cell-screadsim
      celloracle                   /opt/micromamba/envs/celloracle
      cellphonedb                  /opt/micromamba/envs/cellphonedb
      dynamo-moscot                /opt/micromamba/envs/dynamo-moscot
      episcanpy                    /opt/micromamba/envs/episcanpy
      genes2genes-mowgli           /opt/micromamba/envs/genes2genes-mowgli
      ikarus-novosparc             /opt/micromamba/envs/ikarus-novosparc
      mario                        /opt/micromamba/envs/mario
      seacells                     /opt/micromamba/envs/seacells
      shortcake_default            /opt/micromamba/envs/shortcake_default

Note that the base environment does not include any tools other than Jupyter notebook and EEISP. shortcake_default is the default environment with Python3.9 and contains vairous tools as below:

• scanpy
• scvelo
• cellrank
• harmonypy
• kallisto
• anndata2ri
• autogenes
• bbknn
• cellmap
• celltypist
• doubletdetection
• magic-impute
• liana
• pyscenic
• palantir
• constclust
• multivelo
• screcode
• scrublet
• sctriangulate
• snapatac2
• velocyto

The other environments are named after the tools they contain.

4.2 Jupyter

We recommend using Jupyter Notebook (JupyterLab) to use ShortCake:

singularity exec shortcake.sif jupyternotebook.sh

Specify the appropriate kernel (environment) to use them. In addition, the R command and all R tools are usable in the R kernel.

4.3 Rstudio

ShortCake also provides the Rstudio environment:

singularity exec shortcake.sif rstudio

4.4 Command line

Of course, you can also use ShortCake with command line tools. For example:

singularity exec shortcake.sif velocyto run10x -m repeat_msk.gtf <10Xdir> <gtf>

If you want to use a virtual environment from the command line, use the run_env.sh script to activate it:

singularity exec shortcake.sif run_env.sh <environment> <command>

# Example to activate "celloracle" environment
singularity exec shortcake.sif run_env.sh celloracle python -c "import celloracle"

5. Build the image from Dockerfile

First, clone and move to the repository

git clone https://github.com/rnakato/ShortCake
cd ShortCake

5.1 Get your GitHub token

Since the Dockerfile installs many packages from GitHub, first get a GitHub token from your own repository. Next, create Docker_R/.env and Docker_Python/.env files and store the token as follows:

GITHUB_PAT=<your_GitHub_token>

5.2 Build shortcake_seurat and shortcake_r

Move Docker_R directory:

cd Docker_R/

Download the SeuratData dataset using wget.sh in the Docker_R/SeuratData directory:

cd SeuratData/
sh wget.sh

Then build packages:

cd Docker_R
# build both shortcake_seurat and shortcake_r
docker-compose -f docker-compose.yml build
# build shortcake_seurat only
docker-compose -f docker-compose.yml build seurat 
# build shortcake_r only
docker-compose -f docker-compose.yml build r

5.3 Build other flavors

Move to 'Python' directory:

cd ../Docker_Python/

Then build Python packages:

# build all the flavors below
docker-compose -f docker-compose.yml build
# build shortcake_light
docker-compose -f docker-compose.yml build light
# build shortcake
docker-compose -f docker-compose.yml build default
# build shortcake_full
docker-compose -f docker-compose.yml build full
# build shortcake_scvi
docker-compose -f docker-compose.yml build scvi
# build shortcake_rapidsc
docker-compose -f docker-compose.yml build rapidsc