WARNING! - Currently not working due to a bug with spikeinterface. Waiting for an update from them. Use the old version for now at the bottom of this README, or comment on the associated issue.
spikesort_docker is a Python‐based command line tool that performs spike sorting on electrophysiological recordings using the SpikeInterface framework. The tool processes recording files by applying bandpass filtering, whitening, spike sorting (via Kilosort4), waveform extraction, and finally exporting the results to Phy for manual curation. It provides full parameter control through command line arguments.
Note: This tool is designed to be run within a Docker container. A Windows batch file (
spikesort.bat) is provided to launch the container interactively so that you can input all the command line parameters.
- Python 3.12
- Conda (recommended for environment/version management)
- Required Python packages:
spikeinterface==0.102.1kilosort==4.0.30
-
Clone the Repository:
git clone https://github.com/padillacoreanolab/spikesort_docker.git cd spikesort_docker -
Install Dependencies:
It is recommended to use Conda for reproducibility. For example:
conda create -n spikesort python=3.12 --yes conda activate spikesort pip install -r requirements.txt
The provided spikesort.bat file pulls the latest Docker image, stops any running container with the same name, and then launches a new container in interactive mode. Inside the container, the Conda environment is automatically activated and your custom parameters are passed to the app.py script.
Example of the Batch File (spikesort.bat):
Key Points:
- The batch file prompts you for every command line parameter available.
- For the
--output-folder, if you press Enter without typing anything, it defaults to the current directory (.). - The host data folder is mounted to
/spikesortin the container. - Inside the container, the script automatically sources the Conda initialization script, activates the
spikesortenvironment, and runsapp.pywith the provided parameters.
If you prefer to run the tool manually, you can run the Docker container interactively:
-
Run the Container:
docker run -it --rm --name spikesort_c --gpus all -v "/path/to/your/data:/spikesort" padillacoreanolab/spikesort:latest bash -
Inside the Container:
The container drops you into a shell. Activate the Conda environment and run the tool with your parameters:
source /opt/miniconda3/etc/profile.d/conda.sh conda activate spikesort python app.py --data-folder /spikesort --output-folder . [other parameters...]
The tool offers a wide range of configurable parameters. Below is a summary of the available arguments:
-
--data-folder(Required):
Path to the folder containing recording files (searched recursively). -
--output-folder(Optional):
Directory where processed output will be saved (default: current directory.). -
--prb-file(Optional):
Path to a.prbfile for probe configuration. If not provided, a default probe file is used. -
--disable-batch(Optional):
When set (true/false), only one recording file will be processed (batch processing disabled). -
--recording-file(Optional):
Specify a single recording file to process (used when batch processing is disabled). -
--stream-id(Optional):
Stream ID to use when reading recording files (default:"trodes").
-
--freq-min(Optional):
Minimum frequency for bandpass filtering (default:300Hz). -
--freq-max(Optional):
Maximum frequency for bandpass filtering (default:6000Hz). -
--whiten-dtype(Optional):
Data type for whitening (default:"float32").
-
--sort-params(Optional):
JSON string to override default sorting parameters (e.g.,{"parameter_name": value}).
Defaults to an empty JSON object ({}). -
--force-cpu(Optional):
Forces sorting to run on CPU even if a GPU is available.
-
--ms-before(Optional):
Milliseconds before spike for waveform extraction (default:1). -
--ms-after(Optional):
Milliseconds after spike for waveform extraction (default:1). -
--n-jobs(Optional):
Number of jobs for waveform extraction (default:8). -
--total-memory(Optional):
Total memory available for waveform extraction (default:"1G"). -
--max-spikes-per-unit(Optional):
Maximum spikes per unit for waveform extraction (default:2000).
-
--compute-pc-features(Optional):
Compute PC features for Phy export (default:True). -
--compute-amplitudes(Optional):
Compute amplitudes for Phy export (default:True). -
--remove-if-exists(Optional):
Remove existing Phy export folder if it exists.
-
Loading & Probe Configuration:
The tool loads recording files and attaches a probe configuration from a provided.prbfile or a default file. -
Preprocessing:
Recordings are bandpass filtered and whitened before spike sorting. -
Spike Sorting:
Spike sorting is performed using Kilosort4. GPU availability is automatically detected (unless overridden with--force-cpu). -
Waveform Extraction:
Extracts waveforms from the sorted data using pre- and post-spike intervals defined by--ms-beforeand--ms-after. -
Phy Export:
Exports the sorted results and extracted waveforms to a format compatible with Phy for manual curation. -
Output Handling:
Processed data is saved into organized subdirectories (e.g.,proc,ss_output,phy), and recordings that have already been processed are skipped to avoid duplication.
-
app.py
Contains the Python code that manages the spike sorting process. -
Dockerfile
Defines the steps to build the Docker image, including setting up the Conda environment and installing required packages. -
nancyprobe_linearprobelargespace.prb(or.txt)
A default example probe configuration file required for spike sorting. -
spikesort.bat
A Windows batch file that logs into Docker, pulls the latest image, prompts for all command line parameters, and then launches a Docker container that runsapp.py. This is the primary file for running the tool on Windows with Docker installed. -
synapse-spike.ico
An icon file that can be used when creating a desktop shortcut forspikesort.bat.
Whenever changes are made to the code (especially in app.py), you must rebuild and push the Docker image so that local runs (via spikesort.bat) use the updated version.
-
Build the Docker Image:
docker build --no-cache -t spikesort:latest . -
Tag the Docker Image:
docker tag spikesort padillacoreanolab/spikesort:latest
-
Push the Docker Image to Docker Hub:
docker push padillacoreanolab/spikesort:latest
Alternatively, you can run the provided update script:
bash update_docker_image.shWhen running the spikesort.bat file, the Docker image is pulled/updated from Docker Hub and a container is spun up with your local data folder mounted at /spikesort.
To run the tool on HiperGator:
-
Create the Conda Environment:
conda create -n spikesort python=3.12 --yes conda activate spikesort pip install spikeinterface==0.102.1 pip install kilosort==4.0.30
-
Clone or Copy 6AC7 the Repository to HiperGator.
-
Copy Data from Dropbox Using rclone.
-
Run the Tool:
Use the
app.pyfile from the command line as usual, supplying your parameters.
-
Create the Conda Environment:
conda create -n spikesort_old python=3.9 --yes conda activate spikesort_old pip install -r requirements_old.txt
-
Clone or Copy the Repository to HiperGator. OR you could only copy the old_app.py file and the appropriate .prb file. These two are all that is required to run.
-
Copy Data from Dropbox Using rclone.
-
Run the Tool:
Use the
old_app.pyfile from the command line as follows.
conda activate spikesort
python old_app.py <source folder coantaining all .rec files> <output directory where the spikesortedo outputs should be saved>Code was written by @ChristopherMarais. For any questions or support, please contact him.
---