8000 docs: 📝 update developer guide · datajoint/datajoint-python@275e0d7 · GitHub
[go: up one dir, main page]
Skip to content

Commit 275e0d7

Browse files
author
Drew Yang
committed
docs: 📝 update developer guide
1 parent 04ea244 commit 275e0d7

File tree

2 files changed

+127
-51
lines changed

2 files changed

+127
-51
lines changed

‎README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,6 @@ DataJoint (https://datajoint.com).
123123
- [DataJoint Elements](https://datajoint.com/docs/elements/) - Catalog of example pipelines for neuroscience experiments
124124

125125
- Contribute
126-
- [Development Environment](https://datajoint.com/docs/core/datajoint-python/latest/develop/)
126+
- [Contribution Guidelines](https://datajoint.com/docs/about/contribute/)
127127

128-
- [Guidelines](https://datajoint.com/docs/about/contribute/)
128+
- [Developer Guide](https://datajoint.com/docs/core/datajoint-python/latest/develop/)

‎docs/src/develop.md

Lines changed: 125 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,66 @@
1-
# Develop
1+
# Developer Guide
22

3-
Included with the codebase is the recommended development environment configured using [DevContainer](https://containers.dev/).
3+
## Table of Contents
44

5-
## Launch Environment
5+
- [Contribute to DataJoint Python Documentation](#contribute-to-datajoint-python-documentation)
6+
- [Setup Development Environment](#setup-development-environment)
7+
- [Prerequisites](#prerequisites)
8+
- [With Virtual Environment](#with-virtual-environment)
9+
- [With DevContainer](#with-devcontainer)
10+
- [Extra Efficiency, Optional But Recommended](#extra-efficiency-optional-but-recommended)
11+
- [Pre-commit Hooks](#pre-commit-hooks)
12+
- [Integration Tests](#integration-tests)
13+
- [VSCode](#vscode)
14+
- [Jupyter Extension](#jupyter-extension)
15+
- [Debugger](#debugger)
16+
- [MySQL CLI](#mysql-cli)
17+
18+
## Contribute to DataJoint Python Documentation
19+
20+
> Contributions to documentations are equivalently important to any code for the community, please help us to resolve any confusions in documentations.
21+
22+
[Here](https://github.com/datajoint/datajoint-python/blob/master/docs/README.md) is the instructions for contributing documentations, or you can find the same instructions at `$PROJECT_DIR/docs/README.md` in the repository.
23+
24+
[Back to top](#table-of-contents)
25+
26+
## Setup Development Environment
27+
28+
> We have [DevContainer](https://containers.dev/) ready for contributors to develop without setting up their environment. If you are familiar with DevContainer, Docker or Github Codespace, this is the recommended development environment for you.
29+
> If you have never used Docker, it might be easier for you to use a virtual environment through `conda/mamba/venv`, it is also very straightforward to set up.
30+
31+
### Prerequisites
32+
33+
- Clone datajoint-python repository
34+
35+
```bash
36+
# If you have your SSH key set up with GitHub, you can clone using SSH
37+
git clone git@github.com:datajoint/datajoint-python.git
38+
# Otherwise, you can clone using HTTPS
39+
git clone https://github.com/datajoint/datajoint-python.git
40+
```
41+
- If you don't use DevContainer, then either install Anaconda/[Miniconda](https://www.anaconda.com/docs/getting-started/miniconda/install)/Mamba, or just use Python's built-in `venv` module without install anything else.
42+
43+
### With Virtual Environment
44+
45+
```bash
46+
# Check if you have Python 3.9 or higher, if not please upgrade
47+
python --version
48+
# Create a virtual environment with venv
49+
python -m venv .venv
50+
source .venv/bin/activate
51+
pip install -e .[dev]
52+
53+
# Or create a virtual environment with conda
54+
conda create -n dj python=3.13 # any 3.9+ is fine
55+
conda activate dj
56+
pip install -e .[dev]
57+
```
58+
59+
[Back to top](#table-of-contents)
60+
61+
### With DevContainer
62+
63+
#### Launch Environment
664

765
Here are some options that provide a great developer experience:
866

@@ -26,37 +84,48 @@ Here are some options that provide a great developer experience:
2684
- Issue the following command in the terminal to build and run the Docker container: `HOST_UID=$(id -u) PY_VER=3.11 DJ_VERSION=$(grep -oP '\d+\.\d+\.\d+' datajoint/version.py) docker compose --profile test run --rm -it djtest -- sh -c 'pip install -qe ".[dev]" && bash'`
2785
- Issue the following command in the terminal to stop the Docker compose stack: `docker compose --profile test down`
2886

29-
## Features
87+
[Back to top](#table-of-contents)
3088

31-
Once you've successfully launched the development environment, you'll be able to take advantage of our developer tooling to help improve productivity and quality.
89+
## Extra Efficiency, Optional But Recommended
3290

33-
### Syntax Tests
91+
### Pre-commit Hooks
3492

35-
The following will verify that there are no syntax errors.
93+
We recommend using [pre-commit](https://pre-commit.com/) to automatically run linters and formatters on your code before committing.
94+
To set up pre-commit, run the following command in your terminal:
3695

96+
```bash
97+
pip install pre-commit
98+
pre-commit install
3799
```
38-
flake8 datajoint --count --select=E9,F63,F7,F82 --show-source --statistics
100+
101+
You can manually run pre-commit on all files with the following command:
102+
103+
```bash
104+
pre-commit run --all-files
39105
```
106+
This will run all the linters and formatters specified in the `.pre-commit-config.yaml` file. If all check passed, you can commit your code. Otherwise, you need to fix the failed checks and run the command again.
40107

41-
### Integration Tests
108+
> Pre-commit will automatically run the linters and formatters on all staged files before committing. However, if your code doesn't follow the linters and formatters, the commit will fail.
109+
> Some hooks will automatically fix your problem, and add the fixed files as git's `unstaged` files, you just need to add them(`git add .`) to git's `staged` files and commit again.
110+
> Some hooks will not automatically fix your problem, so you need to check the pre-commit failed log to fix them manually and include the update to your `staged` files and commit again.
42111
43-
The following will verify there are no regression errors by running our test suite of unit and integration tests.
112+
If you really don't want to use pre-commit, or if you don't like it, you can uninstall it with the following command:
44113

45-
- Entire test suite:
46-
```
47-
pytest -sv --cov-report term-missing --cov=datajoint tests
48-
```
114+
```bash
115+
pre-commit uninstall
116+
```
49117

50-
- A single functional test:
51-
```
52-
pytest -sv tests/test_connection.py::test_dj_conn
53-
```
54-
- A single class test:
55-
```
56-
pytest -sv tests/test_aggr_regressions.py::TestIssue558
57-
```
118+
But when you issue a pull request, the same linter and formatter check will run against your contribution, you are going to have the same failure as well. So without pre-commit, you need to **manually run these linters and formatters before committing your code**:
119+
120+
- Syntax tests
121+
122+
The following will verify that there are no syntax errors.
58123

59-
### Style Tests
124+
```
125+
flake8 datajoint --count --select=E9,F63,F7,F82 --show-source --statistics
126+
```
127+
128+
- Style tests
60129

61130
The following will verify that there are no code styling errors.
62131

@@ -67,22 +136,44 @@ flake8 --ignore=E203,E722,W503 datajoint --count --max-complexity=62 --max-line-
67136
The following will ensure the codebase has been formatted with [black](https://black.readthedocs.io/en/stable/).
68137

69138
```
70-
black datajoint --check -v
139+
black datajoint --check -v --diff
71140
```
72141

73142
The following will ensure the test suite has been formatted with [black](https://black.readthedocs.io/en/stable/).
74143

75144
```
76-
black tests --check -v
145+
black tests --check -v --diff
77146
```
78147

79-
### Jupyter
148+
[Back to top](#table-of-contents)
149+
150+
### Integration Tests
151+
152+
The following will verify there are no regression errors by running our test suite of unit and integration tests.
153+
154+
- Entire test suite:
155+
```
156+
pytest -sv --cov-report term-missing --cov=datajoint tests
157+
```
158+
159+
- A single functional test:
160+
```
161+
pytest -sv tests/test_connection.py::test_dj_conn
162+
```
163+
- A single class test:
164+
```
165+
pytest -sv tests/test_aggr_regressions.py::TestIssue558
166+
```
167+
168+
[Back to top](#table-of-contents)
80169

81-
Jupyter notebooks are supported in this environment. This means that when you `import datajoint`, it will use the current state of the source.
170+
### VSCode
82171

83-
Be sure to see the reference documentation if you are new to [running Jupyter notebooks w/ VSCode](https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_create-or-open-a-jupyter-notebook).
172+
#### Jupyter Extension
84173

85-
### Debugger
174+
Be sure to go through this documentation if you are new to [Running Jupyter Notebooks with VSCode](https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_create-or-open-a-jupyter-notebook).
175+
176+
#### Debugger
86177

87178
[VSCode Debugger](https://code.visualstudio.com/docs/editor/debugging) is a powerful tool that can really accelerate fixes.
88179

@@ -94,8 +185,12 @@ Try it as follows:
94185
- Select the `Run and Debug` tab
95186
- Start by clicking the button `Run and Debug`
96187

188+
[Back to top](#table-of-contents)
189+
97190
### MySQL CLI
98191

192+
> Installation instruction is in [here](https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-install.html)
193+
99194
It is often useful in development to connect to DataJoint's relational database backend directly using the MySQL CLI.
100195

101196
Connect as follows to the database running within your developer environment:
@@ -104,23 +199,4 @@ Connect as follows to the database running within your developer environment:
104199
mysql -hdb -uroot -ppassword
105200
```
106201

107-
### Documentation
108-
109-
Our documentation is built using [MkDocs Material](https://squidfunk.github.io/mkdocs-material/). The easiest way to improve the documentation is by using the `docs/docker-compose.yaml` environment. The source can be modified in `docs/src` using markdown.
110-
111-
The docs environment can be run using 3 modes:
112-
113-
- **LIVE**: (*recommended*) This serves the docs locally. It supports live reloading on saves to `docs/src` files but does not support the docs version dropdown. Useful to see changes live.
114-
```
115-
MODE="LIVE" PACKAGE=datajoint UPSTREAM_REPO=https://github.com/datajoint/datajoint-python.git HOST_UID=$(id -u) docker compose -f docs/docker-compose.yaml up --build
116-
```
117-
- **QA**: This serves the docs locally. It supports the docs version dropdown but does not support live reloading. Useful as a final check.
118-
```
119-
MODE="QA" PACKAGE=datajoint UPSTREAM_REPO=https://github.com/datajoint/datajoint-python.git HOST_UID=$(id -u) docker compose -f docs/docker-compose.yaml up --build
120-
```
121-
- **BUILD**: This compiles the docs. Most useful for the docs deployment automation. Other modes are more useful to new contributors.
122-
```
123-
MODE="BUILD" PACKAGE=datajoint UPSTREAM_REPO=https://github.com/datajoint/datajoint-python.git HOST_UID=$(id -u) docker compose -f docs/docker-compose.yaml up --build
124-
```
125-
126-
When the docs are served locally, use the VSCode `PORTS` tab (next to `TERMINAL`) to manage access to the forwarded ports. Docs are served on port `8080`.
202+
[Back to top](#table-of-contents)

0 commit comments

Comments
 (0)
0