Development

Development guide for stdlib.

Introduction

We are super excited that you have decided to develop stdlib, and we welcome you to the stdlib developer community. We have done our best to provide a complete environment for testing, benchmarking, documenting, and developing project code. And if you have any ideas as to how we can make it better, let us know!

Before we begin, developing stdlib requires some setup and configuration. What follows is an overview of environment requirements and a sequence of steps for getting up and running with stdlib. We use Git for version control, and for most tasks, we use GNU make (the original task runner) to help us get things done quickly and effectively. For the most part, the project is able to internally manage dependencies for testing, benchmarking, and linting, so, once you follow the steps below, you should be ready to start developing!

So, without further ado, let's get you started!

Prerequisites

Developing and running stdlib requires the following prerequisites:

Git: version control
GNU make: development utility and task runner
GNU bash: an sh-compatible shell
curl, wget, or fetch (FreeBSD): utilities for downloading remote resources
Node.js: JavaScript runtime (version >= 0.10; although the latest stable version is strongly recommended)
npm: package manager
- version > 2.7.0
- if Node < 1.0.0, version > 2.7.0 and < 4.0.0
- if Node < 10.x.x, version > 2.7.0 and < 6.0.0
- if Node < 14.17.0, version > 2.7.0 and < 9.0.0

While not required to run stdlib, the following dependencies may be required for testing, benchmarking, and general development:

Julia: language for technical computing (version >= 1.0)
R: language for statistical computing (version >= 3.4.0)
Python: general purpose language (version >=2.7.x)
pip: Python package manager (version >= 9.0.0; required for automatically installing Python packages, such as lint tools)
gcc & g++ or Clang: C/C++ compilation and linking (g++ version >= 4.8; clang version >= 3.5, Xcode version >=8.3.1 on OS X)
gfortran: Fortran compilation and linking (version >= 4.8)
CMake: cross-platform build environment (version >= 3.4.3)
pandoc: universal document converter (version >= 1.18)
Homebrew: macOS package manager (only required on OS X in order to install shellcheck)

Assuming the requisite language is present on the host machine, the following language libraries can be automatically downloaded and installed using make (see installation):

NumPy: general purpose array-processing library for Python
SciPy: Python library containing numerical routines
Pylint: Python source code analyzer
pycodestyle: Python style guide checker against PEP 8
pydocstyle: Python docstring checker against PEP 257
lintr: static code analysis for R
shellcheck: static code analysis for shell scripts
cppcheck: C/C++ static code analysis

The following external libraries can be automatically downloaded and compiled from source using make (see installation):

Boost: portable C++ libraries
Cephes: C/C++ special functions math library
OpenBLAS: optimized BLAS library
Electron: framework for cross-platform desktop applications
Emscripten: LLVM to JavaScript compiler
WebAssembly Binary Toolkit: suite of tools for WebAssembly

Download

To acquire the source code, first navigate to the parent directory into which you want to place the project Git repository. Because of this project's heavy reliance on GNU make, the directory path should not include spaces or any other shell meta characters such as $ or :, as these characters will cause GNU make and the installation process to fail.

$ cd /path/to/parent/destination/directory

Next, clone the repository.

$ git clone https://github.com/stdlib-js/stdlib.git

If you are wanting to contribute to stdlib, first fork the repository and amend the previous command.

$ git clone https://github.com/<username>/stdlib.git

where <username> is your GitHub username (assuming you are using GitHub to manage public repositories). The repository has a large commit history, leading to slow download times. You can reduce the download time by limiting the clone depth.

$ git clone --depth=<depth> https://github.com/<username>/stdlib.git

where <depth> refers to the number of commits you want to download (as few as 1 and as many as the entire project history). However, limiting clone depth can cause difficulties later when attempting to rebase a pull request on the latest development branch. For simple pull requests, limiting clone depth is likely to work out fine; however, for more complex pull requests, including those depending on upstream changes, limiting clone depth may be a source of Git errors (e.g., due to unrelated Git histories), and, thus, you may be forced to re-clone the repository and start over.

If you are behind a firewall, you may need to use the https protocol, rather than the git protocol.

$ git config --global url."https://".insteadOf git://

Once you have finished cloning the repository into the destination directory, you should see the folder stdlib. To proceed with configuring your environment, navigate to the project folder.

$ cd stdlib

Installation

To install dependencies (e.g., Node.js module dependencies),

$ make install

To run dependency diagnostics,

$ make deps-diagnostics

which will check for various development dependencies. Depending on your host system, you may be missing one or more dependencies. For most day-to-day stdlib development, these dependencies are not necessary and, thus, you do not need to immediately install them. If you are focusing on certain development areas (e.g., adding new math functionality), you'll be able to install the various dependencies on a case-by-case basis (e.g., requisite C/C++ linters, reference libraries, et cetera) using project tooling. For now, feel free to move along, and, if you get stuck or have questions, feel free to reach out to one of the core project maintainers.

To initialize the development environment,

$ make init

Initializing the development environment configures Git hooks and other bells and whistles to aid in development. Git hooks are especially important as they enable automatic linting and testing to ensure that code meets style specifications and does not break.

Verification

To verify your environment, run a sample of project tests.

$ make TESTS_FILTER=".*/math/base/special/sin/.*" test
$ make EXAMPLES_FILTER=".*/math/base/special/sin/.*" examples
$ make BENCHMARKS_FILTER=".*/math/base/special/sin/.*" benchmark

If your environment is properly configured, each command should exit without errors.

Update

If you have previously downloaded stdlib using git clone, you can update an existing source tree from the base project directory using git pull.

$ git pull

If you are working with a forked repository and wish to sync your local repository with the upstream project (i.e., incorporate changes from the main project repository into your local repository), assuming you have configured a remote which points to the upstream repository,

$ git pull --ff upstream <branch>

where upstream is the remote name and <branch> refers to the branch you want to merge into your local copy.

If you have initialized the development environment using make init, updating the source tree will trigger hooks to ensure all development dependencies are up-to-date.

Organization

The stdlib source code is organized as follows:

bin        executable binaries
deps       external library dependencies
dist       distributable files
docs       top-level documentation
etc        configuration files
examples   top-level library examples
lib        library source code
test       top-level tests
tools      development utilities
workshops  workshops

Troubleshooting

Occasionally, new versions of external dependencies may cause conflicts with existing builds. Most of the time, running
```
$ make clean
$ make install
```
will be enough to resolve these conflicts. Otherwise, remove the Git repository, clone, and reinstall.
Some terminal prompts display the current Git branch and its status. Displaying the status may add significant lag to your terminal. You may hide this information within the repository directory by updating the repository's Git configuration. For example, if using GNU bash,
```
$ git config bash.showDirtyState false
```
or if using Zsh,
```
$ git config --add oh-my-zsh.hide-dirty 1
```
Some shells (e.g., Zsh) may require quotes around environment variables to prevent the shell from expanding paths. If this is the case, wrap paths, or values which may be interpreted as paths, in quotes. For example, replace
```
$ make TESTS_FILTER=.*/<pattern>/.* test
```
with
```
$ make TESTS_FILTER=".*/<pattern>/.*" test
```

Editors

This repository uses EditorConfig to maintain consistent coding styles between different editors and IDEs, including browsers. You should be sure to download and setup EditorConfig to ensure that files are automatically configured to use expected indentation and line endings.

Testing

To run all project tests,

$ make test

To run select tests,

$ make TESTS_FILTER=".*/<pattern>/.*" test

where <pattern> is a pattern matching a particular path. For example, to test only base special math functions

$ make TESTS_FILTER=".*/math/base/special/.*" test

where the pattern .*/math/base/special/.* matches any test file whose absolute path contains math/base/special.

To generate a test coverage report,

$ make TESTS_FILTER=".*/<pattern>/.*" test-cov
$ make view-cov

If you are developing a tool (i.e., code located in the tools directory), to run tools tests

$ make TESTS_FILTER=".*/<pattern>/.*" tools-test
$ make TESTS_FILTER=".*/<pattern>/.*" tools-test-cov
$ make view-cov

Similarly, to run benchmarks

$ make BENCHMARKS_FILTER=".*/<pattern>/.*" benchmark

and examples

$ make EXAMPLES_FILTER=".*/<pattern>/.*" examples

Contributing

For contribution guidelines, see the contributing guide.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!