[go: up one dir, main page]
Skip to content

Archiving and reusing builds

In some cases, it can be useful to separate out building tests from running them. Nextest supports archiving builds on one machine, and then extracting the archive to run tests on another machine.

Terms

  • Build machine: The computer that builds tests.
  • Target machine: The computer that runs tests.

Use cases

  • Cross-compilation. The build machine has a different architecture, or runs a different operating system, from the target machine.
  • Test partitioning. Build once on the build machine, then partition test execution across multiple target machines.
  • Saving execution time on more valuable machines. For example, build tests on a regular machine, then run them on a machine with a GPU attached to it.

Requirements

  • The project source must be checked out to the same revision on the target machine. This might be needed for test fixtures and other assets, and nextest sets the right working directory relative to the workspace root when executing tests.
  • It is your responsibility to transfer over the archive. Use the examples below as a template.
  • Nextest must be installed on the target machine. For best results, use the same version of nextest on both machines.

Non-requirements

  • Cargo does not need to be installed on the target machine. If cargo is unavailable, replace cargo nextest with cargo-nextest nextest in the following examples.

Creating archives

cargo nextest archive --archive-file <name-of-archive.tar.zst> creates an archive with the following contents:

  • Cargo-related metadata, at the location target/nextest/cargo-metadata.json.
  • Metadata about test binaries, at the location target/nextest/binaries-metadata.json.
  • All test binaries.

  • Other relevant files:

    • Dynamic libraries that test binaries might link to.
    • Non-test binaries used by integration tests.

    • 0.9.66 Build script output directories for workspace packages that have associated test binaries.

      Currently, OUT_DIRs are only archived one level deep to avoid bloating archives too much. In the future, we may add configuration to archive more or less of the output directory. If you have a use case that would benefit from this, please file an issue.

Source code

Currently, archives do not include the source code for your project. It is your responsibility to ensure that the source code for your workspace is transferred over to the target machine and has the same contents.

Support for this may be added in the future, though there are some tricky matters to handle such as whether to transfer over untracked and ignored files.

Currently, the only format supported is a Zstandard-compressed tarball (.tar.zst).

Adding extra files to an archive

0.9.70

Archives can also have extra files included in them. This can be useful if your build process creates artifacts outside of Cargo that are required for Rust tests.

Use the profile.<profile-name>.archive.include configuration option for this. For example:

Extra archive files in .config/nextest.toml
[profile.default]
archive.include = [
    { path = "my-extra-path", relative-to = "target" },
    { path = "other-path", relative-to = "target", depth = 2, on-missing = "error" },
]

archive.include takes a list of tables, with the following parameters:

  • path — The relative path to the archive.
  • relative-to — The root directory that path is defined against. Currently, only "target" is supported, to indicate the target directory.
  • depth — The recursion depth for directories; either a non-negative integer or "infinite". The default is a depth of 16, which should cover most non-degenerate use cases.
  • on-missing — What to do if the specified path was not found. One of "warn" (default), "ignore", or "error".

NOTE: The following features are not currently supported:

  • Excluding subdirectories (#1456).
  • Paths relative to something other than the target directory (#1457).
  • Per-test-binary and per-platform overrides (#1460).

Help on any of these would be greatly appreciated!

Running tests from archives

cargo nextest list and run support a new --archive-file option. This option accepts archives created by cargo nextest archive as above.

By default, archives are extracted to a temporary directory, and nextest remaps paths to use the new target directory. To specify the directory archives should be extracted to, use the --extract-to option.

Specifying a new location for the source code

By default, nextest expects the workspace's source code to be in the same location on both the build and target machines. To specify a new location for the workspace, use the --workspace-remap <path-to-workspace-root> option with the list or run commands.

Example: Simple build/run split

  1. Build and archive tests:

    Here you should specify all the options you would normally use to build your tests.

    cargo nextest archive --workspace --all-features --archive-file my-archive.tar.zst

  2. Run the tests:

    Archive keeps the options used to build the tests, so you should not specify them again.

    cargo nextest run --archive-file my-archive.tar.zst

Example: Use in GitHub Actions

See this working example for how to reuse builds and partition test runs on GitHub Actions.

Example: Cross-compilation

While cross-compiling code, some tests may need to be run on the host platform. (See Filtering by build platform for more.)

On the build machine

  1. Build and run host-only tests:

    cargo nextest run --target <TARGET> -E 'platform(host)'

  2. Archive tests:

    cargo nextest archive --target <TARGET> --archive-file my-archive.tar.zst

  3. Copy my-archive.tar.zst to the target machine.

On the target machine

  1. Check out the project repository to a path <REPO-PATH>, to the same revision as the build machine.

  2. List target-only tests:

    cargo nextest list -E 'platform(target)' \
    --archive-file my-archive.tar.zst \
    --workspace-remap <REPO-PATH>

  3. Run target-only tests:

    cargo nextest run -E 'platform(target)' \
    --archive-file my-archive.tar.zst \
    --workspace-remap <REPO-PATH>

Manually creating your own archives

You can also create and manage your own archives, with the following options to cargo nextest list and run:

  • --binaries-metadata: The path to JSON metadata generated by cargo nextest list --list-type binaries-only --message-format json.
  • --target-dir-remap: A possible new location for the target directory. Requires --binaries-metadata.
  • --cargo-metadata: The path to JSON metadata generated by cargo metadata --format-version 1.

Making tests relocatable

Some tests may need to be modified to handle changes in the workspace and target directories. Some common situations:

  • To obtain the path to the source directory, Cargo provides the CARGO_MANIFEST_DIR option at both build time and runtime. For relocatable tests, use the value of CARGO_MANIFEST_DIR at runtime. This means std::env::var("CARGO_MANIFEST_DIR"), not env!("CARGO_MANIFEST_DIR").

    If the workspace is remapped, nextest automatically sets CARGO_MANIFEST_DIR to the new location.

  • To obtain the path to a crate's executables, Cargo provides the [CARGO_BIN_EXE_<name>] option to integration tests at build time. To handle target directory remapping, use the value of NEXTEST_BIN_EXE_<name> at runtime.

    To retain compatibility with cargo test, you can fall back to the value of CARGO_BIN_EXE_<name> at build time.

Options and arguments for cargo nextest archive

The output of cargo nextest archive -h:

Build and archive tests

Usage: cargo nextest archive [OPTIONS] --archive-file <PATH>

Options:
  -v, --verbose       Verbose output [env: NEXTEST_VERBOSE=]
      --color <WHEN>  Produce color output: auto, always, never [env: CARGO_TERM_COLOR=always]
                      [default: auto]
  -h, --help          Print help (see more with '--help')

Package selection:
  -p, --package <PACKAGES>  Package to test
      --workspace           Test all packages in the workspace
      --exclude <EXCLUDE>   Exclude packages from the test
      --all                 Alias for --workspace (deprecated)

Target selection:
      --lib                Test only this package's library unit tests
      --bin <BIN>          Test only the specified binary
      --bins               Test all binaries
      --example <EXAMPLE>  Test only the specified example
      --examples           Test all examples
      --test <TEST>        Test only the specified test target
      --tests              Test all targets
      --bench <BENCH>      Test only the specified bench target
      --benches            Test all benches
      --all-targets        Test all targets

Feature selection:
  -F, --features <FEATURES>  Space or comma separated list of features to activate
      --all-features         Activate all available features
      --no-default-features  Do not activate the `default` feature

Compilation options:
      --build-jobs <N>        Number of build jobs to run
  -r, --release               Build artifacts in release mode, with optimizations
      --cargo-profile <NAME>  Build artifacts with the specified Cargo profile
      --target <TRIPLE>       Build for the target triple
      --target-dir <DIR>      Directory for all generated artifacts
      --unit-graph            Output build graph in JSON (unstable)
      --timings[=<FMTS>]      Timing output formats (unstable) (comma separated): html, json

Manifest options:
      --manifest-path <PATH>  Path to Cargo.toml
      --frozen                Require Cargo.lock and cache are up to date
      --locked                Require Cargo.lock is up to date
      --offline               Run without accessing the network

Other Cargo options:
      --cargo-quiet...          Do not print cargo log messages (specify twice for no Cargo output
                                at all)
      --cargo-verbose...        Use cargo verbose output (specify twice for very verbose/build.rs
                                output)
      --ignore-rust-version     Ignore `rust-version` specification in packages
      --future-incompat-report  Outputs a future incompatibility report at the end of the build
      --config <KEY=VALUE>      Override a configuration value
  -Z <FLAG>                     Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for
                                details

Archive options:
      --archive-file <PATH>      File to write archive to
      --archive-format <FORMAT>  Archive format [default: auto] [possible values: auto, tar-zst]
      --zstd-level <LEVEL>       Zstandard compression level (-7 to 22, higher is more compressed +
                                 slower) [default: 0]

Config options:
      --config-file <PATH>
          Config file [default: workspace-root/.config/nextest.toml]
      --tool-config-file <TOOL:ABS_PATH>
          Tool-specific config files
      --override-version-check
          Override checks for the minimum version defined in nextest's config
  -P, --profile <PROFILE>
          The nextest profile to use [env: NEXTEST_PROFILE=]

Build and archive tests

Usage: cargo nextest archive [OPTIONS] --archive-file <PATH>

Options:
  -v, --verbose       Verbose output [env: NEXTEST_VERBOSE=]
      --color <WHEN>  Produce color output: auto, always, never [env: CARGO_TERM_COLOR=always]
                      [default: auto]
  -h, --help          Print help (see more with '--help')

Package selection:
  -p, --package <PACKAGES>  Package to test
      --workspace           Test all packages in the workspace
      --exclude <EXCLUDE>   Exclude packages from the test
      --all                 Alias for --workspace (deprecated)

Target selection:
      --lib                Test only this package's library unit tests
      --bin <BIN>          Test only the specified binary
      --bins               Test all binaries
      --example <EXAMPLE>  Test only the specified example
      --examples           Test all examples
      --test <TEST>        Test only the specified test target
      --tests              Test all targets
      --bench <BENCH>      Test only the specified bench target
      --benches            Test all benches
      --all-targets        Test all targets

Feature selection:
  -F, --features <FEATURES>  Space or comma separated list of features to activate
      --all-features         Activate all available features
      --no-default-features  Do not activate the `default` feature

Compilation options:
      --build-jobs <N>        Number of build jobs to run
  -r, --release               Build artifacts in release mode, with optimizations
      --cargo-profile <NAME>  Build artifacts with the specified Cargo profile
      --target <TRIPLE>       Build for the target triple
      --target-dir <DIR>      Directory for all generated artifacts
      --unit-graph            Output build graph in JSON (unstable)
      --timings[=<FMTS>]      Timing output formats (unstable) (comma separated): html, json

Manifest options:
      --manifest-path <PATH>  Path to Cargo.toml
      --frozen                Require Cargo.lock and cache are up to date
      --locked                Require Cargo.lock is up to date
      --offline               Run without accessing the network

Other Cargo options:
      --cargo-quiet...          Do not print cargo log messages (specify twice for no Cargo output
                                at all)
      --cargo-verbose...        Use cargo verbose output (specify twice for very verbose/build.rs
                                output)
      --ignore-rust-version     Ignore `rust-version` specification in packages
      --future-incompat-report  Outputs a future incompatibility report at the end of the build
      --config <KEY=VALUE>      Override a configuration value
  -Z <FLAG>                     Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for
                                details

Archive options:
      --archive-file <PATH>      File to write archive to
      --archive-format <FORMAT>  Archive format [default: auto] [possible values: auto, tar-zst]
      --zstd-level <LEVEL>       Zstandard compression level (-7 to 22, higher is more compressed +
                                 slower) [default: 0]

Config options:
      --config-file <PATH>
          Config file [default: workspace-root/.config/nextest.toml]
      --tool-config-file <TOOL:ABS_PATH>
          Tool-specific config files
      --override-version-check
          Override checks for the minimum version defined in nextest's config
  -P, --profile <PROFILE>
          The nextest profile to use [env: NEXTEST_PROFILE=]

The output of cargo nextest archive --help:

Build and archive tests

This command builds test binaries and archives them to a file. The archive can then be transferred
to another machine, and tests within it can be run with `cargo nextest run --archive-file`.

The archive is a tarball compressed with Zstandard (.tar.zst).

Usage: cargo nextest archive [OPTIONS] --archive-file <PATH>

Options:
  -v, --verbose
          Verbose output

          [env: NEXTEST_VERBOSE=]

      --color <WHEN>
          Produce color output: auto, always, never

          [env: CARGO_TERM_COLOR=always]
          [default: auto]

  -h, --help
          Print help (see a summary with '-h')

Package selection:
  -p, --package <PACKAGES>
          Package to test

      --workspace
          Test all packages in the workspace

      --exclude <EXCLUDE>
          Exclude packages from the test

      --all
          Alias for --workspace (deprecated)

Target selection:
      --lib
          Test only this package's library unit tests

      --bin <BIN>
          Test only the specified binary

      --bins
          Test all binaries

      --example <EXAMPLE>
          Test only the specified example

      --examples
          Test all examples

      --test <TEST>
          Test only the specified test target

      --tests
          Test all targets

      --bench <BENCH>
          Test only the specified bench target

      --benches
          Test all benches

      --all-targets
          Test all targets

Feature selection:
  -F, --features <FEATURES>
          Space or comma separated list of features to activate

      --all-features
          Activate all available features

      --no-default-features
          Do not activate the `default` feature

Compilation options:
      --build-jobs <N>
          Number of build jobs to run

  -r, --release
          Build artifacts in release mode, with optimizations

      --cargo-profile <NAME>
          Build artifacts with the specified Cargo profile

      --target <TRIPLE>
          Build for the target triple

      --target-dir <DIR>
          Directory for all generated artifacts

      --unit-graph
          Output build graph in JSON (unstable)

      --timings[=<FMTS>]
          Timing output formats (unstable) (comma separated): html, json

Manifest options:
      --manifest-path <PATH>
          Path to Cargo.toml

      --frozen
          Require Cargo.lock and cache are up to date

      --locked
          Require Cargo.lock is up to date

      --offline
          Run without accessing the network

Other Cargo options:
      --cargo-quiet...
          Do not print cargo log messages (specify twice for no Cargo output at all)

      --cargo-verbose...
          Use cargo verbose output (specify twice for very verbose/build.rs output)

      --ignore-rust-version
          Ignore `rust-version` specification in packages

      --future-incompat-report
          Outputs a future incompatibility report at the end of the build

      --config <KEY=VALUE>
          Override a configuration value

  -Z <FLAG>
          Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details

Archive options:
      --archive-file <PATH>
          File to write archive to

      --archive-format <FORMAT>
          Archive format

          `auto` uses the file extension to determine the archive format. Currently supported is
          `.tar.zst`.

          [default: auto]
          [possible values: auto, tar-zst]

      --zstd-level <LEVEL>
          Zstandard compression level (-7 to 22, higher is more compressed + slower)

          [default: 0]

Config options:
      --config-file <PATH>
          Config file [default: workspace-root/.config/nextest.toml]

      --tool-config-file <TOOL:ABS_PATH>
          Tool-specific config files

          Some tools on top of nextest may want to set up their own default configuration but
          prioritize user configuration on top. Use this argument to insert configuration that's
          lower than --config-file in priority but above the default config shipped with nextest.

          Arguments are specified in the format "tool:abs_path", for example
          "my-tool:/path/to/nextest.toml" (or "my-tool:C:\\path\\to\\nextest.toml" on Windows).
          Paths must be absolute.

          This argument may be specified multiple times. Files that come later are lower priority
          than those that come earlier.

      --override-version-check
          Override checks for the minimum version defined in nextest's config.

          Repository and tool-specific configuration files can specify minimum required and
          recommended versions of nextest. This option overrides those checks.

  -P, --profile <PROFILE>
          The nextest profile to use.

          Nextest's configuration supports multiple profiles, which can be used to set up different
          configurations for different purposes. (For example, a configuration for local runs and
          one for CI.) This option selects the profile to use.

          [env: NEXTEST_PROFILE=]

Build and archive tests

This command builds test binaries and archives them to a file. The archive can then be transferred
to another machine, and tests within it can be run with `cargo nextest run --archive-file`.

The archive is a tarball compressed with Zstandard (.tar.zst).

Usage: cargo nextest archive [OPTIONS] --archive-file <PATH>

Options:
  -v, --verbose
          Verbose output

          [env: NEXTEST_VERBOSE=]

      --color <WHEN>
          Produce color output: auto, always, never

          [env: CARGO_TERM_COLOR=always]
          [default: auto]

  -h, --help
          Print help (see a summary with '-h')

Package selection:
  -p, --package <PACKAGES>
          Package to test

      --workspace
          Test all packages in the workspace

      --exclude <EXCLUDE>
          Exclude packages from the test

      --all
          Alias for --workspace (deprecated)

Target selection:
      --lib
          Test only this package's library unit tests

      --bin <BIN>
          Test only the specified binary

      --bins
          Test all binaries

      --example <EXAMPLE>
          Test only the specified example

      --examples
          Test all examples

      --test <TEST>
          Test only the specified test target

      --tests
          Test all targets

      --bench <BENCH>
          Test only the specified bench target

      --benches
          Test all benches

      --all-targets
          Test all targets

Feature selection:
  -F, --features <FEATURES>
          Space or comma separated list of features to activate

      --all-features
          Activate all available features

      --no-default-features
          Do not activate the `default` feature

Compilation options:
      --build-jobs <N>
          Number of build jobs to run

  -r, --release
          Build artifacts in release mode, with optimizations

      --cargo-profile <NAME>
          Build artifacts with the specified Cargo profile

      --target <TRIPLE>
          Build for the target triple

      --target-dir <DIR>
          Directory for all generated artifacts

      --unit-graph
          Output build graph in JSON (unstable)

      --timings[=<FMTS>]
          Timing output formats (unstable) (comma separated): html, json

Manifest options:
      --manifest-path <PATH>
          Path to Cargo.toml

      --frozen
          Require Cargo.lock and cache are up to date

      --locked
          Require Cargo.lock is up to date

      --offline
          Run without accessing the network

Other Cargo options:
      --cargo-quiet...
          Do not print cargo log messages (specify twice for no Cargo output at all)

      --cargo-verbose...
          Use cargo verbose output (specify twice for very verbose/build.rs output)

      --ignore-rust-version
          Ignore `rust-version` specification in packages

      --future-incompat-report
          Outputs a future incompatibility report at the end of the build

      --config <KEY=VALUE>
          Override a configuration value

  -Z <FLAG>
          Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details

Archive options:
      --archive-file <PATH>
          File to write archive to

      --archive-format <FORMAT>
          Archive format

          `auto` uses the file extension to determine the archive format. Currently supported is
          `.tar.zst`.

          [default: auto]
          [possible values: auto, tar-zst]

      --zstd-level <LEVEL>
          Zstandard compression level (-7 to 22, higher is more compressed + slower)

          [default: 0]

Config options:
      --config-file <PATH>
          Config file [default: workspace-root/.config/nextest.toml]

      --tool-config-file <TOOL:ABS_PATH>
          Tool-specific config files

          Some tools on top of nextest may want to set up their own default configuration but
          prioritize user configuration on top. Use this argument to insert configuration that's
          lower than --config-file in priority but above the default config shipped with nextest.

          Arguments are specified in the format "tool:abs_path", for example
          "my-tool:/path/to/nextest.toml" (or "my-tool:C:\\path\\to\\nextest.toml" on Windows).
          Paths must be absolute.

          This argument may be specified multiple times. Files that come later are lower priority
          than those that come earlier.

      --override-version-check
          Override checks for the minimum version defined in nextest's config.

          Repository and tool-specific configuration files can specify minimum required and
          recommended versions of nextest. This option overrides those checks.

  -P, --profile <PROFILE>
          The nextest profile to use.

          Nextest's configuration supports multiple profiles, which can be used to set up different
          configurations for different purposes. (For example, a configuration for local runs and
          one for CI.) This option selects the profile to use.

          [env: NEXTEST_PROFILE=]