Add benchmarks for blame #6920
Merged
Add benchmarks for blame #6920
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Callers may wish to show all the options, even hidden ones, when showing usage. In particular, showing generic help for the CLI should show global options (those that are generally "hidden"). But showing help for a particular command should keep them hidden. Instrument a mechanism to deal with this.
When the `help` command is invoked (because no command was specifi 8000 ed) we may need to clean up the arguments, in particular, to avoid passing `--help` (so that the help command isn't confused, and assumes that it's being invoked as `help --help`).
Move the common option information to a global place, and reuse them. Common options will be global variables. Specify them as _hidden_ for all commands (the main command will pass the SHOW_HIDDEN flag to the usage printer, so that they're visible).
Add a `--no-pager` option for git compatibility.
Teach the benchmark script how to clone the git or kernel repositories, which is useful to have a larger corpus of test data. If a benchmark script wants to `clone git` or `clone linux`, then this will be done. Callers probably want to specify `BENCHMARK_GIT_REPOSITORY` to a previously cloned local repository so that the script does not download the repository repeatedly.
It will never be a good user experience to clone the git or kernel repos from a remote; don't even have a default.
When running without a local test repository, show a helpful error message.
Allow for workflow_dispatch jobs to run, but don't publish their test results; this is useful for testing workflows themselves.
blame on torvalds/linux is too punishing for our current implementation; don't run it (yet).
If a test needs a repo that isn't provide it, mark it as skipped and avoid failing the execution.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
You can't improve what you can't measure; our blame implementation has historically been a poor performer. Add a benchmark suite so that we can start to track improvements.