perf: cache .gitignore content to optimize fsmonitor-backed status#2237
perf: cache .gitignore content to optimize fsmonitor-backed status#2237timadamson wants to merge 1 commit intogit:masterfrom
Conversation
Introduce in-memory caching of .gitignore file contents within the untracked cache to improve performance when fsmonitor is active. When fsmonitor confirms a directory is unchanged, we can safely reuse the cached .gitignore content and OID without re-reading and re-hashing the file from disk. This eliminates expensive prep_exclude() calls that would otherwise open, read, and hash every .gitignore file along the path hierarchy. For repositories with many .gitignore files, this provides significant performance improvements during status operations. The optimization works by: - Storing raw .gitignore content in untracked_cache_dir (memory only) - Skipping prep_exclude() entirely when fsmonitor validates the dir - Reusing cached content when rebuilding exclude stacks for invalidated child directories - Tracking .gitignore file changes via fsmonitor to ensure correctness The exclude patterns are still loaded lazily when actually needed for files in invalidated subdirectories. Add trace2 metrics to measure the effectiveness: gitignore-skipped counts directories where prep_exclude was avoided, and gitignore-cached counts reuses of cached content. Co-authored-by: Forge <forge-canva@users.noreply.github.com>
Welcome to GitGitGadgetHi @timadamson, and welcome to GitGitGadget, the GitHub App to send patch series to the Git mailing list from GitHub Pull Requests. Please make sure that either:
You can CC potential reviewers by adding a footer to the PR description with the following syntax: NOTE: DO NOT copy/paste your CC list from a previous GGG PR's description, Also, it is a good idea to review the commit messages one last time, as the Git project expects them in a quite specific form:
It is in general a good idea to await the automated test ("Checks") in this Pull Request before contributing the patches, e.g. to avoid trivial issues such as unportable code. Contributing the patchesBefore you can contribute the patches, your GitHub username needs to be added to the list of permitted
8000
users. Any already-permitted user can do that, by adding a comment to your PR of the form Both the person who commented An alternative is the channel Once on the list of permitted usernames, you can contribute the patches to the Git mailing list by adding a PR comment If you want to see what email(s) would be sent for a After you submit, GitGitGadget will respond with another comment that contains the link to the cover letter mail in the Git mailing list archive. Please make sure to monitor the discussion in that thread and to address comments and suggestions (while the comments and suggestions will be mirrored into the PR by GitGitGadget, you will still want to reply via mail). If you do not want to subscribe to the Git mailing list just to be able to respond to a mail, you can download the mbox from the Git mailing list archive (click the curl -g --user "<EMailAddress>:<Password>" \
--url "imaps://imap.gmail.com/INBOX" -T /path/to/raw.txtTo iterate on your change, i.e. send a revised patch or patch series, you will first want to (force-)push to the same branch. You probably also want to modify your Pull Request description (or title). It is a good idea to summarize the revision by adding something like this to the cover letter (read: by editing the first comment on the PR, i.e. the PR description): To send a new iteration, just add another PR comment with the contents: Need help?New contributors who want advice are encouraged to join git-mentoring@googlegroups.com, where volunteers who regularly contribute to Git are willing to answer newbie questions, give advice, or otherwise provide mentoring to interested contributors. You must join in order to post or view messages, but anyone can join. You may also be able to find help in real time in the developer IRC channel, |
|
There is an issue in commit b9c329c:
|
Introduce in-memory caching of .gitignore file contents within the untracked cache to improve performance when fsmonitor is active.
When fsmonitor confirms a directory is unchanged, we can safely reuse the cached .gitignore content and OID without re-reading and re-hashing the file from disk. This eliminates expensive prep_exclude() calls that would otherwise open, read, and hash every .gitignore file along the path hierarchy. For repositories with many .gitignore files, this provides significant performance improvements during status operations.
The optimization works by:
The exclude patterns are still loaded lazily when actually needed for files in invalidated subdirectories. Add trace2 metrics to measure the effectiveness: gitignore-skipped counts directories where prep_exclude was avoided, and gitignore-cached counts reuses of cached content.
Thanks for taking the time to contribute to Git! Please be advised that the
Git community does not use github.com for their contributions. Instead, we use
a mailing list (git@vger.kernel.org) for code submissions, code reviews, and
bug reports. Nevertheless, you can use GitGitGadget (https://gitgitgadget.github.io/)
to conveniently send your Pull Requests commits to our mailing list.
For a single-commit pull request, please leave the pull request description
empty: your commit message itself should describe your changes.
Please read the "guidelines for contributing" linked above!