Added new git-repo-scanner options (rate limit, activity filter) #327
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was
linked to
issues
Mar 14, 2021
Member
Author
|
This PR already works (on my machine 😆) as expected 🎉 But adding the functionality in a hurry 🏃🏻♂️ results in a bit messy python code 🙄 The codebase must be refactored and we should introduce a more appropriate python class based implementation:
This would also introduce new possibilities to integrate even more features in the |
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implements a new rate limit configuration
Implements #325
This PR if applied adds a new rate limit configuration to the git-repo-scanner to obey the GitHub and GitLeaks server rate limits. By default this setting is true.
Example
Works for GitHub and Gitlab:
--obey-rate-limit=Truethe scanner will hit the GitHub API every 60 seconds at max (3600s / 60 requests per hour = 60s seconds delay between two request).--obey-rate-limit=Truethe scanner will hit the GitHub API every 0,7 seconds at max (3600s / 5000 requests per hour = 0,7s seconds delay between two request).Implements a new git repository filter based on latest activity
Implements #320
Example
Works for GitHub and Gitlab:
--activity-since-duration: Return git repo findings with repo activity (e.g. commits) more recent than a specific date expresed by an duration (now + duration). A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as '1h' or '2h45m'. Valid time units are 'm', 'h', 'd', 'w'. --activity-until-duration: Return git repo findings with repo activity (e.g. commits) older than a specific date expresed by an duration (now + duration). A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as '1h' or '2h45m'. Valid time units are 'm', 'h', 'd', 'w'.Testing
If you want to test this PR please reference the following docker image:
docker pull securecodebox/scanner-git-repo-scanner:git-repo-rate-limitExample HelmChart Install
You can use the following command to install and test this PR in your environment. For more general informations please check the scanner documentation : https://docs.securecodebox.io/docs/scanners/git-repo-scanner
Checklist
npm testruns for the whole project.