8000 👨‍🏭 Git-Repo-Scanner Refactoring by paulschmelzer · Pull Request #358 · secureCodeBox/secureCodeBox · GitHub
[go: up one dir, main page]

Skip to content

Conversation

@paulschmelzer
Copy link
Contributor
@paulschmelzer paulschmelzer commented Apr 8, 2021

REFACTORING

This PR refactors #327

Description

Implements a new rate limit configuration

Implements #325

This PR if applied adds a new rate limit configuration to the git-repo-scanner to obey the GitHub and GitLeaks server rate limits. By default this setting is true.

Example

Works for GitHub and Gitlab:

--obey-rate-limit: True to obey the rate limit of the GitHub server (default), otherwise False
  • For unauthenticated requests, the rate limit allows for up to 60 requests per hour. Unauthenticated requests are associated with the originating IP address, and not the user making requests.
    • With --obey-rate-limit=True the scanner will hit the GitHub API every 60 seconds at max (3600s / 60 requests per hour = 60s seconds delay between two request).
  • OAuth applications authorized by a user share the same quota of 5,000 requests per hour when they authenticate with different tokens owned by the same user.
    • With --obey-rate-limit=True the scanner will hit the GitHub API every 0,7 seconds at max (3600s / 5000 requests per hour = 0,7s seconds delay between two request).

Implements a new git repository filter based on latest activity

Implements #320

Example

Works for GitHub and Gitlab:

--activity-since-duration: Return git repo findings with repo activity (e.g. commits) more recent than a specific date expresed by an duration (now + duration). A duration string is a possibly signed sequence of decimal numbers, each
                               with optional fraction and a unit suffix, such as '1h' or '2h45m'. Valid time units are 'm', 'h', 'd', 'w'.
--activity-until-duration: Return git repo findings with repo activity (e.g. commits) older than a specific date expresed by an duration (now + duration). A duration string is a possibly signed sequence of decimal numbers, each with
                               optional fraction and a unit suffix, such as '1h' or '2h45m'. Valid time units are 'm', 'h', 'd', 'w'.

Testing

If you want to test this PR please reference the following docker image: docker pull securecodebox/scanner-git-repo-scanner:git-repo-scanner-refactoring

image:
  # image.repository -- Container Image to run the scan
  repository: docker.io/securecodebox/scanner-git-repo-scanner
  # image.tag -- defaults to the charts version
  tag: git-repo-scanner-refactoring

Example HelmChart Install

You can use the following command to install and test this PR in your environment. For more general informations please check the scanner documentation : https://docs.securecodebox.io/docs/scanners/git-repo-scanner

# Optional: Add the secureCodeBox Helm Repo
helm repo add secureCodeBox https://charts.securecodebox.io
# Install HelmChart (use -n to configure another namespace)
helm upgrade --install git-repo-scanner ./scanner/git-repo-scanner \
  --set="image.repository=docker.io/securecodebox/scanner-git-repo-scanner" \
  --set="image.tag=git-repo-scanner-refactoring"
  --set="parserImage.tag=git-repo-scanner-refactoring"

Checklist

  • Test your changes as thoroughly as possible before you commit them. Preferably, automate your test by unit/integration tests.
  • Make sure npm test runs for the whole project.
  • Cleanup the code to make codeclimate happy

@paulschmelzer paulschmelzer added the scanner Implement or update a security scanner label Apr 8, 2021
@paulschmelzer paulschmelzer self-assigned this Apr 8, 2021
@rfelber rfelber changed the title Git repo scanner refactoring 👨‍🏭 Git-Repo-Scanner Refactoring Apr 8, 2021
@twwd twwd self-assigned this Apr 9, 2021
@rfelber
Copy link
Member
rfelber commented Apr 9, 2021

Is this PR ready for review or still in Draft mode? Otherwise please switch from draft to ready :-)

@paulschmelzer paulschmelzer changed the title 👨‍🏭 Git-Repo-Scanner Refactoring Git-Repo-Scanner Refactoring Apr 9, 2021
@paulschmelzer paulschmelzer marked this pull request as ready for review April 9, 2021 13:04
@twwd twwd requested a review from rfelber April 9, 2021 13:04
@paulschmelzer paulschmelzer changed the title Git-Repo-Scanner Refactoring 👨‍🏭 Git-Repo-Scanner Refactoring Apr 9, 2021
@rfelber rfelber enabled auto-merge April 11, 2021 21:16
@rfelber rfelber disabled auto-merge April 11, 2021 21:19
@rfelber rfelber merged commit 447e5fe into main Apr 12, 2021
@rfelber rfelber deleted the git-repo-scanner-refactoring branch April 12, 2021 09:25
secureCodeBoxBot added a commit that referenced this pull request Nov 18, 2021
Signed-off-by: secureCodeBoxBot <securecodebox@iteratec.com>
## Release changes :
## Summary
 
 CODE_OF_CONDUCT.md CONTRIBUTING.md CONTRIBUTORS.md DCO LICENSE LICENSES Makefile README.md SECURITY.md UPGRADING.md Vagrantfile auto-discovery bin common.mk demo-targets docs hook-sdk hooks hooks.mk lurker operator package-lock.json package-lock.json.license package.json package.json.license parser-sdk scanners scanners.mk sdk.mk tests Adds support for `Job` resources - thank you @superbrothers for this contribution!
 CODE_OF_CONDUCT.md CONTRIBUTING.md CONTRIBUTORS.md DCO LICENSE LICENSES Makefile README.md SECURITY.md UPGRADING.md Vagrantfile auto-discovery bin common.mk demo-targets docs hook-sdk hooks hooks.mk lurker operator package-lock.json package-lock.json.license package.json package.json.license parser-sdk scanners scanners.mk sdk.mk tests Adds option to disable colour output when using `pretty` formatting (package only)
 CODE_OF_CONDUCT.md CONTRIBUTING.md CONTRIBUTORS.md DCO LICENSE LICENSES Makefile README.md SECURITY.md UPGRADING.md Vagrantfile auto-discovery bin common.mk demo-targets docs hook-sdk hooks hooks.mk lurker operator package-lock.json package-lock.json.license package.json package.json.license parser-sdk scanners scanners.mk sdk.mk tests Enabled Dependabot and updates dependencies
 
 ## Changelog
 
 b68cabd Update version to 0.15.0 (#372)
 7f54326 Bump github.com/jetstack/cert-manager from 1.6.0 to 1.6.1 (#371)
 9cdecb3 Bump k8s.io/apiextensions-apiserver from 0.22.2 to 0.22.3 (#368)
 6cc7a75 Add support for kind Job (#370)
 0ef3005 Bump github.com/jetstack/cert-manager from 1.5.4 to 1.6.0 (#365)
 e5c797a Bump k8s.io/apimachinery from 0.22.2 to 0.22.3 (#369)
 d597928 Bump k8s.io/apiextensions-apiserver from 0.21.3 to 0.22.2 (#362)
 79cce8c Remove wip probot (#364)
 700c39f Bump github.com/jetstack/cert-manager from 1.4.1 to 1.5.4 (#363)
 fe44171 Fix dependabot (#358)
 374a428 Support parsing for server specs (#356)
 50c618b Add Support for Services (#353)
 7b57f85 Update dependencies and add dependabot (#354)
 2d8282c Make k8s and override packages public (#351)
 47c31d5 Add option to disable printing results in color (#350)
 ca64457 Remove deprecated mountds auditor (#349)
 863e367 Remove example dependency on internal packages (#348)
 
 
 ## Docker images
 
 - `docker pull shopify/kubeaudit:latest`
 - `docker pull shopify/kubeaudit:0.15.0`
 - `docker pull shopify/kubeaudit:v0.15`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scanner Implement or update a security scanner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

0