8000 upgrade cryptography and amazon-kclpy by alexrashed · Pull Request #12005 · localstack/localstack · GitHub
[go: up one dir, main page]
Skip to content

upgrade 8000 cryptography and amazon-kclpy #12005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Dec 11, 2024
Merged

upgrade cryptography and amazon-kclpy #12005

merged 5 commits into from
Dec 11, 2024

Conversation

alexrashed
Copy link
Member
@alexrashed alexrashed commented Dec 9, 2024 •
edited
Loading

Motivation

With #11829 and #11245 we introduced some dependency limits which we need to remove to get up-to-date again.

Changes

  • Remove the pin on cryptography and amazon_kclpy.
  • Upgrade the client config for amazon_kclpy to enable "v2 compatibility mode" (to minimize the amount of necessary changes in this PR while fixing the failing tests).
  • Extend the timeout for test_firehose_kinesis_to_s3 as it was already flaky before.

TODO

  • Fix failing tests:
    • test_firehose_kinesis_to_s3
    • test_lambda_streams_batch_and_transactions
    • test_firehose_stack_with_kinesis_as_source
    • test_run_kcl

TODO in a follow-up

  • Migrate the usage of KCL to v3 (since this PR just enables the v2 compat mode).

@alexrashed alexrashed added the semver: patch Non-breaking changes which can be included in patch releases label Dec 9, 2024
@github-actions GitHub Actions
Copy link
github-actions bot commented Dec 9, 2024 •
edited
Loading

S3 Image Test Results (AMD64 / ARM64)

  2 files    2 suites   4m 10s ⏱️
441 tests 389 ✅  52 💤 0 ❌
882 runs  778 ✅ 104 💤 0 ❌

Results for commit a3c8398.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link
github-actions bot commented Dec 9, 2024 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 53m 15s ⏱️ + 3m 36s
3 881 tests ±0  3 565 ✅ ±0  316 💤 ±0  0 ❌ ±0 
3 883 runs  ±0  3 565 ✅ ±0  318 💤 ±0  0 ❌ ±0 

Results for commit a3c8398. ± Comparison against base commit abb35c3.

♻️ This comment has been updated with latest results.

@alexrashed alexrashed changed the title remove limit on cryptography, update all lock files remove dep limits, update all lock files Dec 9, 2024
@alexrashed alexrashed self-assigned this Dec 9, 2024
@alexrashed alexrashed added this to the 4.1 milestone Dec 9, 2024
@alexrashed alexrashed force-pushed the unpin-cryptography branch 2 times, most recently from 762277f to e0e8b17 Compare December 10, 2024 09:50
This was referenced Dec 10, 2024
Closed
Open
@alexrashed alexrashed requested a review from dfangl December 10, 2024 11:24
@alexrashed alexrashed marked this pull request as ready for review December 10, 2024 11:24
@alexrashed alexrashed marked this pull request as draft December 10, 2024 12:25
@alexrashed alexrashed changed the title remove dep limits, update all lock files upgrade cryptography and amazon-kclpy Dec 10, 2024
dfangl
dfangl reviewed Dec 10, 2024
View reviewed changes
localstack-core/localstack/utils/kinesis/kclipy_helper.py
content = f"""
executableName = {executableName}
streamName = {streamName}
applicationName = {applicationName}
AWSCredentialsProvider = {credentialsProvider}
clientVersionConfig = CLIENT_VERSION_CONFIG_COMPATIBLE_WITH_2x
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8000

Is it necessary to set this?
The AWS help page says:

This KCL 2.x compatibility setting allows your KCL 3.x application to run in a mode compatible with KCL 2.x and use the load balancing algorithm for KCL 2.x until all workers in your consumer application have been upgraded to KCL 3.x. When the migration is complete, KCL will automatically switch to full KCL 3.x functionality mode and start using a new KCL 3.x load balancing algorithm for all running workers.

https://docs.aws.amazon.com/streams/latest/dev/kcl-migration-from-2-3.html

Since we are not migrating anything, using the new load distribution algorithm should have almost no impact on us (unless it uses unsupported features), right?

@alexrashed alexrashed marked this pull request as ready for review December 11, 2024 10:28
@dfangl dfangl self-requested a review December 11, 2024 14:53
dfangl
dfangl approved these changes Dec 11, 2024
View reviewed changes
Copy link
Member
@dfangl dfangl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to get the upgrade :)

@alexrashed alexrashed merged commit 33e59e9 into master Dec 11, 2024
42 checks passed
@alexrashed alexrashed deleted the unpin-cryptography branch December 11, 2024 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dfangl dfangl dfangl approved these changes

Assignees

@alexrashed alexrashed

Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Milestone
4.1
Development

Successfully merging this pull request may close these issues.
2 participants
@alexrashed @dfangl
0