8000 Skip flaky transcribe tests by tiurin · Pull Request #12473 · localstack/localstack · GitHub
[go: up one dir, main page]
Skip to content

Skip flaky transcribe tests #12473

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 3, 2025
Merged

Skip flaky transcribe tests #12473

merged 2 commits into from
Apr 3, 2025

Conversation

tiurin
Copy link
Contributor
@tiurin tiurin commented Apr 3, 2025

Motivation

In this run a couple of errors hint at flakiness:

  • AssertionError: assert 'IN_PROGRESS' == 'COMPLETED'
  • AssertionError: could not finish transcription job: test-transcribe-e98d9bf3 in time .

Changes

Added skip markers.

@tiurin tiurin requested a review from sannya-singal as a code owner April 3, 2025 06:22
@tiurin tiurin added semver: patch Non-breaking changes which can be included in patch releases aws:transcribe Amazon Transcribe labels Apr 3, 2025
@github-actions GitHub Actions
Copy link
github-actions bot commented Apr 3, 2025 •
edited
Loading

LocalStack Community integration with Pro

 2 files  ±    0  2 suites  ±0   28s ⏱️ - 1h 53m 19s
25 tests  - 4 288  6 ✅  - 3 978  19 💤  - 310  0 ❌ ±0 
27 runs   - 4 288  6 ✅  - 3 978  21 💤  - 310  0 ❌ ±0 

Results for commit 76dcfac. ± Comparison against base commit 245ebe0.

This pull request removes 4288 tests. 
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_lambda_dynamodb
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_opensearch_crud
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_search_books
tests.aws.scenario.bookstore.test_bookstore.TestBookstoreApplication ‑ test_setup
tests.aws.scenario.kinesis_firehose.test_kinesis_firehose.TestKinesisFirehoseScenario ‑ test_kinesis_firehose_s3
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_destination_sns
tests.aws.scenario.lambda_destination.test_lambda_destination_scenario.TestLambdaDestinationScenario ‑ test_infra
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_prefill_dynamodb_table
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input0-SUCCEEDED]
tests.aws.scenario.loan_broker.test_loan_broker.TestLoanBrokerScenario ‑ test_stepfunctions_input_recipient_list[step_function_input1-SUCCEEDED]
…
This pull request skips 15 tests. 
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_happy_path
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[None-None]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-2-None]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-3-test-output]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-4-test-output.json]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-5-test-files/test-output.json]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-6-test-files/test-output]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_supported_media_formats[../../files/en-gb.amr-hello my name is]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_supported_media_formats[../../files/en-gb.flac-hello my name is]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_supported_media_formats[../../files/en-gb.mp3-hello my name is]
…

♻️ This comment has been updated with latest results.

sannya-singal
sannya-singal reviewed Apr 3, 2025
View reviewed changes
Copy link
Contributor
@sannya-singal sannya-singal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for jumping on this 🦸 🚀 LGTM 👍

@tiurin
Copy link
Contributor Author
tiurin commented Apr 3, 2025 •
edited
Loading

@sannya-singal had to also add skip for test_transcribe_start_job as it failed during CI run for this PR.
It was also skipped before in #12260

@tiurin tiurin merged commit e86e0c4 into master Apr 3, 2025
31 checks passed
@tiurin tiurin deleted the skip-flaky-transcribe-tests branch April 3, 2025 07:06
@sannya-singal sannya-singal mentioned this pull request Apr 4, 2025
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sannya-singal sannya-singal sannya-singal approved these changes

Assignees
No one assigned
Labels
aws:transcribe Amazon Transcribe semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tiurin @sannya-singal
0