Tags: feczo/DataflowJavaSDK
Tags
[BEAM-167] Fix custom source gzip input to read concatenated gzip files This applies patch from kirpichov@google.com from https://gist.github.com/jkff/d8d984a33a41ec607328cee8e418c174
Merge pull request GoogleCloudPlatform#146 from kennknowles/backport-… …pr-38 Backport apache/beam#38
Fixes a bug in custom unbounded readers Custom unbounded readers are read in bundles of at most 10k elements or 10 seconds. A recent change accidentally removed the 10k element limit. This change reintroduces it and adds a test. The previous test also was passing vacuously because the iteration limit was incorrect (it would always have only one iteration). ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=112723469
Occasionally compact combined state on write. This is needed to avoid arbitrarily large blowup for many-element windows. Now, with a fixed probability (currently 0.2%) when writing the state, instead of doing a blind write we read in all accumulated values, combine, and replace everything with the final combined value. ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=109239045
Version management Declare version 1.2.0. Prepare for version 1.2.1. ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=104700491
v1.2.0
40C6Handle exceptions from within the SDK while processing work The DataflowWorkerHarness when processing work wasn't handling general exceptions from being thrown causing the processing threads to all die leaving a zombie worker. ----Release Notes---- [] ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=104664789
PreviousNext