8000 bpo-41735: Fix thread locks in zlib module may go wrong in rare case. · Pull Request #22126 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

bpo-41735: Fix thread locks in zlib module may go wrong in rare case. #22126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from Apr 27, 2021
Merged

bpo-41735: Fix thread locks in zlib module may go wrong in rare case. #22126

merged 2 commits into from Apr 27, 2021

Conversation

ghost
Copy link
@ghost ghost commented Sep 7, 2020 •
edited by bedevere-bot
Loading

@wjssz
Fix thread locks in zlib module may go wrong in rare case.
10a3fa8 
Setting `next_in` before acquiring the thread lock may mix up compress/decompress state in other threads.
@iritkatriel
Copy link
Member

Can you add a unit test that fails without the fix and passes with it?

@ghost
Copy link
Author
ghost commented Sep 7, 2020

Can you add a unit test that fails without the fix and passes with it?

Run this code:

import zlib
import threading

with open('txt.tar', 'rb') as f:
    b = f.read()
    b = b[:100*1024*1024]
print('read', len(b))

cobj = zlib.compressobj()

def f(name, cobj, b):
    print(name)
    dat = cobj.compress(b)
    print(name, len(dat))

thread1 = threading.Thread(target=f, args=('T1', cobj, b))
thread2 = threading.Thread(target=f, args=('T2', cobj, b))
thread1.start()
thread2.start()

print('ok')

Before the patch, output:

read 104857600
T1
T2
ok
T1 57142315

It seems thread2 did not finish the compressing.

After the patch, output:

read 104857600
T1
T2
ok
T1 57165411
T2 57166659

@ghost
Copy link
Author
ghost commented Sep 7, 2020

Can you add a unit test that fails without the fix and passes with it?

Seems a little troublesome. If the data is large, it may be time-consuming. If the data is small, it may not trigger the bug.

Above code can reliably reproduce the bug.

ambv
ambv reviewed Apr 26, 2021
View reviewed changes
@ambv ambv merged commit 93f4118 into python:master Apr 27, 2021
@bedevere-bot
Copy link

@ambv: Please replace # with GH- in the commit message next time. Thanks!

@ghost ghost deleted the thread_lock branch April 27, 2021 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

5 participants
@iritkatriel @bedevere-bot @ambv @the-knights-who-say-ni @wjssz
0