Memory leak with loop in threading.Thread #1192

FunctionFollower · 2024-11-20T10:30:14Z

FunctionFollower
Nov 20, 2024

Hi everyone

There is a function which select large data

def pymysql_test():
    connection = pymysql.connect(
        'database', 
        'webim_service',
         'password'
    )
    try:
        with connection.cursor() as cur:
            cur.execute('large select')
            cur.fetchall()
    finally:
        connection.close()

    res = None
    connection = None
    cur = None

    return None

There is a loop

def loop():
    q = queue.Queue()
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)
    q.put(pymysql_test)

    while True:
        target = q.get()
        print('run target')
        target()
        time.sleep(0.1)

and start

    z = threading.Thread(target=loop)
    z.start()

With this combination, the memory consumption reaches 0.5 GB and is not released after all requests.
If I create arrays of 0.5 GB instead of queries, then the memory is cleared.
If there is no infinite loop (for i in range(10)) then after 10 requests, the memory is also cleared.
if I call loop in the main thread, then everything is fine

Does anyone have any idea what the problem might be?
Thank you in advance

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory leak with loop in threading.Thread #1192

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 0 comments

Select a reply

Memory leak with loop in threading.Thread #1192

FunctionFollower Nov 20, 2024

Replies: 0 comments

FunctionFollower
Nov 20, 2024