8000 [BUG] [stdlib] Python interop is unsafe for any use off thread#0 (including async, parallel, etc.) · Issue #4724 · modular/modular · GitHub
[go: up one dir, main page]

Skip to content
[BUG] [stdlib] Python interop is unsafe for any use off thread#0 (including async, parallel, etc.) #4724
@miili

Description

@miili

Bug description

Actual behavior

Calling a Mojo function using parallelize from Python crashes with

(gdb) run main.py
Starting program: /home/marius/Development/parstack-mojo/.venv/bin/python3 main.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Initializing Python Mojo module...
[Detaching after vfork from child process 15506]
[New Thread 0x7fffdffff6c0 (LWP 15635)]
[New Thread 0x7fffdf7fe6c0 (LWP 15636)]
[New Thread 0x7fffdeffd6c0 (LWP 15637)]
[New Thread 0x7fffde7fc6c0 (LWP 15638)]
[New Thread 0x7fffddffb6c0 (LWP 15639)]
[New Thread 0x7fffdd7fa6c0 (LWP 15640)]
[New Thread 0x7fffdcff96c0 (LWP 15641)]

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007ffff63ba2d0 in KGEN_CompilerRT_AsyncRT_ParallelismLevel ()
   from /home/[...]/parstack-mojo/.venv/lib/python3.13/site-packages/max/lib/libKGENCompilerRTShared.so
(gdb) backtrace
#0  0x00007ffff58b02d0 in KGEN_CompilerRT_AsyncRT_ParallelismLevel () from /home/.../Development/parstack-mojo/.venv/lib/python3.13/site-packages/max/lib/libKGENCompilerRTShared.so
#1  0x00007ffff7e2e0ce in argmax_masked::argmax_masked(::PythonObject,::PythonObject) () from /home/.../Development/parstack-mojo/__mojocache__/argmax_masked.hash-c33146e5e9f20584.so
#2  0x00007ffff7e3952f in stdlib::python::bindings::PythonModuleBuilder::def_py_function[fn(mut ::PythonObject, mut ::PythonObject) raises -> ::PythonObject](::PythonModuleBuilder&,::StringSlice[::Bool(False), ::Origin[::Bool(False)](StaticConstantOrigin)],::StringSlice[::Bool(False), ::Origin[::Bool(False)](StaticConstantOrigin)]),func="stdlib::python::bindings::PythonModuleBuilder::def_function[AnyTrivialRegType,::PyObjectFunction[$0, ::Bool(False)]](::PythonModuleBuilder&,::StringSlice[::Bool(False), ::Origin[::Bool(False)](StaticConstantOrigin)],::StringSlice[::Bool(False), ::Origin[::Bool(False)](StaticConstantOrigin)])_wrapper(::PythonObject&,::PythonObject&)"<:type [(!kgen.pointer<typevalue<#kgen.instref<tdlib::python::python_object::PythonObject">>> read_mem, !kgen.pointer<typevalue<#kgen.instref<tdlib::python::python_object::PythonObject">>> read_mem, !kgen.pointer<typevalue<#kgen.instref<tdlib::builtin::error::Error">>> byref_error, !kgen.pointer<typevalue<#kgen.instref<tdlib::python::python_object::PythonObject">>> byref_result) throws -> i1, (!kgen.pointer<pointer<none>> read_mem, !kgen.pointer<pointer<none>> read_mem, !kgen.pointer<struct<(pointer<none>, index)>> byref_error, !kgen.pointer<pointer<none>> byref_result) throws -> i1], :struct<((!kgen.pointer<pointer<none>> read_mem, !kgen.pointer<pointer<none>> read_mem, !kgen.pointer<struct<(pointer<none>, index)>> byref_error, !kgen.pointer<pointer<none>> byref_result) throws -> i1) memoryOnly> { rgmax_masked::argmax_masked(::PythonObject,::PythonObject)" }>_closure_0 () from /home/.../Development/parstack-mojo/__mojocache__/argmax_masked.hash-c33146e5e9f20584.so
#3  0x00007ffff6950d6a in cfunction_call () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#4  0x00007ffff69ac68d in _PyEval_EvalFrameDefault () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#5  0x00007ffff6b396fa in PyEval_EvalCode () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#6  0x00007ffff6b9155e in run_eval_code_obj () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#7  0x00007ffff6b90e3e in run_mod.llvm () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#8  0x00007ffff6832547 in pyrun_file () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#9  0x00007ffff68318d5 in _PyRun_SimpleFileObject () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#10 0x00007ffff683158a in _PyRun_AnyFileObject () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#11 0x00007ffff683d07e in pymain_run_file_obj () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#12 0x00007ffff683cd0a in pymain_run_file () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#13 0x00007ffff6ba9d24 in Py_RunMain () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#14 0x00007ffff6baa4aa in pymain_main () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#15 0x00007ffff6baa56c in Py_BytesMain () from /home/.../.local/share/uv/python/cpython-3.13.1-linux-x86_64-gnu/bin/../lib/libpython3.13.so.1.0
#16 0x00007ffff6433ca8 in __libc_start_call_main (main=main@entry=0x555555555150 <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffd9c8) at ../sysdeps/nptl/libc_start_call_main.h:58
#17 0x00007ffff6433d65 in __libc_start_main_impl (main=0x555555555150 <main>, argc=2, argv=0x7fffffffd9c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffffffd9b8) at ../csu/libc-start.c:360
#18 0x0000555555555089 in _start ()

Awesome, the Mojo wrappers are here!!

Expected behavior

No crash.

Steps to reproduce

from python import PythonObject, Python
from python.bindings import PythonModuleBuilder
from python.python import CPython
import math
from memory import UnsafePointer
import time
from os import abort
from sys.info import simdwidthof
from algorithm.functional import parallelize


alias simd_width = simdwidthof[DType.float16]()


@export
fn PyInit_mojo_module() -> PythonObject:
    print("Initializing Python Mojo module...")
    try:
        var m = PythonModuleBuilder("mojo_module")
        m.def_function[parallel_wrapper](
            "parallel_wrapper", docstring="Parallelizing function"
        )
        return m.finalize()
    except e:
        return abort[PythonObject](
            String("error creating Python Mojo module:", e)
        )


fn parallel_wrapper(array: PythonObject) raises -> PythonObject:
    @parameter
    fn calc_max(i_sample: Int) -> None:
        ...

    parallelize[calc_max](100)

    return array
import os
import sys

import max._mojo.mojo_importer  # noqa: F401
import numpy as np

sys.path.insert(0, "")
os.environ["MOJO_PYTHON_LIBRARY"] = ""

import mojo_module

arr = np.random.uniform(size=(10000, 100)).astype(np.float32)


def test_mojo():
    mojo_module.parallel_wrapper(arr)


test_mojo()

System information

Mojo 25.4.0.dev2025060105 (c8bee233)

      Pixi version: 0.29.0
          Platform: linux-64
  Virtual packages: __unix=0=0
                  : __linux=6.12.22=0
                  : __glibc=2.41=0
                  : __archspec=1=icelake
         Cache dir: /home/marius/.cache/rattler/cache
      Auth storage: /home/marius/.rattler/credentials.json
  Config locations: No config files found

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmodular-repomojoIssues that are related to mojo

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0