8000 Officially support Python 3.9 · Issue #1389 · pythonnet/pythonnet · GitHub
[go: up one dir, main page]
Skip to content

Officially support Python 3.9 #1389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
saroad2 opened this issue Feb 19, 2021 · 20 comments
Closed

Officially support Python 3.9 #1389

saroad2 opened this issue Feb 19, 2021 · 20 comments
Milestone
3.0.0

Comments

@saroad2
Copy link
saroad2 commented Feb 19, 2021

Environment

  • Pythonnet version: latest
  • Python version: python 3.9
  • Operating System: Windows
  • .NET Runtime:

Details

  • Describe what you were trying to get done.

My project EddingtonGUI is dependant on Pythonnet. I wanted for quite some time to update it to use Python 3.9 as default, but I noticed that you don't support it yet.

When are you planning on officially supporting Python 3.9? What prevents you from currently supporting it?
I'm asking so I could update my release scheduling accordingly.

Thank you!
@sschukat
Copy link

The latest release 2.5 already supports Python 3.9 see [2.5.2] (https://github.com/pythonnet/pythonnet/releases/tag/v2.5.2). Version 3.0.0 is still in development.

@saroad2
Copy link
Author
saroad2 commented Feb 19, 2021

Ye, but when I took a look at PyPi, Python3.9 is not written as supported.
I guess it's just a release error :)

@filmor
Copy link
Member
filmor commented Feb 19, 2021

After backporting the Python 3.9 support from master to the 2.5 branch, I noticed crashes in CI (I don't remember the details right now). That's why, even though the support is nominally there, I didn't publish the respective wheels and didn't update the trove specifier. You are welcome to give debugging this a go, development happens on master, though, we haven't done any release planning for 3.0 yet.

@lostmsu
Copy link
Member
lostmsu commented Feb 19, 2021

@filmor considering our current master also has crashes on 3.9, random crashes in v2.5 on Python 3.9 could be related.

We should consider 3.9 support in v2.5 as-is, e.g. support installation, but not necessarily fix all crashes.

@Jeff17Robbins
Copy link
Contributor

@lostmsu, is there a recipe for reproducing the current master 3.9 crashes?

@lostmsu
Copy link
Member
lostmsu commented Feb 22, 2021

They happen in CI occasionally in both Ubuntu and MacOS with the normal shutdown mode.
Here's a sample failure: https://github.com/pythonnet/pythonnet/runs/1944796763
The CI script is this: https://github.com/pythonnet/pythonnet/blob/master/.github/workflows/main.yml

@ppena-LiveData
Copy link

Weird, dotnet test --runtime any-${{ matrix.platform }} src/embed_tests/ crashed with a "Fatal Python error: PyGILState_Release: auto-releasing thread-state, but no thread-state for this thread" error. That error (plus the fact that it only happens occasionally) sounds like a race condition, but it looks like it was running PythonEngine.RunSimpleString("import 1234"), which is only a single import statement, so I wonder what could be causing a race condition.

@lostmsu
Copy link
Member
lostmsu commented Feb 22, 2021 •
edited
Loading

Is there a stack trace? Also, which platform?

@ppena-LiveData
Copy link

I haven't tried to reproduce it yet. I was just looking at the CI run you linked to.

@Jeff17Robbins
Copy link
Contributor

The CI crash shows this stack trace:

	0x7fff677da808 - /usr/lib/system/libsystem_c.dylib : abort
	0x109521abd - /Users/runner/hostedtoolcache/Python/3.9.1/x64/lib/libpython3.9.dylib : fatal_error_exit
	0x109521a7a - /Users/runner/hostedtoolcache/Python/3.9.1/x64/lib/libpython3.9.dylib : fatal_error
	0x109526ca4 - /Users/runner/hostedtoolcache/Python/3.9.1/x64/lib/libpython3.9.dylib : _Py_FatalErrorFunc
	0x1095295e5 - /Users/runner/hostedtoolcache/Python/3.9.1/x64/lib/libpython3.9.dylib : PyGILState_Release
	0x10ac5164b - Unknown
	0x10abb1c0c - Unknown
	0x1048bc9a9 - /Library/Frameworks/Mono.framework/Versions/6.12.0/bin/mono : mono_gc_run_finalize
	0x1048dbadc - /Library/Frameworks/Mono.framework/Versions/6.12.0/bin/mono : sgen_gc_invoke_finalizers
	0x1048be6b3 - /Library/Frameworks/Mono.framework/Versions/6.12.0/bin/mono : finalizer_thread
	0x10487cdad - /Library/Frameworks/Mono.framework/Versions/6.12.0/bin/mono : start_wrapper
	0x7fff67914109 - /usr/lib/system/libsystem_pthread.dylib : _pthread_start
	0x7fff6790fb8b - /usr/lib/system/libsystem_pthread.dylib : thread_start

I noticed in pyobject.cs, for example, the the gc finalizer is meant to be suppressed.

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

Does Py.GILState suppress finalization? If so, what code is calling `PyGILState_Release in a magical gc finalizer thread?!

@lostmsu
Copy link
Member
lostmsu commented Feb 24, 2021

That's a very good observation. GILState must not have a finalizer doing PyGILState_Release (via ReleaseLock), because you must call PyGILState_Release in the same thread as PyGILState_Ensure. I believe we should throw a fatal exception instead.

https://docs.python.org/3/c-api/init.html#c.PyGILState_Release

lostmsu added a commit to losttech/pythonnet that referenced this issue Feb 24, 2021
@lostmsu
allow GIL state debugging; require GILState to be properly disposed
17e58ac 
pythonnet#1389 (comment)
@lostmsu lostmsu mentioned this issue Feb 24, 2021
Merged
@lostmsu
Copy link
Member
lostmsu commented Feb 24, 2021 •
edited
Loading

@Jeff17Robbins unfortunately, this is not the culprit: https://github.com/pythonnet/pythonnet/pull/1397/checks?check_run_id=1967913124#step:13:144

@Jeff17Robbins
Copy link
Contributor

@lostmsu I see a completely different stack trace of a crash on Ubuntu and Macos. What's the difference between the "Soft" tests and the "Normal" tests? And...how hard would it be to upgrade to Python3.9.2 to see if it is still a problem with that minor release of CPython?

@lostmsu
Copy link
Member
lostmsu commented Feb 25, 2021

@Jeff17Robbins it's ShutdownMode here.

The issue reproduces with different Python minor versions.

@filmor filmor mentioned this issue Feb 25, 2021
Merged
4 tasks
@Jeff17Robbins
8000 Copy link
Contributor

@lostmsu is there a writeup of ShutdownMode? What does "Soft" soften? Since the crashes seem only to occur due to "Normal" ShutDownMode, it would help to know what the differences are between "Soft" and "Normal"?

@lostmsu
Copy link
Member
lostmsu commented Feb 25, 2021 •
edited
Loading

@Jeff17Robbins soft shutdown mode tries to preserve live CLR objects referenced by Python in order to be able to restore them upon the reinitialization of PythonEngine (the main goal is to support AppDomain reloading).

The code in Runtime.Initialize and Runtime.Shutdown are pretty much the only difference there.

@dlech dlech mentioned this issue Mar 2, 2021
Closed
lostmsu added a commit that referenced this issue Mar 10, 2021
@lostmsu
allow GIL state debugging; require GILState to be properly disposed
1e5338f 
#1389 (comment)
@wdscxsj
Copy link
wdscxsj commented Mar 29, 2021

On my Windows 10 machine, v2.5.2 built by Christoph Gohlke always fails to import in Python 3.9:

Python 3.9.2 (tags/v3.9.2:1a79785, Feb 19 2021, 13:44:55) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import clr
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "C:\Tools\Python3\lib\site.py", line 73, in <module>
    import os
  File "C:\Tools\Python3\lib\os.py", line 29, in <module>
    from _collections_abc import _check_methods
  File "C:\Tools\Python3\lib\_collections_abc.py", line 416, in <module>
    class _CallableGenericAlias(GenericAlias):
TypeError: type 'types.GenericAlias' is not an acceptable base type

@m-rossi m-rossi mentioned this issue May 11, 2021
Closed
@canol
Copy link
canol commented Jun 14, 2021 •
edited
Loading

Hello, one of our internal frameworks at the company I work at depends on pythonnet. But pythonnet not having wheels for Python 3.9 prevents our users from using Python 3.9. We are on Windows 10 platform. Is there anything I can do to help for the Python 3.9 release of pythonnet? I am not familiar with pythonnet or CPython internals but I can maybe try compiling something using Visual Studio and report logs or run a test suit?

@Felk Felk mentioned this issue Jun 26, 2021
Open
@filmor filmor mentioned this issue Jul 2, 2021
Closed
@jaimergp jaimergp mentioned this issue Aug 31, 2021
Merged
11 tasks
@lostmsu lostmsu added this to the 3.0.0 milestone Sep 23, 2021
@matthew1davis matthew1davis mentioned this issue Nov 5, 2021
Open
@lostmsu
Copy link
Member
lostmsu commented Jan 4, 2022

@canol there's now a preview pip package for pythonnet 3.0, that supports Python 3.9 and 3.10

@lostmsu lostmsu closed this as completed Jan 5, 2022
@burque505
Copy link

@lostmsu, the preview pip package for pythonnet 3.0 (I have 3.0.0a2) won't run the demo scripts (helloform.py, splitter.py, or wordpad.py). There are failures with generics and not recognizing 1 and 0 as Boolean. Converting 1 and 10 to True and False helps with helloform.py and splitter.py, but I haven't been able to fix wordpad.py because the error messages are too obscure for me:


Unhandled Exception: Python.Runtime.PythonException: 'int' value cannot be converted to System.Boolean
   at Python.Runtime.PythonException.ThrowLastAsClrException()
   at Python.Runtime.Dispatcher.TrueDispatch(Object[] args)
   at Python.Runtime.Dispatcher.Dispatch(Object[] args)
   at __System_Threading_ThreadStartDispatcher.Invoke()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

Obviously this is a Boolean-related error message but I have no clue where to start looking.
I would humbly request that either a) Boolean and generic related errors be addressed or b) support be added for 2.5.2. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.0.0
Development

No branches or pull requests
9 participants
@filmor @canol @lostmsu @wdscxsj @Jeff17Robbins @saroad2 @sschukat @burque505 @ppena-LiveData
0