8000 SetNoSiteFlag is ignored on Windows unless another PythonEngine call precedes it · Issue #1517 · pythonnet/pythonnet · GitHub
[go: up one dir, main page]
Skip to content

SetNoSiteFlag is ignored on Windows unless another PythonEngine call precedes it #1517

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
bpdavis86 opened this issue Aug 12, 2021 · 1 comment · Fixed by #1659
Closed

SetNoSiteFlag is ignored on Windows unless another PythonEngine call precedes it #1517

bpdavis86 opened this issue Aug 12, 2021 · 1 comment · Fixed by #1659
Labels
bug
Milestone
3.0.0

Comments

@bpdavis86
Copy link
bpdavis86 commented Aug 12, 2021 •
edited
Loading

Environment

  • Pythonnet version: Self-built from commit ec65efe 8/11/2021
  • Python version: 3.8.10
  • Operating System: Windows 10 Pro 20H2 19042.1110
  • .NET Runtime: .NET Core 3.1

Details

  • Describe what you were trying to get done.

    I am trying to disable automatic site run in order to setup a custom virtualenv path.

  • What commands did you run to trigger this issue? If you can provide a
    Minimal, Complete, and Verifiable example
    this will help us understand the issue.

using System;
using System.Collections.Generic;
using Python.Runtime;


namespace TestPythonnet
{
    class Program
    {
        static void Main(string[] args)
        {

            // my base environment
            // also added this to top of PATH in project settings
            var pathToBaseEnv = @"C:\Users\myuser\AppData\Local\Programs\Python\Python38";
            Environment.SetEnvironmentVariable("PYTHONNET_PYVER", "3.8", EnvironmentVariableTarget.Process);

            // this call has no effect
            PythonEngine.SetNoSiteFlag();
            // if you swap this call with previous line, it will work
            PythonEngine.PythonHome = pathToBaseEnv;

            PythonEngine.Initialize();
            using (Py.GIL())
            {
                Console.WriteLine($"sys.executable: {sys.executable}");
                Console.WriteLine($"sys.prefix: {sys.prefix}");
                Console.WriteLine($"sys.base_prefix: {sys.base_prefix}");
                Console.WriteLine($"sys.exec_prefix: {sys.exec_prefix}");
                Console.WriteLine($"sys.base_exec_prefix: {sys.base_exec_prefix}");
                Console.WriteLine("sys.path:");
                foreach (var p in sys.path)
                {
                    Console.WriteLine(p);
                }
                Console.WriteLine();
            }

            PythonEngine.Shutdown();
        }
    }
}

Result:

sys.executable: C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1\TestPythonnet.exe
sys.prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.path:
C:\Users\myuser\OneDrive\Documents\Python Scripts
C:\Users\myuser\AppData\Local\Programs\Python\Python38\python38.zip
C:\Users\myuser\AppData\Local\Programs\Python\Python38\DLLs
C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib
C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1
C:\Users\myuser\AppData\Local\Programs\Python\Python38
C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib\site-packages
C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.18\

The site packages are still added.

If you swap order of PythonEngine.SetNoSiteFlag(); and PythonEngine.PythonHome = pathToBaseEnv;, it works

sys.executable: C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1\TestPythonnet.exe
sys.prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.path:
C:\Users\myuser\OneDrive\Documents\Python Scripts
C:\Users\myuser\AppData\Local\Programs\Python\Python38\python38.zip
C:\Users\myuser\AppData\Local\Programs\Python\Python38\DLLs
C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib
C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1
C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.18\

No site folders added to the path.

In fact, one can workaround the issue by adding any access to the PythonEngine API which uses the DLL before setting the flag, e.g.

var version = PythonEngine.Version;
PythonEngine.SetNoSiteFlag();

The root cause appears to be that the reference count on the DLL is sent to zero after setting Py_NoSiteFlag, so the DLL is unloaded and reloaded by the Delegate class with the original value of Py_NoSiteFlag=0.

Snip from Python.Runtime.Runtime (runtime.cs) with my added comments.

internal static void SetNoSiteFlag()
        {
            var loader = LibraryLoader.Instance;
            IntPtr dllLocal = IntPtr.Zero;
            if (_PythonDll != "__Internal")
            {
                dllLocal = loader.Load(_PythonDll);
                if (dllLocal == IntPtr.Zero)
                {
                    throw new Exception($"Cannot load {_PythonDll}");
                }
            }
            try
            {
                Py_NoSiteFlag = loader.GetFunction(dllLocal, "Py_NoSiteFlag");
                Marshal.WriteInt32(Py_NoSiteFlag, 1); // This works, I can Marshal.ReadInt32 with Immediate window in Debug and get 1
            }
            finally
            {
                if (dllLocal != IntPtr.Zero)
                {
                    loader.Free(dllLocal); // if nobody else loaded _PythonDll, it will potentially get unloaded here :-(
                }
            }
        }
  • If there was a crash, please include the traceback here.

N/A
@bpdavis86
Copy link
Author
bpdavis86 commented Aug 12, 2021 •
edited
Loading

My suggested fix would be to defer the actual setting of Py_NoSiteFlag in the DLL and perform it in Runtime.Initialize after we can be sure the DLL has been loaded and attached to the delegates.

Additionally, it would be nice to implement the PEP 587 configuration API on available Python versions (3.8+) which would allow complete control of the embedded python environment, including effective command line options.

@bpdavis86 bpdavis86 mentioned this issue Aug 12, 2021
Open
@lostmsu lostmsu added this to the 3.0.0 milestone Sep 23, 2021
@lostmsu lostmsu added the bug label Oct 1, 2021
lostmsu added a commit to losttech/pythonnet that referenced this issue Jan 5, 2022
@lostmsu
use the same facility to access Py_NoSiteFlag as the one used to load…
1129a73 
… Python C API functions

fixes pythonnet#1517
This was referenced Jan 5, 2022
Merged
Open
filmor pushed a commit that referenced this issue Jan 6, 2022
@lostmsu
use the same facility to access Py_NoSiteFlag as the one used to load…
7e5cc29 
… Python C API functions (#1659)

fixes #1517
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
3.0.0
Development

Successfully merging a pull request may close this issue.

Use Delegates to access Py_NoSiteFlag losttech/pythonnet
2 participants
@lostmsu @bpdavis86
0