8000 NuGet package creation · Issue #165 · pythonnet/pythonnet · GitHub
[go: up one dir, main page]

Skip to content

NuGet package creation #165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AlexCatarino opened this issue Feb 24, 2016 · 30 comments
Closed

NuGet package creation #165

AlexCatarino opened this issue Feb 24, 2016 · 30 comments

Comments

@AlexCatarino
Copy link
Contributor

In order to use Python for .NET in a .NET app, we need to put Python.Runtime.dll in the references of the Visual Studio project. Unfortunately, this task has to be done manually, because we get this dll via pip (or downloading and compiling from this GitHub repository).

It would be very helpful to create and publish a NuGet package for pythonnet. If a NuGet package is generated at the end of a successful build (as suggested in the previous link), it would be quite easy to maintain a updated NuGet package for pythonnet.

@den-run-ai
Copy link
Contributor

How about python version?

On Wednesday, February 24, 2016, Alexandre Catarino <
notifications@github.com> wrote:

In order to use Python for .NET in a .NET app, we need to put
Python.Runtime.dll in the references of the Visual Studio project.
Unfortunately, this task has to be done manually, because we get this dll
via pip (or downloading and compiling from this GitHub repository).

It would be very helpful to create and publish a NuGet package
https://docs.nuget.org/create/creating-and-publishing-a-package for
pythonnet. If a NuGet package is generated at the end of a successful build
(as suggested in the previous link), it would be quite easy to maintain a
updated NuGet package for pythonnet.


Reply to this email directly or view it on GitHub
#165.

@AlexCatarino
Copy link
8000
Contributor Author

A NuGet package for every supported version like pip/wheels.

@den-run-ai
Copy link
Contributor

Where would the flag be set which version of pythonnet to get from Nuget?
Automagically detect system python or build flag like in pythonnet?

Note that there is limited chocolatey package for pythonnet.

How about clr.pyd?

On Wednesday, February 24, 2016, Alexandre Catarino <
notifications@github.com> wrote:

A NuGet package for every supported version like pip/wheels.


Reply to this email directly or view it on GitHub
#165 (comment)
.

@den-run-ai
Copy link
Contributor

It is possible to push to nuget directly from appveyor. @tonyroberts are you willing to set this up in virtual machine settings?

@tonyroberts
Copy link
Contributor

@denfromufa the clr.pyd file is not needed for embedding python, only for calling .net from python, so no need for it to be in nuget. I don't know enough about how packages should be uploaded to nuget wrt different variants of the same version (eg pythonnet 2.1 would have different packages for each different python version). If you want to investigate further and make suggestions about how the appveyor build should be set up I'm happy to make the changes.

@dmitriyse
Copy link
Contributor

Please check my alpha binary release. It can be packed as nuget in future. (https://github.com/dmitriyse/pythonnet/releases)

@den-run-ai
Copy link
Contributor

@dmitriyse IMO your bundle could be useful to @bc3tech for his chocolatey package, which is currently only limited to 32 bit python 2.7.

@den-run-ai
Copy link
Contributor

@dmitriyse
Copy link
Contributor

I released better implementation of Python.Runtime.dll loader. Feel free to check and use.
https://github.com/dmitriyse/pythonnet/releases/tag/v2.2.0-bin-beta

@dmitriyse
Copy link
Contributor

Hi everybody. I started work that allows to separate Python.Runtime.dll from python specific interop methods. This efforts will finally allows to load appropriate Python.Runtime.Interop-PyXX-UCS-X.dll by simply Assembly.LoadFromFile without any AssemblyResolve handling magic.
Please check this approach and advice me in any points (until is not too late).
dmitriyse@c55f655

@dmitriyse
Copy link
Contributor

Separation completed! You can inspect how it's finally looks. dmitriyse@1ea3773

@vmuriart
Copy link
Contributor

@dmitriyse did you mean only those specific commits are needed or all the commits upto those commits?

@dmitriyse
Copy link
Contributor
dmitriyse commented Jan 14, 2017

This branch contains two significant changes: 1) Port to DotNetCore (with ability to build Mono compatible version through dotnet core compiler) 2) and then separation of logic and Python/Platform specific interops methods.
Port is not finished, it should be debugged under linux some issues are not fixed. Second part (separation) can be merged to a master branch but with little rework.

@dmitriyse
Copy link
Contributor
dmitriyse commented Jan 14, 2017

I plan to debug dotnet core port for linux in the next month and some new build scripts that will helps to build nuget package. I can even built it with wide variety of Python/Platform support but I have no environment and time to test it. If somebody spend time to adopt current (setup.py) build system for my new features, we will finally get cool nuget package that will works in any environment and in mono and dotnet core.

@dmitriyse
Copy link
Contributor
dmitriyse commented Jan 14, 2017

I will merge soon environment detector from my previous binary distribution to coreclr branch in my fork.

@vmuriart
Copy link
Contributor

Looks like Dotnetcore is migrating back to csproj files from xproj. You have some code changes embedded with folder structure changes, making it a bit harder to see what all changed.

@den-run-ai den-run-ai added this to the 2.3.0 milestone Feb 2, 2017
@den-run-ai
Copy link
Contributor

@vmuriart in nuget it is possible to provide -version flag to install specific version of the package. Here are the CPython 3.5+ versions provided by @zooba:

https://www.nuget.org/packages/python/3.5.2

So we can create the following pythonnet versions to match PyPI:

2.2.2-cp27-cp27m-win32.whl
2.2.2-cp27-cp27m-win_amd64
2.2.2-cp34-cp34m-win32
2.2.2-cp34-cp34m-win_amd64
2.2.2-cp35-cp35m-win32
2.2.2-cp35-cp35m-win_amd64
2.2.2-cp36-cp36m-win32
2.2.2-cp36-cp36m-win_amd64

@vmuriart vmuriart modified the milestones: Backlog, 2.3.0 Mar 10, 2017
@den-run-ai
Cop 8000 y link
Contributor

@vmuriart I created a nuget package for Python.Runtime.csproj using the command-line tool:

nuget pack Python.Runtime.csproj

Here is the the output:

pythonnet.2.3.0.nupkg.zip

I uploaded it to https://www.nuget.org/packages/pythonnet

Next step is figuring out multiple versions or tags like I suggested above.

@den-run-ai
Copy link
Contributor

ok, looks like nuget does not support building different versions of nuget packages for all build configurations that pythonnet has:

http://stackoverflow.com/questions/35895601/how-to-specify-different-nuget-package-versions-for-different-build-configuratio

So we have to keep 522=20 .nuspec files for all possible combinations of build configurations.

@vmuriart
Copy link
Contributor

Nuget also enforces a certain version scheme so we couldn't (rather shouldn't) use the aforementioned versioning scheme. One of its pitfalls (look at the nuget python package as an example) is that it will always say there is an update version available if you are using python27.

Another thing to consider is how the libraries that would use this nuget package expect it to work. Do they want to only work with a specific python version, or do they want them to work on whichever the end-user has installed.

I like how nunit packages their nuget solution. Each version is compiled against multiple Net frameworks and all of them are included in the nuget package. Which version is to be used is specified on the project file.

@den-run-ai
Copy link
Contributor

@vmuriart nunit uses the standard mechanism in nuget for selecting the correct .NET Framework:

https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks

This does apply to pythonnet and cannot be re-used. I'm open to alternative solutions, but if nothing comes, then I'll go ahead with the approach that MS did for Python installers in nuget - using versions. I'm fine with update notifications, since the user of nuget package is responsible for picking the correct version of pythonnet. It is nearly impossible to analyze the system for available python versions and pick the correct one from nuget, especially when multiple python version are installed.

We can experiment with a unified runtime approach in a package "pythonnetbin" later.

@dmitriyse
Copy link
Contributor

It is nearly impossible to analyze the system for available python versions and pick the correct one from nuget, especially when multiple python version are installed.

Actually we can rely on current Environment. Even if multiple pythons installed, only one is active from a command line.
I already have some implementation.

        private static string DetectRequiredPythonRuntimDll(out string os, out List<string> librariesPathElements)
        {
            string stdOut;
            string pythonInfoFile = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "python-info.tmp");

            if (0
                == Stdlib.system(
                    $@"python -c ""import sys; print(sys.version_info.major); print(sys.version_info.minor); import array; print(array.array('u').itemsize); import platform; print(platform.architecture()[0]); print(platform.architecture()[1]); import sysconfig; print(sysconfig.get_config_vars('WITH_PYMALLOC')); print(sysconfig.get_config_var('LIBPL'));print(sysconfig.get_config_var('LIBDIR'))"" > {pythonInfoFile}"))
            {
                if (File.Exists(pythonInfoFile))
                {
                    stdOut = File.ReadAllText(pythonInfoFile);

                    File.Delete(pythonInfoFile);
                }
                else
                {
                    throw new InvalidOperationException("Failed to execute python");
                }
            }
            else
            {
                throw new InvalidOperationException("Failed to execute python");
            }
            
            var result = NormalizeOutput(stdOut).Split('\n');
            int majorVersion;
            int minorVersion;
            int charSize;
            bool pyMalloc = false;

            if (result.Length >= 8)
            {
                if (int.TryParse(result[0], out majorVersion) && int.TryParse(result[1], out minorVersion)
                    && int.TryParse(result[2], out charSize))
                {
                }
                else
                {
                    throw new InvalidOperationException("Failed to extract information about python.");
                }

                if (result[3].Trim().Length != 5)
                {
                    throw new InvalidOperationException("Failed to extract information about python.");
                }
            }
            else
            {
                throw new InvalidOperationException("Failed to extract information about python.");
            }

            os = "elf";
            result[4] = result[4].Trim();

            if (result[4] == "WindowsPE")
            {
                os = "win";
            }
            else if (result[4] == "ELF")
            {
                os = "elf";
            }
            else if (result[4] == string.Empty)
            {
                os = "osx";
            }

            if (result[5] == "[1]")
            {
                pyMalloc = true;
            }

            string options = string.Empty;

            if (pyMalloc)
            {
                options += "m";
            }

            librariesPathElements = new List<string>();
            for (int i = 6; i < 8; i++)
            {
                var libPathElement = result[i].Trim();
                if (libPathElement != "None" && libPathElement != string.Empty)
                {
                    librariesPathElements.Add(libPathElement);
                }
            }

            string dllName = "Python.Runtime";
            dllName += "-" + os;
            dllName += "-" + result[3].Substring(0, 2);
            dllName += "-ucs" + charSize;

            dllName += "-" + result[0] + result[1] + options;

            dllName += ".dll";
            return dllName;
        }

@dmitriyse
Copy link
Contributor

Full solution can be inspected here https://github.com/dmitriyse/pythonnet/tree/v2.2.0-bin-RC2

@dmitriyse
Copy link
Contributor
dmitriyse commented Mar 13, 2017

This is dead-end branch. New code should not rely on AssemblyResolve+= ... solutions.
If we need dynamic loading, than we needs to choose interop code splitting approach .
But environment python detection code can be imported.

@vmuriart
Copy link
Contributor

I prefer the interop splitting method, as it would also make it easy to allow pypy to be supported. For now, we could release multiple packages each named for the python version they support and then the version number is pythonnet's actual version number.

@den-run-ai
Copy link
Contributor

@vmuriart now I agree with you, e.g. boost python uses similar convention:

https://www.nuget.org/packages?q=boost_python

@den-run-ai
Copy link
Contributor

@den-run-ai
Copy link
Contributor

@mphielipp i deleted your 4 messages here and moved them to a separate issue:

#472

@den-run-ai
Copy link
Contributor

Apparently there is outstanding issue in nuget with not handling architectures, hence this has to be handled manually in the next release:

#472 (comment)

@HarelM
Copy link
HarelM commented Jun 3, 2019

Any chance to integrate #875 PR? It would be extremely useful to be able to download a NuGet package from appveyor and use it on our local NuGet repository, whether you choose to place it in the global NuGet or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants
0