8000 Error under macOS in PowerShell Core: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception · Issue #1583 · libgit2/libgit2sharp · GitHub
[go: up one dir, main page]

Skip to content

Error under macOS in PowerShell Core: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception #1583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
felixfbecker opened this issue Jun 18, 2018 · 30 comments

Comments

@felixfbecker
Copy link

Reproduction steps

  • Download LibGit2Sharp with the dotnet CLI
  • Import it into PowerShell Core with Import-Module
  • Run [LibGit2Sharp.Repository]::new($PWD) in a git repository

Expected behavior

No error

Actual behavior

Error

Exception calling ".ctor" with "1" argument(s): "The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception."
At line:1 char:1
+ [LibGit2Sharp.Repository]::new($PWD)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : TypeInitializationException

Version of LibGit2Sharp (release number or SHA1)

v0.25.2

Operating system(s) tested; .NET runtime tested

> $PSVersionTable

Name                           Value                                                                                                                                                                     
----                           -----                                                                                                                                                                     
PSVersion                      6.0.2                                                                                                                                                                     
PSEdition                      Core                                                                                                                                                                      
GitCommitId                    v6.0.2                                                                                                                                                                    
OS                             Darwin 17.6.0 Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64                                                             
Platform                       Unix                                                                                                                                                                      
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                                   
PSRemotingProtocolVersion      2.3                                                                                                                                                                       
SerializationVersion           1.1.0.1                                                                                                                                                                   
WSManStackVersion              3.0  
> dotnet --info    
.NET Command Line Tools (2.1.4)

Product Information:
 Version:            2.1.4
 Commit SHA-1 hash:  5e8add2190

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.13
 OS Platform: Darwin
 RID:         osx.10.12-x64
 Base Path:   /usr/local/share/dotnet/sdk/2.1.4/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.5
  Build    : 17373eb129b3b05aa18ece963f8795d65ef8ea54
@ethomson
Copy link
Member

Can you strace this (or the moral equivalent in macOS) to find out where it's trying to open the dylib?

@felixfbecker
Copy link
Author

Here's a trace generated with

sudo dtruss pwsh -c 'Import-Module ./LibGit2Sharp.dll; [LibGit2Sharp.Repository]::new($PWD)' *>> trace.txt

trace.txt

Hope this helps - I can see dylib a couple times in that output but don't really know how to interpret it.

@bording
Copy link
Member
bording commented Jun 20, 2018

@felixfbecker I don't think that command actually captured what @ethomson was hoping to see. There should be calls to dlopen in there trying to find libgit2-b0d9952.dylib or perhaps libgit2-15e1193.dylib (depending on what version of LibGit2Sharp you're using).

@felixfbecker
Copy link
Author

I found dtruss as an equivalent to strace: https://opensourcehacker.com/2011/12/02/osx-strace-equivalent-dtruss-seeing-inside-applications-what-they-do-and-why-they-hang/

Any suggestions what I need to do?

@ethomson
Copy link
Member

Can you clarify what exactly this means?

Download LibGit2Sharp with the dotnet CLI

Did you install the nuget package somewhere? Or did you just download the nuget and unzip it? I'm not familiar with PowerShell so please excuse my ignorance.

@felixfbecker
Copy link
Author
felixfbecker commented Jun 22, 2018

I did the following in an empty folder:

git init
dotnet new classlib
dotnet add package LibGit2Sharp
dotnet restore
dotnet publish -o out
Import-Module ./out/LibGitSharp.dll
[LibGit2Sharp.Repository]::new($PWD)

That creates a csproj, adds the dependency to it, and outputs all DLLs to the out folder. Then I ran the command from above to import the LibGit2Sharp.dll and try to initialize it on the current directory, which yields the error.

Btw, I am only seeing this in PowerShell. When using a simple C# console app and running it with dotnet run it works. I would like to depend on LibGit2Sharp in PowerShell scripts, but using it directly or through https://github.com/webmd-health-services/GitAutomation is broken because of this error.

Is there anything I can help to debug this?

@felixfbecker felixfbecker changed the title Error under macOS: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception Error under macOS in PowerShell Core: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception Jun 25, 2018
@felixfbecker
Copy link
Author

@ethomson is there any way I can help? Were you able to reproduce? I would guess it's reproducible in WSL too

@bording
Copy link
Member
bording commented Jul 25, 2018

@felixfbecker Why are you using Import-Module? LibGit2Sharp isn't a PowerShell module.

@felixfbecker
Copy link
Author

Because that, to my knowledge, is how you import a .NET DLL into PowerShell. I have done so successfully with other .NET packages. I also know that the import itself worked because I get autocompletion for the LibGit2Sharp namespace and the Repository class. It only errors once I call the function. It also seems to work fine on Windows PowerShell. Is there a different way I am not aware of I should be using to use LibGit2Sharp in PowerShell?

@ethomson
Copy link
Member

And it looks like it does load LibGit2Sharp, but in a way that the native library's load path isn't configured correctly. I don't have time to investigate that right now, but if you wanted to, the best way would probably be to build LibGit2Sharp yourself, and figure out where it's trying to load the natives from and where they actually exist on disk, determining how to detect when you're running in this environment and how to properly load the natives.

If you want to try a PowerShell module that wraps LibGit2Sharp, then there's https://github.com/ethomson/gitpowershell/tree/master/GitPowerShell. That may also help to understand what's not working. (Though I haven't tried this on macOS, so it may only continue to show why it's not working).

@felixfbecker
Copy link
Author

How could I find out where they actually exist on disk? Any hints where to look?

@ethomson
Copy link
Member
ethomson commented Jul 25, 2018

They should be near the LibGit2Sharp.dll in a subdirectory. Something like:

find . -iname libgit2\* -print

should help.

@bording
Copy link
Member
bording commented Jul 25, 2018

Given that PowerShell on macOS is PowerShell Core, which runs on .NET Core, I suspect the issue is going to be related to the fact that .NET Core relies on the deps.json file to resolve dependencies from runtimes folders, which works great when your app builds the deps.json file and has all of the relevant information in it.

However, it doesn't work quite as well in scenarios where things are being loaded dynamically, or in some sort of of plugin model.

When you do the dotnet publish -o out step, I suspect you'll see a runtimes folder in your publish output, which means you're doing a portable publish.

If you specify a runtime instead with the -r parameter, you'll get a specific version, and the appropriate native library will have been extracted from the runtimes folder. This might get your scenario working, though admittedly it would be locked to just the single runtime you specify.

If you're looking for something scriptable and cross-platform, I'd suggest looking at creating a .NET Core global tool instead. It lets you create a commandline app that will be cross-platform, and it will handle all the dependency stuff correctly as well. Then you can just call your app from a script.

@felixfbecker
Copy link
Author

Ah, that makes sense. Indeed I get a tree that looks like this:

.
├── PSlibgit.csproj
├── bin
│   └── Debug
│       └── netstandard2.0
│           ├── PSlibgit.deps.json
│           ├── PSlibgit.dll
│           └── PSlibgit.pdb
├── obj
│   ├── Debug
│   │   └── netstandard2.0
│   │       ├── PSlibgit.AssemblyInfo.cs
│   │       ├── PSlibgit.AssemblyInfoInputs.cache
│   │       ├── PSlibgit.csproj.CoreCompileInputs.cache
│   │       ├── PSlibgit.csproj.FileListAbsolute.txt
│   │       ├── PSlibgit.dll
│   │       └── PSlibgit.pdb
│   ├── PSlibgit.csproj.nuget.cache
│   ├── PSlibgit.csproj.nuget.g.props
│   ├── PSlibgit.csproj.nuget.g.targets
│   └── project.assets.json
└── out
    ├── LibGit2Sharp.dll
    ├── PSlibgit.deps.json
    ├── PSlibgit.dll
    ├── PSlibgit.pdb
    └── runtimes
        ├── alpine-x64
        │   └── native
        │       └── libgit2-b0d9952.so
        ├── debian.9-x64
        │   └── native
        │       └── libgit2-b0d9952.so
        ├── fedora-x64
        │   └── native
        │       └── libgit2-b0d9952.so
        ├── linux-x64
        │   └── native
        │       └── libgit2-b0d9952.so
        ├── osx
        │   └── native
        │       └── libgit2-b0d9952.dylib
        ├── rhel-x64
        │   └── native
        │       └── libgit2-b0d9952.so
        ├── win-x64
        │   └── native
        │       ├── git2-b0d9952.dll
        │       └── git2-b0d9952.pdb
        └── win-x86
            └── native
                ├── git2-b0d9952.dll
                └── git2-b0d9952.pdb

24 directories, 28 files

Is there a way LibGit2Sharp could load the appropiate file for the right platform? I want to publish the module to the PowerShellGallery, so shipping just one platform wouldn't work.

Maybe I'm missing something, but .NET Core global tool seems to only be able to produce executable CLI tools that can only output bytes on STDIO and take string command line flags. What I want to do is utilize LibGit2Sharp objects in PowerShell (making use of the object pipeline).

@bording
Copy link
Member
bording commented Jul 25, 2018

Is there a way LibGit2Sharp could load the appropiate file for the right platform?

LibGit2Sharp relies on the framework's capabilities, so it's limited to working in the way that .NET Core works.

Maybe I'm missing something, but .NET Core global tool seems to only be able to produce executable CLI tools that can only output bytes on STDIO and take string command line flags. What I want to do is utilize LibGit2Sharp objects in PowerShell (making use of the object pipeline).

The intention of my suggestion was to use the global tool to do the actual object manipulation in the tool, in C#. Then you could just script the calling of the commandline tool.

@felixfbecker
Copy link
Author

Do you see any way for me to depend on LibGit2Sharp in a PowerShell module? Maybe install the package on first run, or bundle all runtimes and move files around to pick the right one on first run?

@flibustier7seas
Copy link
flibustier7seas commented Aug 10, 2018

@felixfbecker

Do you see any way for me to depend on LibGit2Sharp in a PowerShell module? Maybe install the package on first run, or bundle all runtimes and move files around to pick the right one on first run?

I was getting the same error: "The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception" under Windows. I saw a solution in the PSGit repository.

# Add paths to native libraries

$nativeBinariesPackage = Get-Package LibGit2Sharp.NativeBinaries -RequiredVersion 1.0.217
$nativeBinariesPath = Split-Path $nativeBinariesPackage.Source

# Copy-paste from https://github.com/PoshCode/PSGit/blob/dev/src/PSGit.psm1#L34
${;} = [System.IO.Path]::PathSeparator
switch -Wildcard (Get-ChildItem -Path "$nativeBinariesPath" -Recurse -Filter '*git2-6311e88.*') {
    "*.so"   { $env:LD_LIBRARY_PATH = "" + $_.Directory + ${;} + $Env:LD_LIBRARY_PATH }
    "*.dll"  { $env:PATH = "" + $_.Directory + ${;} + $Env:PATH }
    "*.dyld" { $env:DYLD_LIBRARY_PATH = "" + $_.Directory + ${;} + $Env:DYLD_LIBRARY_PATH }
}

# Load LibGit2Sharp assembly

$package = Get-Package LibGit2Sharp -RequiredVersion 0.25.2
$assemblyPath = (Get-ChildItem -Path (Split-Path $package.Source)  -Recurse  -Filter "*LibGit2Sharp.dll").FullName

Import-Module $assemblyPath

$repositoryPath = [LibGit2Sharp.Repository]::Init("F:\NewRepository")

@felixfbecker
Copy link
Author
felixfbecker commented Aug 10, 2018

I think I managed to workaround it by figuring out the runtime identifier (RID) and copying the native binary next to the LibGit2Sharp DLL (assuming the above folder structure from a dotnet publish). It then finds it correctly.

using namespace System.Runtime.InteropServices

$runtime = if ($IsMacOS) {
    'osx'
}
else {
    $os = if ($IsWindows) {
        'win'
    }
    elseif ($IsLinux) {
        'linux'
        # TODO detect debian, fedora, alpine, rhel
    }
    $arch = [RuntimeInformation]::OSArchitecture.ToString().ToLower()
    "$os-$arch"
}

Copy-Item $PSScriptRoot/Assemblies/runtimes/$runtime/native/*.* ./Assemblies

Import-Module $PSScriptRoot/Assemblies/LibGit2Sharp.dll

What feels wrong about this is that the docs explicitely state that RIDs are opaque identifiers that should NOT be constructed programmatically, but that is exactly what I am doing here.

There are some considerations about RIDs that you have to keep in mind when working with them:

  • RIDs are opaque strings and should be treated as black boxes.
  • Don't build RIDs programmatically.
  • Use RIDs that are already defined for the platform.
  • The RIDs need to be specific, so don't assume anything from the actual RID value.

@felixfbecker
Copy link
Author

I went with constructing the RID and copying the DLL, which seems to work.

https://github.com/felixfbecker/PowerGit/blob/1967da731dbe404b472efc57a04b39acd5b94e96/PowerGit/PowerGit.psm1#L15-L33

Thanks for the help!

@Spiralis
Copy link

Is this commit fixing this, perhaps?fc7b5b3

I am having similar issues, and it seems like I would have to release packages directed at each runtime for this to work.

But, it seems that the change in the commit is not effective in any release yet. Are there plans on a new release anytime soon?

@ethomson
Copy link
Member

@Spiralis - to clarify: you're also trying to load LibGit2Sharp from PowerShell? Or you're doing something different but getting the same exception?

Have you tried the latest prerelease nuget package? It should include the changes you're referring to.

@felixfbecker
Copy link
Author

@Spiralis Hmm, that code looks like it should fix the issue, but it doesn't, because I am using the preview-27 as mentioned here #1563 (comment)

This makes me think that this bug should be reopened and there is a problem in libgit2, and my solution is only a workaround?

@Spiralis
Copy link

@ethomson I am not loading libgit2sharp from Powershell. I am doing a library for https://github.com/cake-build/cake, that when used complains about the missing native libgit libraries.

I can copy them in manually (or automatically in the build-script), but that feels wrong. I am creating a NetStandard 2.0 library, and I shouldn't choose which native libraries the consumer of my library is using.

It seems sensible that all runtime target native libs are published (in the runtimes and native folders). Then the consumer of the dll (as in the running process) will find it since libgit2sharp is smart enough to find which native runtime is needed.

@bording
Copy link
Member
bording commented Aug 24, 2018

@Spiralis Is Cake running on the .NET Framework or .NET Core?

The way the native libraries are discovered is different for each of them. I'm not sure the commit you linked to is really going to help, but I need to know more details about your scenario.

At this point, I'd recommend opening a new issue so we can discuss it since you aren't using PowerShell.

@Spiralis
Copy link

As for all I can find, it is dotnet core. But, how it is running in the end, I am not sure about. By some reason it is not able to find it in the runtimes folder. I was beginning to write a bug-report and tried to make a minimal example. In my tests without cake though, running via dotnet run worked fine. However, when running using as an addin in Cake, it does not find them. For the cake addin I have to copy them into the folder where the other dlls from the project are generated when built.

So, at this stage I am really not sure what is working and not 😕 ... I will need some more time on this I guess. Will post an issue if I am able to pinpoint more accurately what the problem is.

@bording
Copy link
Member
bording commented Aug 25, 2018

@Spiralis That make sense, and is somewhat as expected. If you're referencing LibGit2Sharp directly and running the app via dotnet run, then the appropriate information is in the deps.json file, so .NET Core will automatically resolve everything properly.

However, if your assembly is being loaded by another program that doesn't have a direct reference to LibGit2Sharp, then that application's deps.json doesn't have anything related to LibGit2Sharp in it, so .NET Core will not be able to resolve the correct native library from the runtimes folder.

This is a general limitation of .NET Core's assembly loading design. If you want to use LibGit2Sharp in a plugin with .NET Core, then right now you have to provide your own assembly resolving logic via an AssemblyLoadContext implementation. And, since we have multiple runtimes that vary by linux distro, the logic to get it working everywhere can be somewhat complicated.

@Spiralis
Copy link

I see. Thanks for the response. Are there any samples for the AssemblyLoadContext that does this that you can suggest perhaps?

@Spiralis
Copy link

BTW: What is the intention of the commit I pointed to? https://github.com/cake-build/cake

@Spiralis
Copy link

Update: I found this article that regards to loading an assembly, which seems promising. Just missing the logics for the "which RID/dll to load" strategy. I will continue my search - or potentially reinvent the wheel :)

@bording
Copy link
Member
bording commented Aug 25, 2018

BTW: What is the intention of the commit I pointed to?

That code was an attempt to address some of the complexities involved in resolving the correct native libraries, but it turns out it only really works on Windows, so it's not as helpful as it was thought it would be when the PR that added it was merged.

I see. Thanks for the response. Are there any samples for the AssemblyLoadContext that does this that you can suggest perhaps?

You might take a look at https://github.com/dotnet/sourcelink/tree/master/src/Microsoft.Build.Tasks.Git for ideas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants
0