System.ComponentModel.Composition.Hosting.ExportProvider.GetExports fails with LoaderExceptions

When calling System.ComponentModel.Composition.Hosting.ExportProvider.GetExports we are getting an exception on our build machine and in tests only. No one can reproduce it locally.

Exception:

System.Exception: Failed to get imported subscribers because of an assembly load exception.

LoaderExceptions of exception:

Loading this assembly would produce a different grant set from other instances. (Exception from HRESULT: 0x80131401)

We upgraded to NetStandard 2.0 and .NET Framework 4.7.1 and because of that we had to add the following lines of code to our .csproj files:

<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>

If we remove this, the problem goes away, but then we have other issues. We need this in our test project.

Does anyone have any ideas what the issue may be?