OData client unable to retrieve data

I have a basic implementation of an ASP service providing OData, nothing really special. I am able to connect to the service via a browser and look up the data. I also can connect via fiddler or Postman. Just to get things rolling I wanted to create a simple tool to connect to the service. So I created a Console application, installed OData Connected Service and got some NuGet packages:

• Microsoft.OData.Client
• Microsoft.OData.Core
• Microsoft.OData.Edm
• Microsoft.Spatial
• Simple.OData.Client
• Simple.OData.V3.Client
• Simple.OData.V4.Client

I then connected the services using the OData Connected Service addon, which succeeded and generated an Csdl.xml file:

<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx">
  <edmx:DataServices>
    <Schema Namespace="Service" xmlns="http://docs.oasis-open.org/odata/ns/edm">
      <EntityType Name="User">
        <Key>
          <PropertyRef Name="Id" />
        </Key>
        <Property Name="Id" Type="Edm.String" Nullable="false" />
        <Property Name="Name" Type="Edm.String" />
      </EntityType>
      <EntityType Name="Emitter">
        <Key>
          <PropertyRef Name="Id" />
        </Key>
        <Property Name="Id" Type="Edm.Int64" Nullable="false" />
        <Property Name="Strength" Type="Edm.Double" Nullable="false" />
        <Property Name="Type" Type="Edm.String" />
      </EntityType>
      <EntityContainer Name="Service">
        <EntitySet Name="User" EntityType="Service.User" />
        <EntitySet Name="Emitter" EntityType="Service.Emitter" />
      </EntityContainer>
    </Schema>
  </edmx:DataServices>
</edmx:Edmx>

And also a Reference file, which is too big to post here, but looks fine.

Accessing via browser gives me the correct response:

[{"id":"50ed6fdb-d4e2-4586-b6d8-99d9a1b25a4d","name":"afasdf"},{"id":"c53c3fbc-8d0d-4a72-9576-d1029f215ccf","name":"test"}]

also fiddler, metadata:

HTTP/1.1 200 OK
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true
Server: Microsoft-IIS/10.0
OData-Version: 4.0
X-Powered-By: ASP.NET
Date: Mon, 04 May 2020 13:51:04 GMT
Content-Length: 171

{"@odata.context":"https://localhost:44377/service/$metadata","value":[{"name":"User","kind":"EntitySet","url":"User"},{"name":"Emitter","kind":"EntitySet","url":"Emitter"}]}

and access to the user:

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Mon, 04 May 2020 14:27:21 GMT
Content-Length: 123

[{"id":"50ed6fdb-d4e2-4586-b6d8-99d9a1b25a4d","name":"afasdf"},{"id":"c53c3fbc-8d0d-4a72-9576-d1029f215ccf","name":"test"}]

However, accessing the data using a simple client fails with an exception:

> System.AggregateException
  HResult=0x80131500
  Message=One or more errors occurred. (The response payload is a not a valid response payload. Please make sure that the top level element is a valid Atom or JSON element or belongs to 'http://docs.oasis-open.org/odata/ns/data' namespace.)
  Source=System.Private.CoreLib
  StackTrace:
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at ConsoleApp1.Program.Main(String[] args) in Program.cs:line 37

  This exception was originally thrown at this call stack:
    [External Code]
    ConsoleApp1.Program.Print() in Program.cs

Inner Exception 1:
InvalidOperationException: The response payload is a not a valid response payload. Please make sure that the top level element is a valid Atom or JSON element or belongs to 'http://docs.oasis-open.org/odata/ns/data' namespace.

The code:

class Program
{
    static Service.Service service = new Service.Service(new Uri("https://localhost:44377/Service"));
    static DataServiceCollection<User> User = new DataServiceCollection<User>();

    static private Task<IEnumerable<User>> GetUsersAsync(IQueryable<User> query)
    {
        DataServiceQuery<User> dsquery = (DataServiceQuery<User>)(query);
        return Task.Factory.FromAsync<IEnumerable<User>>(dsquery.BeginExecute, dsquery.EndExecute, null);
    }

    static private async Task Print()
    {
        IQueryable<User> query = from u in service.User select u;
        var users = await GetUsersAsync(query);
        User.Load(users);

        foreach (var v in User)
            Console.WriteLine(v.Id);
    }

    static void Main(string[] args)
    {
        Print().Wait(); // line 37
    }
}

I am fairly new to OData and ASP services and can't figure out the issue. I've tried several different things, async access (like the code above) and sync (which was kinda worse). I also tried different tutorials (where nearly all of them ar for .Net-Framework), none have worked so far. I also tried Simple Odata Client...

All projects are made in .Net-Core 3.1, and the test app I also tried in .Net-Framework...

Im fiddling on this thing for... feels like decades... Any hint is appreciated, or may even a direction to a fully working example.