ASP.NET Core MVC: The located assembly's manifest definition does not match the assembly reference

I tried to deploy my Asp.net core mvc projekt to our server, but the ISS only logs this error

System.IO.FileLoadException: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The located assembly's manifest definition does not match the assembly reference. (0x80131040)
File name: 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'

It does work localy, but not on the server.

I found this Could not load file or assembly Microsoft.Extensions.DependencyInjection.Abstractions, Version=1.1.0.0 thread about the problem, but the accepted answer is for the .net Framework, not .net core (and did not work, I did try it) and the later answers are about Azure functions which I dont use.

I use net core 3.1

So, this was probably a noob mistake: I use Core 3.1 but NuGet added the references for 5.0

<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="5.0.0.0" />

After downgrading to the 3.1.14 Version, everything

    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="3.1.14" />