.NET Core 3.1 web API builds fine locally, but gets errors when deploying via Azure DevOps

I had a similar problem a couple weeks ago:

I'm getting the error "Microsoft.AspNetCore.Hosting.Abstractions, Version=3.1.0.0" in .NET Core 3.1 app

I can run it from Visual Studio 2019 locally with no issues, but whenever I try to run it after deploying via Azure DevOps, I get this error:

Unhandled exception. System.IO.FileLoadException: Could not load file or assembly 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The located assembly's manifest definition does not match the assembly reference. (0x80131040)
File name: 'Microsoft.Extensions.DependencyInjection.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
   at System.Signature.GetSignature(Void* pCorSig, Int32 cCorSig, RuntimeFieldHandleInternal fieldHandle, IRuntimeMethodInfo methodHandle, RuntimeType declaringType)
   at System.Reflection.RuntimeMethodInfo.get_ReturnType()
   at Microsoft.AspNetCore.Hosting.StartupLoader.FindMethod(Type startupType, String methodName, String environmentName, Type returnType, Boolean required)
   at Microsoft.AspNetCore.Hosting.StartupLoader.HasConfigureServicesIServiceProviderDelegate(Type startupType, String environmentName)
   at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.UseStartup(Type startupType, HostBuilderContext context, IServiceCollection services)
   at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<>c__DisplayClass12_0.<UseStartup>b__0(HostBuilderContext context, IServiceCollection services)
   at Microsoft.Extensions.Hosting.HostBuilder.CreateServiceProvider()
   at Microsoft.Extensions.Hosting.HostBuilder.Build()
   at KindRegards.Web.Api.Program.Main(String[] args) in /home/vsts/work/1/s/src/Api/Program.cs:line 10

I don't fully understand how all of this ties together, so I tried adding:

    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="5.0.0" />

but that didn't work. I have other "Microsoft.Extension" packages that are using version 3.1.8, so I tried matching those by adding:

    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="3.1.8" />

But then I couldn't run it locally, and I get this error:

NU1605  Detected package downgrade: Microsoft.Extensions.DependencyInjection.Abstractions from 5.0.0 to 3.1.8. Reference the package directly from the project to select a different version. 
 KindRegards.Web.Api -> KindRegards.Plugins.PaymentProcessors.GreenMoney -> Microsoft.Extensions.Http 5.0.0 -> Microsoft.Extensions.DependencyInjection.Abstractions (>= 5.0.0) 
 KindRegards.Web.Api -> Microsoft.Extensions.DependencyInjection.Abstractions (>= 3.1.8)    Web.Api Z:\Projects\KindRegards\src\Api\Web.Api.csproj  1   

I'm not sure how to even begin digging into this. Any help would be appreciated.

It happened with me before while using Azure DevOps and the solution was to check what is Azure DevOps using and I matched the project with it, I mean your target should be the version that Azure DevOps agent is using.

I do not remember exactly, but most likely I downgraded my app version to match the Azure DevOps.

This happens generally when your Visual Studio automatic updates are enabled so your .Net Core nugets become more up to date than the ones used in Azure DevOps