Identity login function stopped working Blazor Wasm Asp.net core hosted

I've been playing with a blazor wasm asp.net core hosted project for a few weeks now, yesterday I added a few controllers on the server side and everything was working fine (Also logging in).. Came back today and identity server just stopped logging in?? I can reach the api (and thus the database) through swagger but when I go to the login page of identity and login with a user the iis just shuts down.. no logs, no idea why. I can register users I can pretty much do anything else but login..

When I login the last thing I see in the debug log console is:

  "Name": "Token Issued Success",
  "Category": "Token",
  "EventType": "Success",
  "Id": 2000,
  "ClientId": "MyProject.Client",
  "ClientName": "MyProject.Client",
  "RedirectUri": "https://localhost:44367/authentication/login-callback",
  "Endpoint": "Authorize",
  "SubjectId": "d0053d07-e530-4e49-acfe-dd9a35a18d12",
  "Scopes": "openid profile MyProject.ServerAPI",
  "GrantType": "authorization_code",
  "Tokens": [
    {
      "TokenType": "code",
      "TokenValue": "****so1w"
    }
  ],
  "ActivityId": "80000012-0000-fa00-b63f-84710c7967bb",
  "TimeStamp": "2020-09-01T07:57:14Z",
  "ProcessId": 23820,
  "LocalIpAddress": "::1:44367",
  "RemoteIpAddress": "::1"
}
The program '' has exited with code -1 (0xffffffff).
The thread 0x0 has exited with code 0 (0x0).
The program 'localhost:44367' has exited with code -1 (0xffffffff).
The program '[23820] iisexpress.exe' has exited with code -1 (0xffffffff).

Anyone maybe an idea as to why this started happening all of the sudden??

I've read that it might have to do with sdk references so to be as complete as possible I've added my .csproj files for the client, server and the shared projects.

Server:

  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <UserSecretsId>MyProject.Server-1C8DF269-9BED-44E5-A7A0-C94CD6D45333</UserSecretsId>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.Server" Version="3.2.1" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="3.1.7" />
    <PackageReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Design" Version="3.1.4" />
    <PackageReference Include="Swashbuckle.AspNetCore.Swagger" Version="5.5.1" />
    <PackageReference Include="Swashbuckle.AspNetCore.SwaggerGen" Version="5.5.1" />
    <PackageReference Include="Swashbuckle.AspNetCore.SwaggerUi" Version="5.5.1" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\Client\MyProject.Client.csproj" />
    <ProjectReference Include="..\Shared\MyProject.Shared.csproj" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore" Version="3.1.7" />
    <PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="3.1.7" />
    <PackageReference Include="Microsoft.AspNetCore.Identity.UI" Version="3.1.7" />
    <PackageReference Include="Microsoft.AspNetCore.ApiAuthorization.IdentityServer" Version="3.1.7" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="3.1.7" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="3.1.7">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>

</Project>

Client:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netstandard2.1</TargetFramework>
    <RazorLangVersion>3.0</RazorLangVersion>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly" Version="3.2.1" />
    <PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.Build" Version="3.2.1" PrivateAssets="all" />
    <PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.DevServer" Version="3.2.1" PrivateAssets="all" />
    <PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.Authentication" Version="3.2.1" />
    <PackageReference Include="Microsoft.Extensions.Http" Version="3.1.7" />
    <PackageReference Include="System.Net.Http.Json" Version="3.2.1" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\Shared\MyProject.Shared.csproj" />
  </ItemGroup>

</Project>

Shared:

  <PropertyGroup>
    <TargetFramework>netstandard2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="System.ComponentModel.Annotations" Version="4.7.0" />
    <PackageReference Include="System.Text.Json" Version="4.7.2" />
  </ItemGroup>

</Project>

UPDATE:

After more research I found that this is only happening on the chrome browser.

UPDATE 2:

Tried all configuration options I could find for samesite cookies and those also didn't work, also tried configuring openIdConfiguration but that also didn't work.