webapi: have different nuget versions of a package in the same solution

I have a webapi project (project A) that has a dependency another class library project (project B). The project B has a nuget package that has a dependency of the package System.IdentityModel.Tokens.Jwt with version greater than 5.2 (I have the version 5.2.2). But the project A is using the nuget Microsoft.Owin.Security.Jwt and System.IdentityModel.Tokens.Jwt. Now, the guys that are maintaining these nugets screwed some things and made some breaking changes in these packages. If I'm using Microsoft.Owin.Security.Jwt version 4.0 (the last one) and System.IdentityModel.Tokens.Jwt version 5.2.2, I'm getting the error

Could not load type 'Microsoft.Owin.Security.Jwt.IIssuerSecurityTokenProvider' from assembly 'Microsoft.Owin.Security.Jwt, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

This is because they've renamed some classes (you can see the discussion in this thread). So in project A I've used the nuget Microsoft.Owin.Security.Jwt with version 3.1.0, but this has some incompatibility with the package System.IdentityModel.Tokens.Jwt version 5.2.2 because I'm getting the error:

Could not load type 'System.IdentityModel.Tokens.TokenValidationParameters' from assembly 'System.IdentityModel.Tokens.Jwt, Version=5.2.2.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.

So I've lowered the version of the nuget System.IdentityModel.Tokens.Jwt to the version 4.0.4 and now my project starts, but when I'm calling a controller that uses a helper class from the project B that has to do with some identity server classes, my api clashes with the error:

Could not load file or assembly 'System.IdentityModel.Tokens.Jwt, Version=5.2.2.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

So is there a way to make my solution work with two versions of the nuget System.IdentityModel.Tokens.Jwt (4.0.4 and 5.2.2)?