Upgrading IdentityServer AccessTokenValidation from version 3 to 4 leads to HTTP 500.19

Currently we have a working ASP.NET WebAPI 2 project which is compiling and running with IdentityServer3.AccessTokenValidation. After uninstalling the NuGet package and installing the newer IdentityServer4.AccessTokenValidation the project now stops running and exhibits an HTTP Error 500.19 - Internal Server Error. We've tried a lot of things but we're not sure how to identity the root issue and resolve. Consistently upgrading to IdentityServer4 reproduces the error.

Detailed Error Information:

Module:       CustomErrorModule
Notification: SendResponse
Handler:      Not yet determined
Error Code:   0x80070490
Config Error: The configuration section 'system.webServer/httpErrors' cannot be read because it is missing a section declaration
Config File:  \\?\C:\Users\joe\AppData\Local\dftmp\Resources\91a2e717-5753-4ca7-b31d-40e068e91e84\temp\temp\RoleTemp\applicationHost.config

At present all the Identity Server code in Startup.cs is commented out so essentially the only change is the replacement of the IdentityServer libraries which reliable recreates this issue

For additionally context, here are some details:

• This is WebAPI 2
• .NET 4.6
• This would be hosted as an Azure Cloud Service but currently we are just working locally until we identify the issue
• We had no httpErrors section defined and adding one which had no impact on the presence of the error
• This listed file does not exist
• We have made no permissions changes and shouldn't have too since its a Cloud Service

UPDATE 1

After additional analysis we have observed another error which is occurring just before the one mentioned above. We have confirmed the same thing is also occurring on the 'clean' VM mentioned in the comments

Detailed Error Information:

Module:       CustomErrorModule
Notification: SendResponse
Handler:      ExtensionlessUrl-Integrated-4.0
Error Code:   0x80070002
Config Error: Cannot read configuration file
Config File:  applicationHost.config

Our config section contains the following handlers. None of this changed as part of the upgrade and removing it has no impact on the error

<handlers>
  <remove name="ExtensionlessUrlHandler-Integrated-4.0" />
  <remove name="OPTIONSVerbHandler" />
  <remove name="TRACEVerbHandler" />
  <add name="ExtensionlessUrlHandler-Integrated-4.0" path="*." verb="*" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
</handlers>

V4 is for ASP.NET Core only...