Publishing to Azure causes "Internal server error" error sometimes after publishing from VS2017

I've got a simple .Net CORE website, which works fine locally. If I create a new webapp in Azure and publish to it, everything works fine. If I publish the project again after a random number of publishes, the site crashes with "Internal server error" 500.2 error messages. It still happens if I don't change any code and just republish the same code again. Creating a new webapp in Azure and publishing the same code to that has everything working fine again.

Digging into Azure I find the error ErrorCode = '0x80004005' : 8000808c Which translates to Error code: 0x80004005 means a file missing or can't be accessed.

The full error being

MODULE_SET_RESPONSE_ERROR_STATUS
Warning 
ModuleName="AspNetCoreModule", Notification="EXECUTE_REQUEST_HANDLER", HttpStatus="502", HttpReason="Bad Gateway", HttpSubStatus="5", ErrorCode="Unspecified error
 (0x80004005)", ConfigExceptionInfo=""

I've tried changing appsettings.json, changing the version of CORE in the project, deleting the wwwroot directory from the server and everything else that's on here .NET Core app unable to start in IIS due to ErrorCode = '0x80004005 : 80008083 amongst other suggestions.

Anyone got any idea how publishing a project to Azure can break like this.