I deleted all instances of Netwonsoft.json and now my TFS 2017 build agent fails - how do I get it back?

This has been plaguing us for some time and every time I update other NuGet package versions it seems to re-appear for some reason. If I use MSBuild to build my solution it works fine but using TFS build the build fails because it uses aspnet_compiler.exe and gives the following message:

error ASPCONFIG: Could not load file or assembly 'Newtonsoft.Json,  
Version=11.0.0.2, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of 
its dependencies. The located assembly's manifest definition does not match the 
assembly reference. (Exception from HRESULT: 0x80131040)

The command I'm using that fails is:

C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe -v 
localhost_53906 -p PICCS.WebServices\PICCS.WebHost\ -u -f PrecompiledWeb\localhost_53906\

This happens whether I use version 11.0.0.1 or 11.0.0.2 and I've narrowed it down to the website in the project - not a .csproj. I've tried deleting the Newtonsoft folders from my packages folder and re-getting them but that didn't help. Ultimately I deleted all newtonsoft instances from my computer and reinstalled versions 11.0.0.1 and 11.0.0.2 and then I can build using the command above. But now TFS build agent fails because it can't find Newtonsoft.json.9.0.1. How can I get that back? Reinstall VS? Reinstall .net? I tried just creating a new agent but that fails with this error:

Error: assembly specified in the dependencies manifest was not found -- package:
'newtonsoft.json', version: '9.0.1', path: 'lib/netstandard1.0/Newtonsoft.Json.dll'