Error: Unable to get the remote environment variable 'WindowsDirectory' using msdeploy tempagent

When using msdeploy to deploy an IIS app to a remote server, using the Web Deploy On Demand (tempagent) feature, I am getting this error:

./myApp.deploy.cmd : Error: Unable to get the remote environment variable 'WindowsDirectory' on the computer 
'myserver.mydomain.com'.
At line:1 char:1
+ ./myApp.deploy.cmd /T /M:$target /U:$username /P:$ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Error: Unable t...com'.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
Error: The object invoked has disconnected from its clients. (Exception from HRESULT: 0x80010108 (RPC_E_DISCONNECTED))
Error count: 1.

I'm using the auto-generated deploy script that gets created by the Package target in msbuild. I'm running in the directory that all those artifacts live. This is my script to run that script:

$target = 'myserver.mydomain.com'
$username = 'myusername'
$password = 'mypassword'


./myApp.deploy.cmd /T /M:$target /U:$username /P:$password /G:true

This is the script that is generated by the .cmd script, the same exact error happens if i run it directly:

SetParameters from:
"C:\Users\MaxCadmin\Desktop\Package\myApp.SetParameters.xml"
You can change IIS Application Name, Physical path, connectionString
or other deploy parameters in the above file.
-------------------------------------------------------
 Start executing msdeploy.exe
-------------------------------------------------------
 "C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -source:package='C:\Users\MaxCadmin\Desktop\Package\myApp.zip' -dest:auto,computerName="myserver.mydomain.com",userName="myusername",password="mypassword",includeAcls="False",tempAgent="true" -verb:sync -disableLink:AppPoolExtension -disableLink:ContentExtension -disableLink:CertificateExtension -setParamFile:"C:\Users\MaxCadmin\Desktop\Package\myApp.SetParameters.xml" -whatif 
Info: Using ID 'bce210a0-29f9-4859-8fc2-f308c6ce5e72' for connections to the remote server.

I'm using the tempagent feature so that I don't have to go install Web Deploy on all my targets, and keep them all updated to the same version. I much prefer the prospect of using the Temp Agent, removing the need for any special configuration of the targets at all. I've confirmed that my target in this case does not have Web Deploy installed on it.

I believe it's the Temp Agent that is throwing the disconnect error at the end of the error message. I'm more concerned about the error regarding the inability to get the remote variable.

For what it's worth, I went on the remote server and created that env variable, with the value matching that of windir, which is C:\Windows, and proved to myself that it was present in a new shell. But it didn't change the error at all.