Windows Powershell - On Software Release - How To Kill Unkillable Process(s) Before Software Deployment Without Restarting

I have a series of issues that are not unique in and of them selves, but collectively are very unique. I have a series of helper console applications(C#), that do various things such as connect to and modify SSAS databases, access external API's, etc... These are invoked from a SQL Agent job on a periodic basis.

Every once in a while and for some completely unknown reason, one of these console applications will become unresponsive, this can cause the SQL agent to lock up, or it may continue to run but you won't be able to restart the service, it sort of abandons the process, such that the process is running but not doing anything, it's doesn't consume CPU cycles, or use any significant amount of memory, but it's "Unkillable", i.e. you can't stop it from the task manager or any other way. Looks something like this:

These can stack, sometimes you'll have multiple unresponsive processes on a single box. The only way to clear up this mess is to reboot the server. I have tried so many ways of killing the process(s), without success.

This causes serious issues for my automated release process, periodically my console apps are updated, when that occurs and this issue is present you get something such as the following, when you attempt to copy to the app's folder with the overwrite flag set to true (Robocopy in this case, but you can imagine this would be an issue in any other tool too):

2020-10-23T18:24:36.9534370Z 2020/10/23 13:24:36 ERROR 32 (0x00000020) Copying File XXX\Autofac.dll 2020-10-23T18:24:36.9535263Z The process cannot access the file because it is being used by another process.

Again to solve, you then need to take the advise of @FakeName here, and reboot the server (by hand) and rerun the release, a major pain. What I would like to do is find a way to programmatically via Powershell kill a "unkillable" process without rebooting my machine.

When you start looking around at ways to do this you soon find that there are some contradictory statements on this point. First "Raymon Chen" of Microsoft states:

So users can kill any process they want (given sufficient privileges), they can stop any program from stealing focus, and they can delete any file they want (again, given sufficient privileges).

@ralphtheninja states something similar for this closely related question. However if you dig deeper, you will see that there are many, though not official supported or 100% fool proof ways to to create a unkillable process. The most popular and most frequently mentioned reason for a "unkillable process" is the issue of buggy driver code. Since I'm not using any drivers that I'm aware of, and see this issue across different servers and versions of windows, and see this issue in multiple console apps doing various different things, I'm lead to believe that none of these methods or explanations have any relation to my issue.

If I take a step back and look at the entire concept of "the process" in windows, it seems to me that this type of problem has much more to do with the lack of clear direction and consistent statements from Microsoft and other parties as it related to the administration of processes, to summarize:

1. It's not clear whether an administrator with full access "should" be allowed to kill all (and I really mean this "all" word literally, not a metaphor) processes through powershell (or any other method such as the task manager).

2. It's not clear which command/method a task should/can be killed with in powershell or in the UI (clearly demonstrated by the proliferation of third party tools in the UI, and various ways via CMD or powershell), "Stop-Process" is clearly preferred, "Stop-Process" clearly doesn't kill "All" processes.

3. It's not clear what reasons a task should/can not be terminated, how to address these reasons, and for those of us who don't develop window device drivers (other then blanket statements such as "reboot your system") how to diagnose and determine the root cause of why applications that we develop can become "unkillable" in windows.

I have yet to find (with significant research):

• A single reason or list of reasons for why a process would become unkillable.

• A definitive guide of any sort to diagnosing issues of this kind (in C# or any other language excluding guides specific to driver development).

• A guide to understanding the different ways of killing a process, explaining not only how but why the community (stack and others) have so many different ways of doing the same thing, and most importantly why each different way would or would not work on a specific process.

Is rebooting the server (or some other non-process related workaround) before every software release the only solution to this issue?