Error while unloading appdomain. (Exception from HRESULT: 0x80131015), inside Windows Service

I receive this error in a windows service. This is the same service that I've previously discussed in my question here

The code is revised to use Parallel.ForEach (my own version as this is a 3.5 windows service). The reason for the Parallel use is down to the fact that it simply took too long to Unload each domain and running them in parallel should prove to be faster (appears to be even though there is only one thread that's doing each Unload?!).

Based on other posts, I can only guess that this is somehow down to the fact I am using a ThreadPool Thread to Unload the AppDomains. I just can't see how to avoid it?

public partial class SomeService : ServiceBase
{
    private Manager _appDomainManager;

    protected override void OnStop()
    {
        _appDomainManager.Dispose();
    }
}

public class Manager : IDisposable
{
    public void Dispose()
    {
        Log.Debug("Disposing");
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected virtual void Dispose(bool disposing)
    {
        if (_disposed) return;
        if (disposing)
        {
            // dispose managed resources
            Parallel.For(0, appdomains.Length, UnloadAppDomian);
        }

        _disposed = true;
    }
}

private UnloadAppDomain(int appDomainIndex);

public static class Parallel35
{
    public static void For(int start, int end, Action<int> action)
    {
        var waitHandles = new WaitHandle[end - start];
        for (int j = 0; j < waitHandles.Length; j++)
        {
            waitHandles[j] = new ManualResetEvent(false);
        }

        for (int i = start; i < end; i++)
        {
            int i1 = i - start;
            ThreadPool.QueueUserWorkItem(
                state =>
                {
                    try
                    {
                        action((int) state);
                    }
                    finally
                    {
                        ((ManualResetEvent) waitHandles[i1]).Set();
                    }
                }, i);
        }
        WaitHandle.WaitAll(waitHandles);
    }
}

I tracked down this as a bug to one of the AppDomains on exit waiting for a WaitHandle that's never set.

If a thread does not abort, for example because it is executing unmanaged code, or because it is executing a finally block, then after a period of time a CannotUnloadAppDomainException is thrown in the thread that originally called Unload.

The AppDomain now unloads relatively quickly and my service stops quite quickly.