FatalExecutionEngineError in Constructor (with Service Fabric ApplicationList.First())

1

EDIT: I think the underlying issue is that First() was throwing in the middle of a constructor. From the end of this question:

this code is being called in a constructor for an object created in RunAsync(), mostly because it's an easy way to get it to run the code without having to interact with the service after deployment.

See my answer (the accepted one) if you're having this issue, since more likely than not something is throwing an exception in a constructor or something similar. Otherwise, read on to see the original question which has been preserved below.


I've got a service fabric application consisting of two services, one of which needs to dynamically discover HTTP endpoints of the other one's instances/partitions. Here's part of the code I've written to do this:

ServicePartitionResolver sfResolver = ServicePartitionResolver.GetDefault();
FabricClient fabricClient = new FabricClient();

// Find the correct application
ApplicationList apps = fabricClient.QueryManager.GetApplicationListAsync().Result;

// ERROR HERE!    
Application app = apps.First(a => a.ApplicationName.OriginalString.Equals("fabric:/MyApplication", StringComparison.OrdinalIgnoreCase));

// Find the correct service
ServiceList services = fabricClient.QueryManager.GetServiceListAsync(app.ApplicationName).Result;

// Same error will happen here too...
Service service = services.First(s => s.ServiceName.OriginalString.Equals("MyService", StringComparison.OrdinalIgnoreCase));

// Find all partitions
ServicePartitionList partitions = fabricClient.QueryManager.GetPartitionListAsync(service.ServiceName).Result;

foreach (Partition partition in partitions)
{
    // Do stuff with the endpoints - not really important for this question
}

At the indicated line, I get some variation of the following error when I debug the service on a local cluster:

FatalExecutionEngineError occurred
Message: Managed Debugging Assistant 'FatalExecutionEngineError' has detected a problem in 'C:\SfDevCluster\Data\_App\_Node_2\blah\blah\blah'.
Additional information: The runtime has encountered a fatal error. The address of the error was at 0xf17ac5d6, on thread 0x1f28. The error code is 0x80131623. This error may be a bug in the CLR or in the unsafe or non-verifiable portions of user code. Common sources of this bug include user marshaling errors for COM-interop or PInvoke, which may corrupt the stack.

What's strange is that I can copy that line verbatim into the immediate window in VS 2015 and it runs without issue and returns the expected result. Additionally, in an early prototype when I was learning how to use the service fabric libraries, I was iterating over the list in a foreach loop without error, so this variant of the above actually works:

ServicePartitionResolver sfResolver = ServicePartitionResolver.GetDefault();
FabricClient fabricClient = new FabricClient();

// Find the correct application
ApplicationList apps = fabricClient.QueryManager.GetApplicationListAsync().Result;
Application app;
foreach (Application a in apps)
{
    if (a.ApplicationName.OriginalString.Equals("fabric:/MyApplication", StringComparison.OrdinalIgnoreCase))
    {
        app = a;
        break;
    }
}

// etc...

I suspect that LINQ somehow isn't playing nicely with these service fabric types. If it matters, this code is being called in a constructor for an object created in RunAsync(), mostly because it's an easy way to get it to run the code without having to interact with the service after deployment - this seems to be the real issue. Any idea what's going on or if there's any remedy for this apart from using the less pretty foreach?

c#
asked on Stack Overflow Jan 25, 2018 by wlyles • edited Sep 10, 2018 by wlyles

1 Answer

2

As it turns out, a FatalExecutionEngineError can occur when the garbage collection heap gets corrupted. The list of ways this can happen is astoundingly long, but in this instance it was an unhandled exception being thrown during the service's RunAsync() method. This crashed the service, leaving some turds in the gc heap in the process, which in turn crashed the debugger.

answered on Stack Overflow Mar 20, 2018 by wlyles

User contributions licensed under CC BY-SA 3.0