troubleshooting error code 1000 application crash

Problem solved alert. Read the final update first. .................................................

I have vb6 application that calls a c# library via COM

The C# library is Framework 4.5.2

If I build the COM library on a particular machine running VS2017 15.5.6 I don't have a problem.

If I checkout the same code and build it on a different machine ( I tried 2 of them) with vS2017 15.5.2 in a particular record I get an application crash.

The error occurs on the line of code

if (edge.Extra == null) // given edge is not null and Extra is a property

In the Windows Event log there is

Faulting application name: jtJobTalk.exe, version: 1.0.0.0, time stamp: 0x5a9f5b1c
Faulting module name: ntdll.dll, version: 6.3.9600.18895, time stamp: 0x5a4b127e
Exception code: 0xc00000fd
Fault offset: 0x0006d46c
Faulting process ID: 0xb74
Faulting application start time: 0x01d3b5c77355520d
Faulting application path: C:\jobtalk\jtJobTalk.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report ID: d1374fd2-21ba-11e8-8272-d050999dc03c

I tried sfc scan and no problems were reported.

On yet another computer (running Windows 7) I get the error

A new guard page for the stack cannot be created

How can I further troubleshoot this issue?

I am afraid to update the VS version on the good machine in case this causes me to be unable to release.

[Update]

After putting in some calls to MessageBox.Show I have established that the error was caused by an object referencing itself from within it's own constructor.

It has taken me a day at least to find this out. I am looking for any pearls of wisdom that could have helped me diagnose the issue in an easier way.

Exception code: 0xc00000fd

That is STATUS_STACK_OVERFLOW. A very common mishap, and hard to debug, so much so that they named a popular programmer web site after it. You are getting the raw version of this mishap with little assistance from the debugger. Always an issue in interop code, you can't rely on the managed debugger engine to help you diagnose it. Google is your best bet, all of the top hits for this phrase help you get on the right path to start fixing it.

If I build the COM library on a particular machine...

That is being unlucky, turning over the wrong stones to look for a cause can bog you down for a while. SOE is never caused by build problems or a buggy VS version, always a coding mistake. The most basic reason it would occur on one machine but not another is that you don't test the program with the exact same data. Or made a quicky coding change that looked very innocent, but wasn't.

if (edge.Extra == null)

That is one of the very common causes for SOE, a buggy property getter. Something like this:

public class Example {
    private Foo edge;
    public Foo Edge {
        get { return Edge; }   // Oops, meant edge
    }
}

You can certainly look at this for a while and never see it. It would be nice if the compiler had a diagnostic for it, but the C# compiler does not have the necessary plumbing to ferret this out. The other very common cause is a field initializer:

public class Example {
    private Example foo = new Example();
    // etc...
}

Which can easily get more convoluted from there, when for example you create an instance of another class, and that class creates an Example object in its constructor. And the C# language supports writing recursive code, it is one of the standard programming techniques. If that code is any more complex than O(log(n)) then you can always easily crash it with too much data.

...any pearls of wisdom

Yes, there is one. If you don't have the managed debugging engine helping out with exceptions then diagnosing errors gets pretty hard to do. The VB6 runtime can provide you with the exception message, but not the Holy Stack Trace. Info that is lost in the transition between the two very different runtime environments.

But you can have that cake and eat it too, the trick is to get the managed debugger to start the VB6 IDE. Right-click your C# class library project > Properties > Debug tab. Select the "Start external program" radio button and type "C:\Program Files (x86)\Microsoft Visual Studio\VB6\VB6.exe". You can optionally set the "Command line arguments" to the full path of your .vbp project, that way the IDE automatically loads your VB6 project. Use Debug > Windows > Exception Settings and tick the "Common Language Runtime Exceptions" checkbox so it displays the tick mark. This makes the debugger stop on any C# exception, before it is passed to your VB6 code.

Press F5 and the VB6 IDE starts running. Press F5 again to start your VB6 code. Any mishap in the C# code now causes the managed debugger to step in. Usually the display automatically switches to the VS IDE, but sometimes you have to click the blinking taskbar button. You get to look at the code that threw the exception and use Debug > Windows > Stack Trace to find out how it got there.

I'm not 100% sure that this also works to diagnose SOE, the VB6 runtime might step in too soon to allow the CLR to see the exception. I don't have VB6 installed anymore to check. Please try it and let me know.