Possible stack overflow, but no error is being raised

0

I'm using Python 3.7, VSCode for some geographic data processing (forest fire detection). My script is exiting in the middle of a recursive function, without any error nor traceback...

This is the code that crash (I deleted every line that seemed not essential to the understanding of the function, trying to make it more clear):

def SentinelSecondaire(dicoRef, dicoSRef, Rels, SNT, SRefTraite=list()):
    for sref, srefv in sorted(dicoSRef.items(), key=lambda x: x[1]):
        if sref not in SRefTraite:
            Slies1 = list()
            for r in Rels:
                if r[0] == sref:
                    if r[3] not in dicoSRef.keys() and r[3] in SNT:
                        Slies1.append(r[3])
                elif r[3] == sref:
                    if r[0] not in dicoSRef.keys() and r[0] in SNT:
                        Slies1.append(r[0])
            idRef = dicoSRef[sref]
            Slies1 = list(set(Slies1))
            if len(Slies1) > 0:
                for s in Slies1:
                    dicoRef[idRef].append(s)
                    dicoSRef[s] = idRef
                    if s in SNT:
                        SNT.remove(s)
                        SSec = SentinelSecondaire(dicoRef, dicoSRef, Rels, SNT, SRefTraite)
                        dicoRef = SSec[0]
                        for ss in SSec[1].keys():
                            if ss not in dicoSRef.keys():
                                dicoSRef[ss] = idRef
                        SNT = SSec[2]
            SRefTraite.append(sref)
    return [dicoRef, dicoSRef, SNT]

where

Rels = [["D1", ..., date, "S1", ..., date1], ..., ["D2", ..., date, "S400", ..., date1]]

dicoRef = {
"D1": ["S1", "S30"],
"D2": ["S201", "S400"]
}

dicoSref = {
"S1": "D1",
"S30": "D1",
...,
"S400": "D2"
} # Some kind of reversed dicoRef

I have of course set sys.setrecursionlimit(high_enough) and that is why I think that I'm getting a stack overflow error. Moreover, when testing my script with PyCharm, I have the following exit message : "Process finished with exit code -1073741571 (0xC00000FD)" that is referenced on the web as a stack overflow error.

I'm looking for a way to overcome this issue, either changing my function to an iterative one or to a tail-recursive one, or pushing the stack size limit farther... This looks like an DFS graph function, but the for loops are confusing me. I didn't write the program, I'm just trying to make it work with new (bigger) set of data.

Edit: Link to the code and file (it is difficult to provide more organized testing data, as the length of data is 10^5 to 10^6 elements).

https://github.com/Kaupahcox/Graph_issue.git

On my machine, the crash happens somewhere around traitement of D15.

python
stack-overflow
graph-theory
asked on Stack Overflow May 26, 2021 by Côme DAVAL • edited May 27, 2021 by Côme DAVAL

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0