Unhandled exception at 0x00007FF74F27A526 in OpenARK-SDK.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x000000EEC5803FD8)

The following code snippet causes stackoverflow.

Unhandled exception at 0x00007FF74F27A526 in OpenARK-SDK.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x000000EEC5803FD8).

How can I fix it? It is part of an open-source repository I am contributing to and I hope not to do a major change here:

/***
Recursively performs floodfill on depthMap
***/
void DepthCamera::floodFill(int x, int y, cv::Mat& depthMap, cv::Mat& mask, double max_distance)
{
    if (x < 0 || x >= depthMap.cols || y < 0 || y >= depthMap.rows || depthMap.at<cv::Vec3f>(y, x)[2] == 0.0)
        return;
    if (closeEnough(x, y, depthMap, 4, max_distance)) {
        mask.at<cv::Vec3f>(y, x) = depthMap.at<cv::Vec3f>(y, x);
        depthMap.at<cv::Vec3f>(y, x)[0] = 0;
        depthMap.at<cv::Vec3f>(y, x)[1] = 0;
        depthMap.at<cv::Vec3f>(y, x)[2] = 0;
    }
    else {
        return;
    }

    floodFill(x + 1, y, depthMap, mask, max_distance);
    floodFill(x - 1, y, depthMap, mask, max_distance);
    floodFill(x, y + 1, depthMap, mask, max_distance);
    floodFill(x, y - 1, depthMap, mask, max_distance);
}

/***
Check whether candidate point is close enough to neighboring points
***/
bool DepthCamera::closeEnough(int x, int y, cv::Mat& depthMap, int num_neighbors, double max_distance)
{
    int num_close = 0;
    if (x - 1 < 0 || depthMap.at<cv::Vec3f>(y, x - 1)[2] == 0 || Util::euclidianDistance3D(depthMap.at<cv::Vec3f>(y, x), depthMap.at<cv::Vec3f>(y, x - 1)) < max_distance) {
        num_close++;
    }
    if (x + 1 >= depthMap.cols || depthMap.at<cv::Vec3f>(y, x + 1)[2] == 0 || Util::euclidianDistance3D(depthMap.at<cv::Vec3f>(y, x), depthMap.at<cv::Vec3f>(y, x + 1)) < max_distance) {
        num_close++;
    }
    if (y - 1 < 0 || depthMap.at<cv::Vec3f>(y - 1, x)[2] == 0 || Util::euclidianDistance3D(depthMap.at<cv::Vec3f>(y, x), depthMap.at<cv::Vec3f>(y - 1, x)) < max_distance) {
        num_close++;
    }
    if (y + 1 >= depthMap.rows || depthMap.at<cv::Vec3f>(y + 1, x)[2] == 0 || Util::euclidianDistance3D(depthMap.at<cv::Vec3f>(y, x), depthMap.at<cv::Vec3f>(y + 1, x)) < max_distance) {
        num_close++;
    }

    if (num_close >= num_neighbors) {
        return true;
    }

    return false;
}

and

double Util::euclidianDistance3D(cv::Vec3f pt1, cv::Vec3f pt2)
{
    double dx = pt1[0] - pt2[0];
    double dy = pt1[1] - pt2[1];
    double dz = pt1[2] - pt2[2];

    return sqrtf(dx*dx + dy*dy + dz*dz);
}

It is called by:

void DepthCamera::computeClusters(double max_distance, double min_size)
{
    clusters.clear();
    cv::Mat depthMap = cv::Mat::zeros(depthMap.rows, depthMap.cols, depthMap.type());
    cv::medianBlur(xyzMap, depthMap, 3);
    cv::Mat mask = cv::Mat::zeros(depthMap.rows, depthMap.cols, depthMap.type());
    for (int r = depthMap.rows - 1; r >= 0; r--) {
        for (int c = 0; c < depthMap.cols; c++) {
            if (depthMap.at<cv::Vec3f>(r, c)[2] > 0.2) {
                mask = cv::Mat::zeros(depthMap.rows, depthMap.cols, depthMap.type());
                floodFill(c, r, depthMap, mask, max_distance);
                cv::Mat channels[3];
                cv::split(mask, channels);
                if (cv::countNonZero(channels[2]) > min_size) {
                    cv::medianBlur(mask, mask, 3);
                    clusters.push_back(mask.clone());
                }
            }
        }
    }
}

Please let me know if further information is needed. Basically, when I get closer to the camera, the stackoverflow exception happens.

Here's a screenshot of the callstack:

Here's a screencast of me trying to run without debugging it.

The ending conditions for your recursion are incomplete or incorrect.

If you look at floodfill, two cases return and one triggers recursion. That tells us the recursion block is incorrect. Specifically, that closeEnough() is not working as intended since you've triggered a case that never returns true.

You would look at the variables being passed to closeEnough to see why they never satisfy it. Just looking at the code I can tell that this is never true for the case where the stack overflows:

  if (num_close >= num_neighbors) {

I could guess at which conditions are not being met, but you can do that much more easily by looking at the values being passed to closeEnough.

If that isn't enough, please give us the values.