Large 2D array in C - Stack OverFlow error

3

I haven't used C in long time, and I'm having an issue filling a 2D array from a CSV. The file format is like this:

Node, In, Out

1,200,10393

...

This is essentially an array representation of a linked list. There are 150000 elements and whenever I try to fill the array I get an error saying "Unhandled exception at 0x000000013facb957 in main.exe: 0xC00000FD: Stack overflow." I'm on a 64-bit machine with 16GB of RAM and I'm using VS C++ 2010 Express with an x64 build configuration.

int main(int argc, char *argv[])
{

int counter = 0;
char line [ 1024 ];
int map[150000][2] = {0};
char *comma = ",";
char *token;
int index;
int in, out;
char* end;
int nodeID;
FILE *fp;

fp = fopen("mapsorted.txt","r"); // read mode

if( fp == NULL )
{
  perror("Error while opening the file.\n");
  exit(EXIT_FAILURE);
}

//Skip header line
fgets ( line, sizeof line, fp );

while ( fgets ( line, sizeof line, fp ) != NULL) /* read a line */
{
    //first part - the index for storage
    token = strtok(line,comma);
    index = strtol(token,&end,10);

    //second part
    token = strtok(NULL,comma);
    in = atoi(token);

    //third part
    token = strtok(NULL,comma);
    out = atoi(token);

    //store in array
    map[index][0] = in;
    map[index][1] = out;
}
fclose ( fp );
}

The code seems to work when I allocate a smaller array, but fails when it is this large. I think I should have enough memory to be able to handle an array of this size.

c
arrays
stack-overflow
asked on Stack Overflow Dec 30, 2012 by user1489497 • edited Sep 29, 2018 by Cœur

2 Answers

7
int map[150000][2];

seems at least 2 * 4 * 150000 bytes (assuming a modern 32-bit architecture), which is roughly 1.2MB. Knowing that modern OSes typically set up a stack size of a few megabytes, this could actually be the problem. Your computer having several gigabytes of RAM doesn't mean all of it can be consumed by your process, especially not on the stack. For large arrays, try malloc()ating some memory on the heap:

int (*map)[2] = malloc(sizeof(*map) * 150000);

or

int *map = malloc(150000 * 2 * sizeof(*map));

(pay attention to the dimensions in the second case!), or declare it as static to move it out of stack space:

static int map[150000][2];

Or simply make it a global variable to achieve a similar behavior:

int map[150000][2];

void foo()
{
    ...
}
answered on Stack Overflow Dec 30, 2012 by (unknown user) • edited Dec 30, 2012 by (unknown user)
2

The array is too big to fit in the stack. Try moving it outside of the function:

static int map[150000][2] = {0};
int main(int argc, char *argv[])
{

and so on.

answered on Stack Overflow Dec 30, 2012 by Erik Ekman

User contributions licensed under CC BY-SA 3.0