Errors when trying to input a large amount of data from csv file

0

I'm trying to read data into c++ from a csv file. Although the program compiled, there was no output. When I ran the debugger I saw there were some "Unhandled Exceptions". One stated that there was a stack overflow. Another said "0xC0000005: Access violation reading location 0x001000000". I'm not really sure what these mean, but when I tested a very similar program with a smaller set of data, it worked.

In my current code, I've declared 12 arrays, each representing a column. Each array contains 537578 elements, representing every row.

int raw_num = 537578;
int num = 537577;
std::string raw_User_ID[537578];
std::string raw_Product_ID[537578];
std::string raw_Gender[537578];
std::string raw_age[537578];
std::string raw_Occupation[537578];
std::string raw_City_Category[537578];
std::string raw_Stay_In_Current_City_Years[537578];
std::string raw_Marital_Status[537578];
std::string raw_Product_Category_1[537578];
std::string raw_Product_Category_2[537578];
std::string raw_Product_Category_3[537578];
std::string raw_Purchase[537578];

/* Arrays below are for conversion of data types later on but are not used in this part */

double User_ID[537577];
std::string Product_ID[537577];
char Gender[537577];
std::string age[537577];
int Occupation[537577];
char City_Category[537577];
std::string NumYearsInCity[537577];
bool Marital_Status[537577];
int Product_Category_1[537577];
int Product_Category_2[537577];
int Product_Category_3[537577];
double Purchase[537577];

std::ifstream infile;
infile.open("BlackFriday.csv");
if (!infile.is_open()) {
    std::cout << "File not found" << std::endl;
}
else {
    int count = 0;
    while (!infile.eof()) {
        std::getline(infile, raw_User_ID[count], ',');
        std::getline(infile, raw_Product_ID[count], ',');
        std::getline(infile, raw_Gender[count], ',');
        std::getline(infile, raw_age[count], ',');
        std::getline(infile, raw_Occupation[count], ',');
        std::getline(infile, raw_City_Category[count], ',');
        std::getline(infile, raw_Stay_In_Current_City_Years[count], ',');
        std::getline(infile, raw_Marital_Status[count], ',');
        std::getline(infile, raw_Product_Category_1[count], ',');
        std::getline(infile, raw_Product_Category_2[count], ',');
        std::getline(infile, raw_Product_Category_3[count], ',');
        std::getline(infile, raw_Purchase[count], '\n');
        count++;
    }
}

I outputted a few array elements to ensure the data was entered properly, but there was no output. Also, the code exited -1073741571 and not 0.

c++
asked on Stack Overflow Apr 17, 2019 by chineetim • edited Apr 17, 2019 by chineetim

1 Answer

1

The stack overflow means that you allocated more memory than is available in the stack, and applications often get terminated for that. Large arrays should be allocated on the heap you could do this with pointers, but if you don't have any restrictions, I would recommend using std::vector<std::string> product_id(537577); instead of std::string .... you can treat the vector like you would an array, and the vector will do the memory management for you.

answered on Stack Overflow Apr 17, 2019 by Jonathon K

User contributions licensed under CC BY-SA 3.0