how to increase allocated memory for my cpp program, g++/vs

here is the code:

dataSet->A = (int *)malloc(sizeof(int) * dataSet->n * dataSet->m);
dataSet->B = (int *)malloc(sizeof(int) * dataSet->n * dataSet->m);
dataSet->C = (int *)malloc(sizeof(int) * dataSet->n * dataSet->m);

Variables m and n are coming from user input using scanf(). Using small values for m and n, there is no problem but when I use m = n = 16384 so that each array is a 1GB one, using G++ my program crashes. And using Microsoft Visual Studio 2017, first access to B causes an exception:

Access violation writing location 0x00000000

(It's kinda odd that I'm using Windows 10 64bits and address above is 32 bits)

Looks like malloc is not allocating B and C correctly.

OS: win 10 64 bits RAM: 16 GB