PyCharm failed when trying to read a big .csv file while spyder succeeded

1

I have both pycharm and spyder installed on our remote desktop. I personally prefer pycharm and having been developing using it. Everything is fine until I found that I can not read a .csv file which is bigger than 1 GB in pycharm, it told me python crashed and the log shows "Process finished with exit code -1073740940 (0xC0000374)". At first I thought maybe the .csv file has been broken. But in spyder, the pandas succeeded reading it. So the .csv file is good. I do not know why, I did not change the configuration of my PyCharm, maybe someone did, because there are several other colleagues who also have access to this remote desktop.

I am somewhat sure that the file is OK and I had been reading .csv files which are even bigger than 20GB in pycharm using pandas, so anybody have any idea about this?

pandas
pycharm
spyder
asked on Stack Overflow Jul 24, 2018 by Colin

2 Answers

0

Try changing the

idea.max.content.load.filesize=20000

line in the ide.properties. It controls the

Maximum file size (kilobytes) IDE is able to open.

answered on Stack Overflow Jul 24, 2018 by Nipun Sampath
0

This should open a 30mb file.

You can get to your properties file from Help > Edit Custom Properties.

idea.max.intellisense.filesize=30000
idea.max.content.load.filesize=30000

You may need to increase the total memory allotment in pycharm after you do this. I know for me, pycharm slowed to a crawl while trying to read in a file this big.

More details on these settings: https://www.jetbrains.com/help/pycharm/file-idea-properties.html

answered on Stack Overflow Mar 18, 2021 by cvrooman • edited Mar 18, 2021 by cvrooman

User contributions licensed under CC BY-SA 3.0