Pandas drop_duplicates -> Fatal Python error: deallocating None

2

I have a code that checks an Excel sheet, and if it finds some changes, then takes a snapshot (Pandas Dataframe) of the whole sheet and saves it to a csv with a timestamp. It has been running all day doing it's job correctly, but usually once or twice a day I got the error message below:

Fatal Python error: deallocating None

Current thread 0x000013d4 (most recent call first):
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\algorithms.py", line 1655 in take_nd
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\internals\blocks.py", line 1238 in take_nd
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\internals\managers.py", line 1235 in <listcomp>
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\internals\managers.py", line 1235 in reindex_indexer
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\internals\managers.py", line 1350 in take
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\generic.py", line 3359 in _take
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\frame.py", line 2969 in _getitem_bool_array
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\frame.py", line 2918 in __getitem__
  File "C:\Program Files (x86)\Python37-32\lib\site-packages\pandas\core\frame.py", line 4637 in drop_duplicates
  File "C:\Python projects\RTD_recorder_1thrd_Pandas.py", line 79 in <module>

This is the code which causes the freeze:

final_result = pd.concat([final_result, difference]).drop_duplicates(keep="first", subset=indexes)

Has anyone any idea what could go wrong here? Here is the full code if it helps: https://www.codepile.net/pile/BRx6W2e0

python
pandas
drop-duplicates
asked on Stack Overflow Apr 5, 2019 by RLaszlo

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0