Cannot pickle object: maximum recursion depth exceeded

I'm trying to pickle objects that I generate in a script to process them afterwards, but I get this error:

File "<ipython-input-2-0f716e86ecd3>", line 1, in <module>
pickle.dump(thisperson, output, pickle.HIGHEST_PROTOCOL)
RecursionError: maximum recursion depth exceeded while pickling an object

Here thisperson is relatively simple (or so I thought): a name and a dictionary containing attributes:

class Attributes:
    def __init__(self, what_attr, attr):
        self._what_attr = what_attr
        self._attributes = attr

    @property
    def what_attr(self):
        return self._what_attr
    @what_attr.setter
    def what_attr(self, new_value):
        self._what_attr = new_value

    @property
    def attributes(self):
        return self._attributes
    @attributes.setter
    def attributes(self, new_value):
        self._attributes = new_value


class Person:
    def __init__(self, watname, watage):
        self._myname = watname
        self._age = watage
        self._attributes = []

    @property
    def attributes(self):
        return self._attributes
    @attributes.setter
    def attributes(self, attributes_list):
        self._attributes = attributes_list

    @property
    def myname(self):
        return self._myname
    @myname.setter
    def myname(self, value):
        self._myname = value

    @property
    def age(self):
        return self._age
    @age.setter
    def age(self, value):
        self._age = value

the pickle function looks like this:

import pickle

def save_person(person, mypath):
    import os
    if not os.path.exists(mypath):
        os.makedirs(mypath)
    my_path = mypath + str(person.myname)
    if not os.path.exists(my_path + '/'):
        os.makedirs(my_path + '/')
    with open(my_path + '.pkl', 'wb') as output:
        pickle.dump(person, output, pickle.HIGHEST_PROTOCOL)

This error seems to be coming from the fact that the class contains a string and a dict because if I keep attributes as an empty dict I don't get the error.

Is there another way I can pickle objects like these?

I tried increasing sys.setrecursionlimit, but that didn't help.

Edit: .... i can't reproduce the error if i use a short version of the code .... go figure. @MinhNguyen something weird is that i create my attributes dict in a function (attr = get_attributes(data)), but when i create the same dict myself, i can pickle it ....

thisperson = Person("Kenny", 22, {})
attr = get_attributes(data)  # returns {'eyes': 'blue', 'hair': 'brown'}
this_attr = Attributes(person_name, attr)
this_person.attributes.append(this_attr)
save_person(this_person, main_folder)

Exception ignored in: 
'_pydevd_frame_eval.pydevd_frame_evaluator_win32_37_64.get_bytecode_while_frame_eval'
 RecursionError: maximum recursion depth exceeded while calling a Python object
 Process finished with exit code -1073741819 (0xC0000005)

but if i do the following it works:

 attr = {'eyes': 'blue', 'hair': 'brown'}
 this_attr = Attributes(person_name, attr)
 this_person.attributes.append(this_attr)
 save_person(this_person, main_folder)

Solution

Alright so after further digging, i found here that pickle doesnt mingle with dict well and will often crash.

So the workaround was that i dropped the Attribute class completely and now my Person's _attribute is a string of dict self._attributes = str(dict()) and whenever i want to add things to it, i'll do an eval(), update() and str() again.

This seem to be working so far...

pickle is known to have a number of limitations. One of them you amply described - it does not handle well dictionaries where items are created on the fly.

I suggest you use dill as a direct pickle replacement. It can handle this case and many more. Just do this:

import dill as pickle

and leave the rest of the code intact. It should work nicely.