So today I found myself trying to figure out how to pickle objects that are based on dynamically constructed classes. This is probably the simplest instance of metaclass magic in Python — the use of the ‘type’ builtin, a metaclass. A metaclass is a class whose objects are themselves class specifications. People use metaclasses to modify the default way classes are built, or in my case to build classes dynamically (say from a soup of mixins).
Anyway, the following code ran into problems.
import pickle classes = {} classes['ClassName'] = type('ClassName', (),{}) obj = classes['ClassName']() pickle.dump(obj,open('test.pck','w'))
Attempting to run this code gives:
Traceback (most recent call last): File "/tmp/py17808UNS", line 8, in pickle.dump(obj,open('test.pck','w')) File "/usr/lib/python2.5/pickle.py", line 1362, in dump Pickler(file, protocol).dump(obj) File "/usr/lib/python2.5/pickle.py", line 224, in dump self.save(obj) File "/usr/lib/python2.5/pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "/usr/lib/python2.5/pickle.py", line 401, in save_reduce save(args) File "/usr/lib/python2.5/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib/python2.5/pickle.py", line 562, in save_tuple save(element) File "/usr/lib/python2.5/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib/python2.5/pickle.py", line 748, in save_global (obj, module, name)) PicklingError: Can't pickle : it's not found as __main__.ClassName
So what was I trying to do? I’ve implemented a learning agent as an object and constructed a bunch of mixins that specify variations of the training algorithm, reward function, and various other modifiable agent properties. What I want to do is take the cross product of all the variations, testing each on the cluster here at UT. Of course I want to pickle each agent so that I can inspect the resulting performance later without having to rerun a thousand experiments.
I only have a small number of variations, so explicitly naming all the mixin combinations isn’t too onerous, but I am curious as to whether lack of serialization inhibits the use of mixins in Python programming generally.
Or maybe I’m just programming orthogonal to the norm.
