After working with an issue in PyOgre I decided to take a closer look at Psyco (the Python JITer). Taking a look at how Psyco works is like learning how sausages are made. You just don’t want to go down that road.
This isn’t to say that Psyco is poorly written, badly executed, or anything like that. I just mean that the actual internals of Psyco are incredibly complex, and not something that can be picked up in a short amount of time. I am a big fan of Psyco though (its actual use, anyway). It’s a must-have for when you need to do something mathematically intensive in python.
However, there is a big problem with Psyco that seems to be the pink elephant in the room: it’s no longer being actively developed, and there are no replacements for it. In fact, even though it works for Python 2.4, it really has not been updated with some of the newer features. More specifically, it cannot optimize generators or generator expressions which is a huge drawback considering generators were meant to help with speed and memory issues.
Why isn’t it being actively developed anymore? Its author moved on to create something called PyPy, a Python interpreter…written in Python. A goal of the PyPy project is to translate the PyPy code from Python into C, and using JIT techniques (like Psyco) to speed it up even further.
To me, PyPy seems like an interesting academic project. It’s amazing that they are currently reporting passing 90% of Python’s regression tests, but even if this were at 100%, how much would this really help? A faster Python interpreter implementation would certainly be desirable, but it will never replace the original Python interpreter (even if PyPy can bootstrap itself).
I think a more useful Python implementation would be to rumpling the Python interpreter in C++. Making the Python/C api much nicer with C++ features such as function overloading, and using class inheritance instead of PyObject structs. More useful, but again would not replace the original Python.
I’m not really against PyPy. It’s certainly an interesting project, and maybe they can somehow make their generated C code run faster than Python-with-Psyco. However there is still a need for a Python JITer, and eventually someone will have to pick up the slack by either developing on Psyco again or starting a new JITer.
I would love to be the person to do that, but to be honest I have a fully plate already and JITer construction is not currently in my skillset. The Psyco code base is incredibly clean and well documented, but it would take a lot of time to become familiar with it and actually be able to modify the code to do something useful without breaking it.
I have to take an independent study before I graduate with a Computer Science degree. I think if Psyco is still inactive by that time, I am going to try to convince the department that learning to develop a JITer would be a good topic for study.
