Sunday, July 22, 2007

__traits and Pyd

D 2.003 adds compile-time reflection, very much in the manner of what my previous post was asking for. These new features are all included under the umbrella of the new __traits keyword, which the grammar refers to as the TraitsExpression.

I've re-written Pyd about three times. The first time was when I wrote my own pseudo-tuple library (now defunct). The second time was when I replaced that with Tomasz Stachowiak's (aka h3r3tic) tuple library (which later evolved into std.bind). The third time was when D received proper language support for tuples. This last rewrite was characterized by a vast reduction in code.

Subsequent updates to Pyd have focused on its class-wrapping features, particularly the various insanities I've introduced to handle polymorphic behavior properly. Despite my best efforts, there are still some rather serious flaws in it:

  • Inherited methods are not automatically placed in the generated shim class. This means that, to get the full power of Pyd's polymorphism wrapping, you need to explicitly wrap all of the methods of a class, including the inherited ones. I don't believe this is documented, either, but, then again, it has never come up.
  • There are some serious issues with symbol identifier lengths, which I am still grappling with.

With __traits, all of this simply goes away. Wrapping a class, any class, could easily be made to look like this:

wrap_class!(Foo);

And it would all Just Work. I hope to have it done before the conference.

Naturally, this means future versions of Pyd will probably require D 2.003 or newer. I do regret having to do this. It means that, in addition to getting all this new support for __traits in there, I will have to get Pyd and the Python/C API bindings compliant with the new const semantics. This could very well end up being harder, but I won't know until I try.

1 comment:

EricR said...

Hi and thank you for PyD !

I'm experiencing problems whith the garbage collector when I try to use PyD
under Linux with the GDC compiler
gdc (GCC) 4.1.1 20060524 ( (gdc 0.22, using dmd 1.004)).

I obtained the simplest example by modifying the rawexample of the pyd source. Here is the D code:

// A module written to the raw Python/C API.

module rawexample;
import python;
import std.gc;

extern(C)
PyObject* myCollect(PyObject* self, PyObject* args) {
fullCollect();
Py_INCREF(Py_None);
return Py_None;
}

PyMethodDef[] rawexample_methods = [
{"myCollect", &myCollect, METH_VARARGS, ""},
{null, null, 0, null}
];

extern(C)
export void initrawexample() {
PyObject* m = Py_InitModule("rawexample", rawexample_methods.ptr);
}

So I define a single function myCollect which only calls fullCollect(). I compile this into a shared library, using the following setup file:

from celerid.support import setup, Extension

projName = 'rawexample'

setup(
name=projName,
version='0.1',
ext_modules=[
Extension(projName, ['rawexample.d', ],
extra_compile_args = [
],
raw_only=True)
],
)

then I run the following test.py file

import os.path, sys
import distutils.util

# Append the directory in which the binaries were placed to Python's sys.path,
# then import the D DLL.
libDir = os.path.join('build', 'lib.%s-%s' % (
distutils.util.get_platform(),
'.'.join(str(v) for v in sys.version_info[:2])
))
sys.path.append(os.path.abspath(libDir))
import rawexample

rawexample.myCollect()
rawexample.myCollect()
print "test.py: done"

This results in nothing printed and a segmentation fault. If I call myCollect only once, the message "test.py: done" is printed but then there is still a segmentation fault.

Note that I designed this example after some mysterious crashes of a larger program which didn't call directely "fullCollect", but I suspected the gc because the program failed only after a large number of iterations...

If you have an idea about how to fix this, please let me know !

EricR