2011-07-20

Reflection by introspective frontal lobotimization in D





Many of the widely used languages of today have built in runtime
reflection. From the top of my head i can mention Java, Python, Ruby,
C#, Lua, SmallTalk - popular languages with reflection mechanisms,
that allow you to inspect objects at runtime and call methods based on
name-lookups. Even C++ can get some reflection mechanisms when
used together with Qt and parsed with the moc. D on the other hand,
does not have reflection. And the first day - or was it the second - I
tinkered with D, I wanted to write something that would naturally call
for reflecion had I written it in, say Python.

D does however have something that the other already mentioned
languages does not have: A very powerful compile time code generation
and inclusion mechanism, the mixin. AFAIKU probably only surpassed by
the macro capabilities featured by common lisp. A quote from
Alexandrescus book, The D Programming Language, (p.47) seems fitting:
"If expressions were various kinds of screwdrivers, mixin would be a
power screwdriver with exchangeable heads, adjustable clutch, a brain
surgery adapter, built in wireless camera, and speech recognition." D
runs a full-fledged interpreter during compilation, which enables you
to write D code that generates D code, and have it executed during
compilation in order to compile in the code generated from it as part
of your program. Awsomeness. It's so powerful it makes me go all
GEB. Mixins allows you to fill a really nice bag of both tricks and
treats and this blog post is about how to exploit them together with
the compile-time inspection mechanisms in D, in order to create a rude
form of reflection so as to look up and call objects function based on
its name.

Lets say I have a class Foo as follows:

class Foo {
  void callOne(){
  }
  void callTwo(){
  }
  void dontCallMe(){
  }
}

I want to extract a subset of these, say all that have names that
begin with "call", and build a collection of callables that enables me
to call them when I have an Foo instance. Using a mixin, I could for
example compile in the following code:

mixin("static function(Foo)[] functions = 
    [function(F f){ f.callOne(); },
     function(F f){ f.callTwo(); }]");

The 'functions' array is added to what ever scope the mixin reside
in. Unfortunately, this is not in it's entirety deducible compile-time
by the D-compiler (known bug). But that doesn't matter, there is a
simple workaround - write static function wrappers and add these to
the function table, instead of anonymous functions. The following
little expansion compiles and do exactly that.

mixin("
  static void callOne(Foo f){
    f.callOne();
  }

  static void callTwo(Foo f){
    f.callTwo();
  }

  static function(Foo)[] functions = {&callOne, &callTwo};");

Now I have an array with two functions, that allows me to call the
wanted functions just through indexing in this manner:

Foo f = new Foo;
functions[0](f); // calls f.callOne()
functions[1](f); // calls f.callTwo()

Now we need only two things: Something to deduce the functions we
should wrap in this manner while compiling, and something to generate
the code we need for those functions. Lets do the code generating
first - this is where the magic is.

string generateFunctionTables(string[] names){
    string res = "";
    foreach(name; names){
      res ~= "static " ~ name ~ "(T t){ t." ~name ~ "();}\n";
    }
    res ~= "static void function(T)[] callFunctions = [";
    foreach(name; names){
      res ~= "&"~name~",";
    }
    res ~="];";
    return res;
}

mixin(genereateFunctionTable(["callOne", "callTwo"]));

While compiling, the included D-interpreter kicks in and call
generateFunctionTable during compilation, and build the code we want
depending on the input array of function names (and, I might mention,
it give fairly sensible compile-time errors if the function names supplied
are wrong.)

The final part is next to trivial - there are introspection mechanisms
available at compile-time, that allows listing all functions on a type,
and maybe match them with a string prefix:

string[] enlistFunctions(){
  string[] res;
  foreach(func; __traits(allMembers, Foo)){
    if(func.indexOf("call") == 0){
      res ~= func;
    }
  }
  return res;
}

This function can be called during compilation, and the results can be
used as arguments to our already defined code generator. We get the
table we want, simply by using this line:

mixin(genereateFunctionTable(enlistFunctions()));

So far, We've only known about the class Foo, so this is not very
general. Wrapping it in a templated scope improves things a bit. The
final code looks like this:

import std.stdio;
import std.array;
import std.string;

class Foo{
  void callOne(){
    writeln("One Called");
  }
  void callTwo(){
    writeln("Two called");
  }
  
  void dontCallMe(){
    assert(false);
  }
};

template NamedFunctions(T, string prefix = "call"){

  string buildStaticFunctionWrappers(string[] names){
    string res = "";
    foreach(name; names){
      res ~= "static " ~ name ~ "(T t){ t." ~name ~ "();}\n";
    }
    res ~= "static void function(T)[] callFunctions = [";
    foreach(name; names){
      res ~= "&"~name~",";
    }
    res ~="];";
    return res;
  }

  string[] enlistFunctions(){
    string[] res;
    foreach(func; __traits(allMembers, T)){
      if(func.indexOf(prefix) == 0){
        res ~= func;
      }
    }
    return res;
  }

  mixin(buildStaticFunctionWrappers(enlistFunctions()));
  static string[] functionNames = enlistFunctions();
}

int main(){
  Foo f = new Foo;
  foreach(idx; 0..NamedFunctions!Foo.functionNames.length){
    writeln("\nExecuting:" ~ NamedFunctions!Foo.functionNames[idx]);
    NamedFunctions!Foo.callFunctions[idx](f);
  }
  return 0;
}

In order to make it a bit more general I've wrapped it in a
parameterized scope, parameterized on the type of the object and a
matching string for the functions. In addition I added the name-table
so that I can match function with function name (I was unable to
generate a name-function map during compilation, the compiler frowned,
but this structure give me what I want indirectly. The name-function
map can be constructed runtime though.)

The output is as follows:

banach:Source rolvseehuus$ ./reflect

Executing:callOne
One Called

Executing:callTwo
Two called

In summary, I've demonstrated how to use compile time introspection
and mixins in order to extract certain functions from a class based on
a naming convention, enabling code to indirectly identify these
functions by a name-lookup that can be extended to a reflection
database. It can be improved further, but it stands on it's own as an
example (I wish I had it a week ago.:) I can think of several places
where this might be useful - like when interfacing with other
languages, receiving messages from another process, or say - building
an xUnit-esque testing framework for D.