Dynamic Pipeline in MetaSharp

I’ve been working on a prototype of a Dynamic Pipeline for MetaSharp using the new Linq Expressions in .NET 4. I’m pretty darn excited about it, here is what I have working so far:

[Test]
public void BinaryVariablesTest()
{
    var result = this.pipeline.Eval<int>("x * y", new { x = 3, y = 7 });
    Assert.AreEqual(21, result);
}

What I’m doing here is compiling the string as MetaSharp code then using a node visitor to generate the appropriate linq expressions. The trick here is the context parameter being passed in. This object contains all of the properties and methods allowed to be called from the expression, essentially it’s the “this” parameter. To get this to work you can use the handy dandy ConstantExpression. Here is how I’m doing it:

public class ReferenceExpressionVisitor 
    : TransformVisitor<linq.Expression, ReferenceExpression>
{
    protected override IEnumerable<linq.Expression> Visit(
        IEnumerable<linq.Expression> items, 
        ReferenceExpression visitedObject)
    {
        IDynamicContextService parameterService = 
            this.ServiceProvider.Locate<IDynamicContextService>();
    
        yield return linq.Expression.PropertyOrField(
            linq.Expression.Constant(parameterService.Context),
            visitedObject.Name);
    }
}

The constant allows us to call properties or fields on the context object directly. You should even be able to attach lambda expressions to fields so you can call those as well. I have to do some restructuring in order to enable everything like Method calls and whatnot but it should all be doable. Very sweet. Also the dynamic pipeline will have to have some type of caching provider, clearly you wouldn’t want to rebuild these expressions every single time you try to execute it. Optionally I could just give back a delegate instead of executing it directly, then let the caller do all the caching. That sounds easier and more flexible actually now that I think about it.

What’s cool about this though isn’t the fact that you can compile expressions dynamically (which is pretty cool but there are others out there already) but the fact that you will be able to dynamically compile DSLs… which reduce to dynamic expressions. Imagine this:

when truck is late:
  apply discount(.1);

when truck is early:
  apply bonus(.1);

when truck cost > $10,000:
  notify

You compile your DSL into MetaSharp nodes, transform it into Common language nodes then transform that into dynamic linq expressions. Very sweet! You might transform this into something like:

if(truck.late)
{
    context.discount(.1);
}
if(truck.early)
{
    context.bonus(.1);
}
if(truck.cost > 10000)
{
    context.notify();
}

Where you’d have a custom ShippingContext object with all of the methods and properties you wanted to expose in your DSL. This would be really handy for creating all sorts of systems that have the potential to have rapidly changing rules and also, potentially, enable business analysts to author their own rules.

The Illusion of Strongly Typed XML with C#

This is essentially a redo of an earlier blog post of mine on how to create the illusion of a strongly typed xml in Boo using Duck Typing, except this time we’re using the dynamic keyword in C#.

http://cid-dfcd2d88d3fe101c.skydrive.live.com/embedrowdetail.aspx/blog/justnbusiness/XmlBuilder.cs

Here is an example of how it might look to use such a thing.

dynamic builder = new XmlBuilder();
builder.Main.One = "Hello";
builder.Main.Two = "World";

Console.WriteLine(builder.ToString());

Console.WriteLine(
    "{0} {1}!",
    builder.Main.One,
    builder.Main.Two);

builder = new XmlBuilder();
builder.Main["One"] = "Hello";
builder.Main["Two"] = "World";

Console.WriteLine(builder.ToString());

Console.WriteLine(
    "{0} {1}!",
    builder.Main["One"],
    builder.Main["Two"]);

XmlBuilder is a class I created myself, it inherits from DynamicObject and it intercepts TryGetMember, TrySetMember, TryGetIndex and TrySetIndex in order to allow you to navigate an internal XmlDocument object. Here is the output when the above is run.

<Main><One>Hello</One><Two>World</Two></Main>
Hello World!
<Main One="Hello" Two="World" />
Hello World!

Notice the use of properties corresponds to drilling into elements and the use of indexers corresponds to drilling into Attributes. This is just a proof of concept but you could see how this sort of thing would be useful.

Here is the complete XmlBuilder class that makes it all happen.

public class XmlBuilder : DynamicObject
{
    private Func<string, XmlElement> createElement;
    private Func<string, XmlAttribute> createAttribute;
    private XmlNode node;

    public XmlBuilder()
    {
        this.node = new XmlDocument();
        this.createElement = s => ((XmlDocument)this.node).CreateElement(s);
        this.createAttribute = s => ((XmlDocument)this.node).CreateAttribute(s);
    }

    private XmlBuilder(
        XmlNode node, 
        Func<string, XmlElement> createElement, 
        Func<string, XmlAttribute> createAttribute)
    {
        this.node = node;
        this.createElement = createElement;
        this.createAttribute = createAttribute;
    }

    public override bool  TryGetMember(GetMemberBinder binder, out object result)
    {
        XmlNode found = this.node[binder.Name];
        if (found == null)
        {
            found = this.createElement(binder.Name);
            this.node.AppendChild(found);
        }

        result = new XmlBuilder(found, this.createElement, this.createAttribute);
        return true;
    }

    public override bool TrySetMember(SetMemberBinder binder, object value)
    {
        XmlNode found = this.node[binder.Name];
        if (found == null)
        {
            found = this.createElement(binder.Name);
            this.node.AppendChild(found);
        }

        found.InnerText = value.ToString();
        return true;
    }

    public override bool TryGetIndex(GetIndexBinder binder, object[] indexes, out object result)
    {
        if (indexes.Length != 1)
        {
            throw new InvalidOperationException("You may only specify a single index.");
        }

        XmlNode found = null;
        if (indexes[0] is string)
        {
            string name = (string)indexes[0];
            found = this.node.Attributes[name];
            if (found == null)
            {
                found = this.createElement(name);
                this.node.AppendChild(found);
            }
        }
        else if (indexes[0] is int)
        {
            int index = (int)indexes[0];
            found = this.node.Attributes[index];
        }

        if (found != null)
        {
            result = new XmlBuilder(found, this.createElement, this.createAttribute);
            return true;
        }
        else
        {
            result = null;
            return false;
        }
    }

    public override bool TrySetIndex(SetIndexBinder binder, object[] indexes, object value)
    {
        if (indexes.Length != 1)
        {
            throw new InvalidOperationException("You may only specify a single index.");
        }

        XmlAttribute found = null;
        if (indexes[0] is string)
        {
            string name = (string)indexes[0];
            found = this.node.Attributes[name];
            if (found == null)
            {
                found = this.createAttribute(name);
                this.node.Attributes.Append(found);
            }
        }
        else if (indexes[0] is int)
        {
            int index = (int)indexes[0];
            found = this.node.Attributes[index];
        }

        if (found != null)
        {
            found.InnerText = value.ToString();
            return true;
        }
        else
        {
            return false;
        }
    }

    public override string ToString()
    {
        return this.node.InnerXml;
    }
}

Member Exists – Dynamic C# 4.0

I have been using the dynamic keyword a little bit in C# and I have recently run into a small problem and I have been trying to figure out the most elegant way to resolve it. The problem is, when you try to invoke a member that does not exist on a dynamic object it throws an exception, there is no easy way to detect if the member exists built-in to accommodate this.

For example what I would really like is an ‘exists’ keyword, something like:

dynamic instance = new { Foo = "Hello World!" };
if (exists(instance.Foo))
{
Console.WriteLine(instance.Foo);
}

But there appears to be no magic ‘exists’. Turns out the only way to do this is with reflection. So I created some helper extension methods to enable the following example:

object instance = new { Foo = "Hello World!" };

if (instance.Reflection().Exists().Foo)
{
    string value = instance.Reflection().Call().Foo;
    Console.WriteLine(value);
}

Interestingly enough the Foo in this case is a call to a DynamicObject I created that can intercept calls to members and returns a bool if they exist, never calling them. The Reflection().Call() simply casts any object into a dynamic object for dynamic calling.

I also found out that you can’t use the dynamic Type as the first parameter in an extension method. Dynamic objects and extension methods don’t really play together very nicely.

Download the full example here:

https://onedrive.live.com/redir?resid=DFCD2D88D3FE101C%21341

LambdaExpression.CompileToMethod … not nearly as cool as I had hoped.

I was messing around with the new dynamic expressions in C# 4 (found in System.Linq.Expressions). One new thing I noticed was the method CompileToMethod on LambdaExpressoin. The first parameter is a MethodBuilder so I got really excited, finally an easy way to create dynamic methods on Types!

Wrong, turns out the method has to be Static, also it cannot accept, as a parameter, the Type the method is being built on. So this basically completely negates the entire purpose. At this point you might as well simply compile it to a delegate and use that instead. Bummer.

Here is an explanation of why:

After much investigation, it turns out this is by won’t fix for CLR 4.0

You can’t use expression trees to model instance methods or constructors. First problem: you can’t create the “this” parameter, because all you have in hand is a TypeBuilder, but that is not the final type. We can create a delegate with the TypeBuilder, but CLR won’t let us create a LambdaExpression with that delegate (because the type is not finished). You can workaround that by making the “this” parameter be of type object, then you end up with meaningless casts. Worse, calls to your own methods don’t work, because MethodBuilder doesn’t implement enough of reflection for expression trees to do their normal sanity checks.

DynamicMethods run into their own problems. When emitting into a DynamicMethod, we can’t add our Closure argument to the signature of the DynamicMethod. Without a closure,

  • DynamicMethods run into some serious limiations:
    They can’t have nested lambdas at all (due to a CLR limitation: can’t ldftn to a DynamicMethod).
  • Some things that are O(1) or amortized O(1) become O(N), such as RuntimeVariables and Switch on strings. This is really sneaky for the user who won’t expect things to suddenly be slower.

This needs work we plan for ETs v3, and the design around the support will likely change.

One potential work around I have contemplated but haven’t gotten up the energy to try would be to try to generate a matching interface first, then pass that around as a parameter to the static method. You’d have to give access to all fields through explicitly implemented properties though, destroying encapsulation in the process. So if suppose you wanted to generate something like this:

public class Test
{
    private int foo;

    public int Bar()
    {
        return this.foo;
    }
}

Instead you would generate something like this:

public interface ITest
{
    int foo { get; set; }

    int Bar();
}

public class Test : ITest
{
    private int foo;

    int ITest.foo
    {
        get { return this.foo; }
        set { this.foo = value; }
    }

    public int Bar()
    {
        return Test.Bar(this);
    }

    public static int Bar(ITest test)
    {
        return test.foo;
    }
}

The benefits of using the Linq expressions to build your method bodies quickly diminish with this sort of workaround however. You might as well just go back to the ILGenerator (*shudders*). This would probably work because the static Bar method can accept a parameter of a Type already dynamically created.

CSharp Optionally Dynamic Calls

I’ve been reading about how the C# team is considering putting the option to call dynamic code within C# directly and I have been looking at some of the syntax they have been proposing publicly. It also seems like there are a handful of people who are proposing their own ideas of how to express various features of C# they wish existed so I thought I’d propose one of my own.

Personally I find the idea of a dynamic method or a dynamic block a little clunky. So I would like to propose a different possible solution.

The whole idea is that in C# the compiler has to know the type of a variable so it can guarantee that the method you are trying to call exists. This is largely the purpose of interfaces. However, there are some circumstances where you might prefer to call a method dynamically. The dynamic method block syntax Charlier Calvert is proposing is very reminiscent of the “unsafe” synatx in C# however if you really wanted to make the language more dynamic in general I think you could simply have the option to call the memeber with something other than a ‘.’ to indicate that it should be a dynamic call. Something more like:

string example = obj:Example();

In this case I'm suggesting that merely replacing the '.' with a ':' will cause the application to call the Example method dynamically. I belive this would still conform to the explicit nature of C# while making dynamic calls quite flexible. If the colon was an undesierable character it would be simple enough to just use something else.

In this case you would not get much intellisense related to the parameters or return value of Example wich I believe would prevent it from being used except in circumstances where it was absolutely needed. Also you would only get runtime exceptions if the types of the parameters and return values were incorrect. It would probably have to assume that the return Type was actually an object if you used the "var" syntax.

Also, you would need some way to check to see if the member existed at the very least:

string example;
if(exists(obj:Example))
   example = obj:Example();

Of course, this should be essentially the same for fields, properties and events. For example:

string example = obj:Example; // field or property
obj:ExampleEvent += new EventHandler(OnExampleEvent);