One Man’s Walk in work

So we’re working with Amazon’s SimpleDB. It’s pretty sweet, though the ruby libraries for it are still a bit primitive. One of the problems you run up against when you’re writing integration tests against it is eventual consistency.

Take this test :

it "should save" do
  customer = Customer.create!(:name => 'bob', :email => 'bob@example.com')
  customer = Customer.find(customer.key)
  customer.name.should == 'bob'
end

The way SimpleDB works, you’re assured that Customer.find will work…eventually, but not right away.

For a couple days we contented ourselves to just run the integration tests a couple times until they didn’t error out. But that got old.

Enter “eventually” :

it "should save" do
  customer = Customer.create!(:name => 'bob', :email => 'bob@example.com')
  customer = eventually { Customer.find(customer.key) }
  customer.name.should == 'bob'
end

It’s a very simple method (below) that just retries the passed in block until it succeeds, timing out after 10 tries. Super simple, works like a charm. Thank you ruby.

Here’s the source :

def eventually(tries = 0, &block)
  yield
rescue
  raise if tries >= 10
  sleep 0.5
  eventually(tries + 1, &block)
end

I will say, I can’t help but smile every time I write “eventually” in a test…

It’s very easy to call out to methods in markaby, but it’d be nice if you could actually customize the dsl as well.

For example, on many of our pages we have a bottom row that has buttons that look a certain way. So on every page, we have :

  table(:width => "100%") do
    tr do
      td.left do
        previous_button
        first_button
      end
      td.center do
        print
      end
      td.right do
        next_button
        last_button
      end
    end
  end

The code for the actual buttons changes, and after a few tries to extract the whole thing into a single method, we gave up. Our efforts had made it harder to read, not easier. There was always just a little too much variance, and it didn’t feel right.

What we really wanted to write was :

  last_row do
    column do
      previous_button
      first_button
    end
    column do
      print
    end
    column do
      next_button
      last_button
    end
  end

This lets the buttons that change all stay in the view, and gets rid of the skeleton and positional stuff that doesn’t change. Furthermore, it’s DRY and puts all that positional logic in one place instead of scattered across 20 views.

How to do this?

I wrote a test (in RSpec) that looks something like :

describe ApplicationHelper do
  it "should generate a table from a buttons method" do
    last_row(:columns => 2) do
      column do
        "foo"
      end
      column do
        "bar"
      end
    end.should == '<table width="100%"><tr>' +
                    '<td class="left">foo</td>' + 
                    '<td class="right">bar</td>' +
                  '</tr></table>'
  end
end

After a bunch of fiddling and poking around, I finally made the test (and a couple others) pass with this code in my ApplicationHelper :

def last_row(options, &block)
  markaby do
    table(:width => "100%") do
      tr do
        LastRowContext.new(self, options[:columns]).
                                  instance_eval(&block)
      end
    end
  end
end
 
class LastRowContext
  def initialize(markaby, columns)
    @markaby, @column_count, @column_index = 
                            markaby, columns, 0
  end
 
  def column(&block)
    alignment = case @column_index += 1
    when 1 : :left
    when @column_count : :right
    else :center
    end
 
    @markaby.instance_eval do
      td(:class => alignment, &block)
    end
  end
end

I’m sure this could get cleaned up more; this was the work of less than an hour. In particular, if you did this often, you could extract a common MarkabyContext superclass that had some convenience methods. The point is, this is really easy to do, and we shouldn’t be scared to try “Language level refactorings” like this.

Ever since Joe, Neal, Zak and I started working on our DSL book, we’ve had an outstanding question that we had no good concise and simple answer for.

How is a DSL different than an API?

We’ve spent hours talking, arguing and asking others how they would answer this question. Still the answers we’ve had so far have been too complicated and didn’t feel “right”, until today.

I just got back from breakfast w/ Rob Mee, and we talked about DSLs among other things. His answer to this question is far too simple, so simple, in fact that I think we might finally have it.

Unlike an API, a DSL has it’s own grammar.

But what if it’s an internal DSL? Well, it still has an implicit grammar over and above its host language grammar.
——
Rob has a background in Linguistics and drew parallels between the difference between Pidgin Languages and Creole Languages

A Pidgin Language is one that forms when adults from 2 disparate languages come into contact and try to communicate. It has a minimal grammar and verb forms. It often follows subject – verb – object order and uses extra words for tenses. There are no native speakers of a Pidgin Language.

However, when adults that speak a Pidgin Language teach it to their children, a funny thing happens : their children add grammar to it. They become native speakers and the language morphs into a Creole Language.

Creole Languages are much richer than their Pidgin parents. It’s possible to more easily express complex concepts in Creole than Pidgin. They have more structure and can be more concise and exact as a result.

Interesting.

(if you’re interested in Pidgin & Creole, Rob recommends The Language Instinct)

I am struck by how my friends, my english major, only use a computer for e-mail and myspace friends are unwittingly becoming programmers.

It’s all google’s fault.

They’ve gone and taught my friends some domain specific languages. Any of this look familiar?

chinese near 5503 roosevelt way, seattle

on google maps. Or

dentist appointment in seattle from 4pm to 5pm

on google calendar.

What is this world coming to? And more importantly, if “dentist appointment from 4pm to 5pm” is a DSL, what about the interface that lets you select an appointment over that same time period by dragging and dropping? Where is the line between DSL and UI? Or is it starting to blur a little?

The above is source code. It is a dsl, kind of, but it’s a particular kind of dsl that is trying to look and feel like natural language. It is completely readable by a non-programmer, and I dare say it’s writeable by one too. It is Inform 7 , one of the coolest things I have seen in a while. (see Beyond Domain Specific Languages )

Put this together with the simple little DSLs that google is doing or the internal natural language-ish DSLs that my colleague Jay Fields has been working on in ruby, and I think we are beginning to see people approach programming in a new way.

This is definitely stuff that we’ll talk about in our upcoming DSLs in Ruby book. But more importantly, it might be stuff that I use on my next project!
——
Above and beyond the language, there is also a sweet IDE for Inform 7. We’ll be looking at what we can learn from it here at Intentional. You should too.

It has a simple UI, is targeted toward non programmers, and is very polished and well thought out. Good stuff.

I’ve been pretty quite lately.

Well I’ve been busy, stuff I’ve been working on that I’ll hopefully be posting more about include :

• Playing with Jetbrains MPS – it’s their LanguageWorkbench, and it’s nice, very nice. If you’re totally new to the whole space, you’ll probably find it a bit frustrating, as it’s still very early days of the EAP and it’s a bit rough. But, for me, I’m coming to it from someone who’s wrestled w/ a lot of the problems that MPS has already solved, and it’s refreshing to see their solutions. More Later…
• Playing with JackRabbit – this is a java content repository. Something like this is the perfect foundation for a language workbench to be built on top of. More Later…
• Starting AddressBook – this is a .net port of OSX’s Address Book. I’m pretty close to releasing version 0.1, so next couple days you should see a download available here. I’m already using it for my addresses, and it’s so much better than outlook… More Later…
• Intentional in Budapest – I’m in Europe right now, working with the Intentional team in Budapest after taking 2 weeks vacation in Sweden. I know, rough life.

Leading a band——
Tamar, one of my good friends and a singer was telling me a story about the first time she led a band. She was singing Route 66, and after she got up on stage, she said:

Route 66 in E Flat
Drummer, I want a big boom after "get your kicks"
We're gonna run through it once, mess around,
come back in on the bridge and I wanna tag it on the end.

a one, a two, a one-two-three-four...

In 15 seconds, she was able to communicate to them (a band she’d never sang with) exactly how she wanted them to perform the song. It would have taken her half an hour to communicate the same thing to me, but she and they spoke the same language. It was a Domain Specific Language or DSL.
——
Examples in Software——
In software, you might have different domain specific languages, but the concept is the same. They are able to provide more concise, precise and quickly understandable representations within a domain.

Let’s look at a couple examples:

NAnt

This is a domain specific language implemented in XML that many java and c# developers are familiar with. NAnt is used to build .net projects, below you’ll see a simple build file. Notice how all the first level constructs have to do 100% with the domain of building an application.

<project name="MyProject" default="build">
  <target name="clean">
    <delete failonerror="false">
      <fileset>
        <include name="build/**"/>
        <include name="**/bin/**"/>
        <include name="**/obj/**"/>
      </fileset>
    </delete>
  </target>
 
  <target name="compile">
    <solution solutionfile="MyProject.sln" configuration="Debug" outputdir="build"/>
  </target>
 
  <target name="test" depends="compile, refresh-books">
    <exec program="../tools/NUnit/TestRunner.exe">
      <arg value="MyProject.Tests.dll" />
    </exec>
  </target>
 
  <target name="build" depends="compile, test"/>
</project>

Active Records in Ruby on Rails

There is a lot of buzz right now around Ruby on Rails. One of the things that Rails does incredibly elegantly is it’s OR (object/relational) mapping.

 class Recipe

This code just mapped the Recipe object to the recipes table in the database and whatever fields it has. It associates every recipe with a category, and every category with a list of recipes.

Again, the common pattern is that there is not a lot of code here that is not specific to the domain. Domain specific relationships like “belongs_to” and “has_many” are represented by top level constructs.
——
Summary——
In trying to explain Language Workbenches and “intentional programming”, I often talk about the value of DSL’s, but it’s sometimes hard to come up with examples that someone will connect with. Here I gave a totally non-technical example, an xml based example and a ruby based example.

What examples of other types of Domain Specific Lanaguages can you think of?

For the last 3 months that I’ve been back at ThoughtWorks, I’ve been working with Charles Simonyi at Intentional Software on their Intentional Editor. It is one of an emerging set of tools that Martin Fowler is about to brand “Language Workbenches”.

It’s pretty amazing stuff. I am somewhat limited as to what I can say about the specifics of their editor, but I want to start talking about the field of Language Workbenches in general.

I’m still figuring out what I want to talk about, separating my blog into personal / work sections (which was long overdue) is a part of it. I believe Language Workbenches are quite possibly the next big thing in software, and I think we have some interesting road ahead of us.

In the near future expect posts on things like DSL’s (Domain Specific Languages), or what a next generation IDE might look like.

And if you want to know more, definitely stay current w/ http://blog.intentsoft.com/ and http://martinfowler.com/bliki