One Man’s Walk in work

I think I blew Steve Conover’s mind last week.

He, like me, and every other sane rational person I’ve ever met, thinks that when you say “line up” or “line down” in a text editor, it should take the line(s) you are currently on and move them up, in their entirety. Furthermore, that if you press “line up” 5 times, the line(s) you are currently on should move up 5 lines.

Textmate, as much as I love it, does not do this, and this has long been a source of contention and strife among my colleagues at Pivotal.

This, however, is a story with a happy ending. Last week, Steve and I were pairing on Textmate, and he was complaining that line up / down wasn’t working properly. A couple hours later, I had a brilliant insight. This is how you fix Textmate’s line up / down :

Pull up textmate and type :

command-option-M
command-shift-L
command-control-up arrow
command-option-M
control-option-m

This will start recording a macro, select the current line, move the current line up, stop recording the macro, and then save the macro.

Next, name your macro “line up” in the dialog that just popped up, and give it a key equivalent of command-control-up arrow.

Congratulations, you just fixed line up!

Now let’s do the same with line down :

command-option-M
command-shift-L
command-control-down arrow
command-option-M
control-option-m

This time, name it “line down”, and give it a key equivalent of command-control-down arrow.

You of course don’t have to do all this with keyboard shortcuts, but it’s really impressive if you do

So we’re working with Amazon’s SimpleDB. It’s pretty sweet, though the ruby libraries for it are still a bit primitive. One of the problems you run up against when you’re writing integration tests against it is eventual consistency.

Take this test :

it "should save" do
  customer = Customer.create!(:name => 'bob', :email => 'bob@example.com')
  customer = Customer.find(customer.key)
  customer.name.should == 'bob'
end

The way SimpleDB works, you’re assured that Customer.find will work…eventually, but not right away.

For a couple days we contented ourselves to just run the integration tests a couple times until they didn’t error out. But that got old.

Enter “eventually” :

it "should save" do
  customer = Customer.create!(:name => 'bob', :email => 'bob@example.com')
  customer = eventually { Customer.find(customer.key) }
  customer.name.should == 'bob'
end

It’s a very simple method (below) that just retries the passed in block until it succeeds, timing out after 10 tries. Super simple, works like a charm. Thank you ruby.

Here’s the source :

def eventually(tries = 0, &block)
  yield
rescue
  raise if tries >= 10
  sleep 0.5
  eventually(tries + 1, &block)
end

I will say, I can’t help but smile every time I write “eventually” in a test…

If you’re like me, you value traveling light. If you’re like me, anything that can live in digital form does. I haven’t seen most of the actual DVD’s I own for quite some time now. But the movies on them all live on a hard drive that’s hooked up to my television. I love the instant gratification. I love that I can download a movie to my laptop and watch it on the plane.

Delicious Library makes this even cooler. As much as I love having everything digital, I miss being able to take a quick glance at 100 covers and recognize the one I want. I miss the association between images and memories. Delicious gives this back to me.

However, it’d be really cool if I could go browsing in my delicious “library”, find the movie I want, and double click it to play. Turns out you can easily extend the program with applescript, and a couple hours after I decided such a script didn’t already exist on the interweb, it now does.

This applescript will open vlc with the file associated to a media object in delicious. If there isn’t one already, it will prompt you for this file and then remember it for next time. It’s really simple, and happy

-- this will play the currently selected file in vlc using either 
-- the associated url or prompting you to find the file on disk
 
tell first document of application "Delicious Library 2"
    set selectedMedia to selected media
    repeat with selectedMedium in selectedMedia
        set movieUrl to associated URL of selectedMedium
 
        if movieUrl starts with "file://" then
            set movieFile to rich text 8 through end of movieUrl
        else
            set movieFile to (choose file with prompt "Find the file for " \
                                            & name of selectedMedium & ":")
 
            set associated URL of selectedMedium to "file://" & movieFile
        end if
 
        tell application "VLC"
            activate
            open movieFile
            fullscreen
        end tell
    end repeat
end tell

Who would have thought I’d be doing php?

Anyway, yesterday I found myself doing php and really wanting it to hurt less. I downloaded a textmate bundle for phpunit from github. It wasn’t quite what I wanted, so I forked it and hacked it, and made it work a bit more like the rspec bundle.

It now has:

Commands

• run tests (in current file)
• run single test

Snippets

• test

And much to my surprise, it was way easy to do. All in ruby, all with specs testing it.

This article was going to be telling all y’all to come over to my cool fork of phpunit, but I sent the original author a pull request yesterday, and I just looked, and he’s pulled all my changes. Rock!

So go forth and enjoy the goodness that an extensible editor like textmate, oss code like the phpunit bundle, and github can bring about.

PHPUnit.bundle for textmate

Wouldn’t it be nice if when an error happened in your application, you could not only see the stack trace, but click on a line and jump to the offending code? This is not groundbreaking stuff, I know, I had this like 10 years ago in C++ and later Java fat clients, and I’m sure other languages & IDEs had it too – but somehow in moving to writing web apps in Ruby, I lost it.

I want it back damnit!

Turns out it’s pretty easy to get back (at least it is if you use textmate) – check it out.

Custom Error Page

First, to get a custom error page for your project, add something like this to your application.rb :

def rescue_action_locally(*args)
  render :template  => "application/public_error", :layout => false
end
 
alias rescue_action_in_public render_action_locally

Note, if you’re using exception notifiable, you probably want to change the last line to something like :

alias render_404 rescue_action_locally
alias render_500 rescue_action_locally

We use markaby, so our public_error template looks something like this; it’s probably a good idea to keep this simple and not use a layout, just in case the error came from the layout :

html do
  head do
    title action_name
    stylesheet_link_tag 'error'
  end
  body do
    div.error do
      div.message do
        h1 "Whoops"
 
        p "We detected an error.  Don't worry, though, 
we've been notified and we're on it."
      end
    end
  end
end

Adding a Stack Trace w/ Links to the Error Page

So, it would be helpful to us for our error page to tell us more in our development and staging environments. We do use exception notifiable, so we don’t actually need or want it to say anything else to a real user in production. Adding this to our template, it now looks like this :

html do
  head do
    title action_name
    stylesheet_link_tag 'error'
  end
  body do
    div.error do
      div.message do
        h1 "Whoops"
 
        p "We detected an error.  Don't worry, though, 
we've been notified and we're on it."
      end
 
      if RAILS_ENV != 'production'
        div.stack_trace do
          h2 "Stack Trace"
          div { link_to_code $!.to_s.to_s.gsub("\n", "
") }
          hr
          div { link_to_code $!.backtrace.join("
") }
        end
      end
    end
  end
end

What’s that “link_to_code” method in there?

It’s a method in application_helper that replaces any path with a textmate url to open up that file on your local system and jump to the offending line. Check it out :

def link_to_code(text)
  text.gsub(/([\w\.-]*\/[\w\/\.-]+)\:(\d+)/) do |match|
    file = $1.starts_with?("/") ? $1 : File.join(RAILS_ROOT, $1)
    link_to match, "txmt://open?url=file://#{file}&line=#{$2}"
  end
end

That’s it. Suddenly, stack traces are friendly again!

So, following rSpec’s example, we’re moving from JIRA to Lighthouse. Check us out at http://cruisecontrolrb.lighthouseapp.com/

I have to say I wasn’t thrilled about Jira. It does everything, and it does it slowly, with a UI that is bloated at best. Lighthouse, on the other hand, keeps it REALLY simple. And does not a lot more than what we need. It also just opened itself up for OSS projects (which is cool).

How did we migrate all of our stories from JIRA to Lighthouse? I thought you’d never ask.

Lighthouse exposes a slick RESTful API for managing your data. I exported the JIRA issues into a csv file, wrote a short ruby script, waited about 5 minutes for it to run, and I was done.

Here’s the script:

#!/usr/bin/env ruby
require 'rubygems'
require 'csv'
 
JiraIssue = Struct.new(:issue_type, :key, :status, :assign_to, 
                       :summary, :description, :fix_version, 
                       :reporter, :priority, :original_estimate, 
                       :votes)
 
def load_issues(csv_file)
  header = true
  issues = []
  CSV.open(csv_file, 'r') do |row|
    if header
      header = false
    else
      issues << JiraIssue.new(*row)
    end
  end
  issues
end
 
issues = load_issues(File.dirname(__FILE__) + "/jiraissues.csv")
 
# this code is just to examine what our JIRA data looks like
def show(issues, field)
  puts "#{field} = #{issues.map{|i| i.send(field)}.uniq.inspect}"
end
 
show issues, :issue_type
show issues, :status
show issues, :assign_to
show issues, :fix_version
show issues, :reporter
show issues, :priority
show issues, :original_estimate
show issues, :votes
 
# these hashes translate between JIRA and Lighthouse
states = {
  "Open" => 'open',
  "Resolved" => 'resolved',
  "Closed" => 'invalid'
}
 
users = {
  "Unassigned" => nil,
  "Alexey Verkhovsky" => 10545, 
  "Rolf Russell" => 17992, 
  "Jeremy Stell-Smith" => 10638, 
  "Zhang Lin" => nil, 
  "Jeff Xiong" => nil
}
 
milestones = {
  "1" => 9885,
  "1.1" => 9886, 
  "1.2" => 9887, 
  "1.3" => 9888, 
  "1.4" => 9889, 
  "\302\240 " => nil, 
}
 
require File.dirname(__FILE__) + "/lib/lighthouse"
include Lighthouse
 
Lighthouse.account = "cruisecontrolrb"
# enter your own user, password here...
Lighthouse.authenticate "jeremystellsmith@gmail.com", "*******" 
 
issues.each do |i|
  # enter your own project id here...
  ticket = Ticket.new :project_id => 9150
 
  ticket.title = i.summary
  ticket.body = i.description
  ticket.state = states[i.status.to_s.strip]
  ticket.assigned_user_id = users[i.assign_to.to_s.strip]
  ticket.milestone_id = milestones[i.fix_version]
  ticket.tags

There was another step. I ran the first part of the script a couple times to tell me what the different issue_types, statuses, assigned users, fix versions, etc were that my JIRA data was using. I then found or created the appropriate data in Lighthouse and added some hashes to translate for me.

All in all, I’m pretty thrilled with how easy this was.

Rails 2.0.1 is out, in general, it’s pretty nice. Markaby doesn’t play too nice w/ it. Markaby requires every named route to have a .to_s on the end of it. Yuck.

After migrating my 3rd project to rails 2.0.1, I got sick of adding .to_s to each named routes, and did some digging.

Apparently, somehow a string coming from a named route is not exactly a string. Sure

  apples_path().class == String

but

  String === apples_path()

is NOT true. I couldn’t quite track down why this was but it eventually leads to a

ActionView::TemplateError (undefined method `string_path' for #) on line #10 of messages/index.mab:

actionpack-2.0.1/lib/action_controller/polymorphic_routes.rb:27:in `send!'
actionpack-2.0.1/lib/action_controller/polymorphic_routes.rb:27:in `polymorphic_url'
actionpack-2.0.1/lib/action_controller/polymorphic_routes.rb:31:in `polymorphic_path'
actionpack-2.0.1/lib/action_view/helpers/url_helper.rb:79:in `url_for'
actionpack-2.0.1/lib/action_view/helpers/url_helper.rb:144:in `link_to'
...

After an hour of trying in vain to find the source of the problem, I found a simple hack that fixes this. Should have thought of this before, but add this code to your application_helper.rb

  def string_path(string)
    string
  end

If anyone else has a better solution, please pass it over!

So I’m a big fan of iWork ‘08, pages is nice enough, doesn’t get in my way, etc. Numbers is pretty awesome. It’s been a while since someone did something revolutionary in the spreadsheet world. Numbers is it.

I do have a couple gripes.

1) if I open a word or excel file, when I save, it should be saved back to that same word or excel file. Not all my friends have macs, let alone iWork, why can’t I save back to the file I opened? And if I have to do the extra export step, why does iWork forget where the file came from? Grr.

2) I am a total excel power user, and numbers is still missing a bunch of features for addins, macros, etc. It doesn’t even have applescript support yet as far as I can tell (I’m sure it will, though, and I am glad they released before they had it) I’ve been getting by on a manual export to csv so I can munge data.

3) and what this blog post is about, is that iWork does not play nice w/ subversion.

iWork stores it’s files in “bundles” which are actually directories even though they look like files from finder. When you have stored one of these bundles in subversion, there will be .svn directories inside of it, containing subversion metadata. The problem comes that every time you save a file in iWork, the entire bundle is replaced w/ the new copy…and ALL .svn directories are destroyed.

This basically renders subversion useless for iWork files. This is no good, there are some pretty painful workarounds on the net, and there are a few scripts that turned up, but I wrote my own.

This script (I call it svn_restore_dirs) will search any subdirectories of the current working directory for files ending in .numbers, .pages, etc. It will then redownload .svn directories for any of these that are missing theirs.

Seems to work, and is simple enough. I wish apple would get their act together.

Enjoy the script

#!/usr/bin/env ruby
 
require 'fileutils'
 
EXTENSIONS_TO_SEARCH = %w(numbers pages key graffle)
TMP_DIR = "/tmp/missing_svn"
 
def find_directories_without_svn_dirs
  search_pattern = EXTENSIONS_TO_SEARCH.map {|ext| "-name \"*.#{ext}\""}.join(" -or ")
  dirs = `find . \\( #{search_pattern} \\) -type d`.split("\n")
  dirs.reject {|dir| File.exists?(File.join(dir, ".svn"))}
end  
 
def get_url_for(file)
  info = `svn info "#{file}"`
  raise "can't parse #{info}" unless info =~ /URL\: (.*)/
  return $1
end
 
def copy_svn_dirs(from, to)
  to = File.expand_path(to)
  Dir.chdir from do
    Dir["**/.svn"].each do |svn_dir|
      FileUtils.mv svn_dir, File.join(to, svn_dir)
    end
  end
end
 
find_directories_without_svn_dirs.each do |dir|
  url = get_url_for(File.dirname(dir))
 
  puts "replacing svn dirs for #{dir} from #{url}/#{File.basename(dir)}"
 
  FileUtils.rm_rf(TMP_DIR) if File.exists?(TMP_DIR)
  `svn co "#{url}/#{File.basename(dir)}" #{TMP_DIR}`
 
  copy_svn_dirs(TMP_DIR, dir) if $? == 0
end

I must have written various versions of this code 20 times so far in my career, but never so quickly and cleanly. I love ruby.

    min, sec = sec / 60, sec % 60
    hour, min = min / 60, min % 60
    day, hour = hour / 24, hour % 24
    week, day = day / 7, day % 7
 
    [
      week > 0 ? "#{week} weeks" : nil,
      day > 0 ? "#{day} days" : nil,
      hour > 0 ? "#{hour} hours" : nil,
      min > 0 ? "#{min} minutes" : nil,
      sec > 0 ? "#{sec} seconds" : nil
    ].compact.join(", ")

The really cool thing is the multiple assignment that ruby lets you do. I haven’t used it for something exactly like this before, but I do use it often and it’s really nice.

I feel like there should be a better way to do the bottom half of this, but this is pretty damn readable, I think. You might have to have a little rubyFU to remember what compact does – which is get rid of the nils.

Anyway, this was about 15 minutes of work (and I did it test first).

It’s very easy to call out to methods in markaby, but it’d be nice if you could actually customize the dsl as well.

For example, on many of our pages we have a bottom row that has buttons that look a certain way. So on every page, we have :

  table(:width => "100%") do
    tr do
      td.left do
        previous_button
        first_button
      end
      td.center do
        print
      end
      td.right do
        next_button
        last_button
      end
    end
  end

The code for the actual buttons changes, and after a few tries to extract the whole thing into a single method, we gave up. Our efforts had made it harder to read, not easier. There was always just a little too much variance, and it didn’t feel right.

What we really wanted to write was :

  last_row do
    column do
      previous_button
      first_button
    end
    column do
      print
    end
    column do
      next_button
      last_button
    end
  end

This lets the buttons that change all stay in the view, and gets rid of the skeleton and positional stuff that doesn’t change. Furthermore, it’s DRY and puts all that positional logic in one place instead of scattered across 20 views.

How to do this?

I wrote a test (in RSpec) that looks something like :

describe ApplicationHelper do
  it "should generate a table from a buttons method" do
    last_row(:columns => 2) do
      column do
        "foo"
      end
      column do
        "bar"
      end
    end.should == '<table width="100%"><tr>' +
                    '<td class="left">foo</td>' + 
                    '<td class="right">bar</td>' +
                  '</tr></table>'
  end
end

After a bunch of fiddling and poking around, I finally made the test (and a couple others) pass with this code in my ApplicationHelper :

def last_row(options, &block)
  markaby do
    table(:width => "100%") do
      tr do
        LastRowContext.new(self, options[:columns]).
                                  instance_eval(&block)
      end
    end
  end
end
 
class LastRowContext
  def initialize(markaby, columns)
    @markaby, @column_count, @column_index = 
                            markaby, columns, 0
  end
 
  def column(&block)
    alignment = case @column_index += 1
    when 1 : :left
    when @column_count : :right
    else :center
    end
 
    @markaby.instance_eval do
      td(:class => alignment, &block)
    end
  end
end

I’m sure this could get cleaned up more; this was the work of less than an hour. In particular, if you did this often, you could extract a common MarkabyContext superclass that had some convenience methods. The point is, this is really easy to do, and we shouldn’t be scared to try “Language level refactorings” like this.