out of time (mat brown on programming)

ruote

Mon, 06 Jul 2009 17:21:31 -0400

ruote:

Workflow engine written in Ruby

redis is a key-value data store

Mon, 15 Jun 2009 15:50:13 -0400

redis is a key-value data store

Sunspot 0.8 is out

Tue, 26 May 2009 17:58:00 -0400

On Friday, I released the next milestone in Sunspot, version 0.8. This version doesn’t add to or change any of the basic functionality, but does add some advanced features which the app I work on for my day job happens to demand. Here’s a rundown:

Direct access to the Query API

Users of Sunspot will doubless be familiar with Sunspot’s search DSL, which gives an English-like interface for constructing search parameters. In some cases, however, such a DSL is actually counterproductive, particularly when searches are being built by an intermediate object, and thus not necessarily all in one place. So, the new methods Sunspot.new_search() and Search#query() are exposed, and the Sunspot::Query class itself is now part of the public API. What I have in mind in particular here is an application of the Go4 Builder pattern, along with ActiveRecord’s hash-initializer pattern, to elegantly translate web query parameters into a Sunspot search. Here’s a stripped-down example of what I think the code will look like to do that:

class EventSearchBuilder
  attr_reader :search


  def initialize(options = {})
    @search = Sunspot.new_search(Event)
    options.each_pair do |attr, value|
      if respond_to?("#{attr}=")
        send("#{attr}=", value)
      end
    end
  end


  def when=(day_string)
    case day_string
    when 'future'
      @search.query.add_restriction(:start_time, :greater_than, Time.now)
    when 'past'
      @search.query.add_restriction(:start_time, :less_than, Time.now)
    else
      date_time = Date.parse(day_string).to_time
      @search.query.add_restriction(:start_time, :between, date_time..(date_time + 1.day))
    end
  end


  def page=(page)
    @search.query.paginate(page)
  end


  def sort=(field)
    @search.query.order_by(field)
  end
end

Then in controller code, it’s as simple as:

def search
  @search = EventSearchBuilder.new(params).search
  @search.execute!
end

Dynamic Fields

I wouldn’t be surprised if I’m the only person who ever uses this feature of Sunspot, but just in case, let’s look at a real-world example. Let’s say part of my data model uses free-form key-value pairs, which use a constrained (but user-definable) set of keys and free-form values. I’ll call my model KeyValuePairs.

The trick I would like to pull here is that I would like to treat each key as a separate field in search, so that I can constrain, order, facet, etc. on the values for one key without them being affected by other keys. Since the keys are user-defined, I can’t just set up normal fields at build time; they need to be defined at index time. Enter Sunspot’s dynamic fields (we’ll use Sunspot::Rails’s wrapper API here):

class Business < ActiveRecord::Base
  has_many :key_value_pairs


  searchable do
    dynamic_string :key_value_pairs do
      key_value_pairs.inject({}) do |hash, pair|
        hash.merge(pair.key.to_sym => pair.value)
      end
    end
  end
end

This sets up a dynamic field which is populated using the given block. What’s important there is that the field is populated using a hash - the keys of the hash become individual dynamic fields, and the values populate those fields in the index. The “base name” of the field is key_value_pairs, which is used to namespace the dynamic names that come out of the hash.

Working with dynamic fields is a lot like working with regular ones, except in the query, calls are wrapped in a dynamic block:

Business.search do
  dynamic :key_value_pairs do
    with(:cuisine, 'Sushi')
    facet(:atmosphere)
  end
end

Naturally, those field names (:cuisine, :atmosphere) wouldn’t be hard-coded in a real application, since they would not be known at build time.

Dirty Sessions

Sessions now track whether any operations have been performed since the last time a commit was issued. The Session#dirty? method answers that question, and the Session#commit_if_dirty does exactly what it sounds like. Useful methods if you want to keep your commits to a minimum (you do) but you may have various parts of the code issuing Sunspot operations without any central knowledge on the part of your application.

That’s all for now

Sunspot 0.9 is up next; the main goal for that version is to replace solr-ruby with RSolr as the low-level Solr interface, which will open the door to more features in future versions (query-based faceting, LocalSolr support, etc.), but probably won’t have much effect on the API for that version (other than supporting use of the faster Curb library for the HTTP communication with Solr).

Installing alternate Ruby versions as optional packages

Thu, 21 May 2009 17:09:00 -0400

As a developer of Ruby libraries and applications, I’d like to make sure my code works in all of the major ruby implementations, but I’ve also got my “main” Ruby, the one that has been with me through thick and thin and happens to be the version installed on our production servers. The other Rubies need a place on my machine, but I’d like that place to be out of the way and have no chance of conflicting with my main Ruby installation or anything else I’ve got installed.

Fortunately, the omniscient beings who created the Filesystem Hierarchy Standard anticipated this need of mine, and in their wisdom created the /opt directory for this purpose. Unlike a normal package installation, which installs files in various places across your file system - /usr/bin, /usr/lib, /etc, /var, and the like - optional package installations put everything into a single subdirectory of /opt, where they’re fairly isolated from the rest of the system.

So, here’s how I installed YARV and JRuby as optional packages. This should work for anyone using Linux or Mac OS X¹:

Download the packages

Find a nice directory for downloads.

$ wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-p129.tar.gz
$ wget http://dist.codehaus.org/jruby/1.2.0/jruby-bin-1.2.0.tar.gz

Install YARV

$ sudo mkdir -pv /opt/ruby-1.9.1-p129
$ tar xzvf ruby-1.9.1-p129.tar.gz
$ cd ruby-1.9.1-p129
$ ./configure --prefix=/opt/ruby-1.9.1-p129
$ make
$ sudo make install

Install JRuby

$ sudo tar -C /opt -xzvf jruby-bin-1.2.0.tar.gz
$ sudo rm -v /opt/jruby-1.2.0/bin/*.bat

You can also remove most of the directories in the /opt/jruby-1.2.0/lib/native directory, except the one that corresponds to your architecture. If in doubt, leaving them all in won’t hurt.

Installing Gems

Assuming you’ve got RubyGems installed in your main Ruby installation, you don’t need to install it for your other installations - you can simply run the existing gem script using the various binaries, and it’ll work the way you want (installing the gems inside those optional package directories). For example:

$ sudo /opt/ruby-1.9.1-p129/bin/ruby -S gem install rake
$ sudo /opt/jruby-1.2.0/bin/jruby -S gem install rake

Using the small rubies script I covered in this post makes the process of installing gems (and doing anything else) in your various ruby versions considerably less painful.

¹If you use MacPorts, which you probably do, you’ve got a bunch of software installed in a standard hierarchy inside of the /opt/local directory. That isn’t really the way it was intended to be used, but it won’t conflict with the installations covered in this post.

A 15-line alternative to multiruby

Thu, 21 May 2009 16:27:00 -0400

Today’s mission was to get Sunspot working in all the major Ruby implementations (MRI, YARV, JRuby). I personally use MRI 1.8.6p114, and hadn’t had a need to install any other Ruby implementations, so I first tried out multiruby, which handles all of the installing and running of different Ruby versions. Alas, it didn’t work very well - after all, package management is a nontrivial problem, and multiruby does attempt to perform that function in a sense. The packages didn’t install.

I also wasn’t a big fan of installing a whole filesystem hierarchy inside a hidden directory in my home directory (some people like this — follow your own path, young jedi); and it required I install the entire ZenTest gem, which I otherwise have not found much use for.

So, I went ahead and just installed YARV and JRuby as optional packages, installed gems for them, ran the spec suite under each version, and fixed the bugs that came up. Great! But I did notice that this involved a lot of typing out full paths to the various Ruby binaries, particularly since I will want to be running the specs under all the versions from now on. This is not a difficult problem to solve, I thought to myself. Enter quick 15-line script:

#!/usr/bin/env ruby
require 'rubygems'
gem 'escape'
require 'escape'

File.open(File.join(ENV['HOME'], '.rubies')) do |file|
  file.each_line do |bin|
    bin.sub!(/\n$/, '')
    STDERR.puts("Executing in #{`#{Escape.shell_command([bin, '-v'])}`}")
    fork do
      exec(Escape.shell_command([bin].concat(ARGV)))
    end
    Process.wait
  end
end

Then I just set up my ~/.rubies file, containing full paths to my various Ruby binaries:

/usr/local/bin/ruby
/opt/ruby-1.9.1-p129/bin/ruby
/opt/jruby-1.2.0/bin/jruby

And a quick rubies -S rake runs my suites in all the relevant ruby versions. Genius? No, but it gives me all the useful functionality of multiruby with the control over installation that I crave. Hopefully it’ll help you too.

Sunspot 0.7 is out

Wed, 29 Apr 2009 12:10:00 -0400

Momentous news: Yesterday I released the 0.7 version of Sunspot, my library for awesome Solr interaction in pure Ruby. This is the first release that I consider basically feature complete, meaning there aren’t any gaping holes in the feature set - not that there aren’t more features in the pipeline! Read on for all the new goodies.

Documentation!

Sunspot is now fully documented. In order to distinguish the public and private APIs, I made liberal use of :nodoc: on classes and methods that are not part of the public API. So, everything in the RDoc is fair game; if you find yourself needing to call methods that aren’t in the RDoc, let me know so I can expose what you need in the public API.

API-private methods are still documented in the code.

Less magic in the search DSL

In earlier versions, the search DSL looked like this:

Sunspot.search(Post) do
  with.blog_id 1
  with.average_rating.less_than 4
end

Now it looks like this:

Sunspot.search(Post) do
  with :blog_id, 1
  with(:average_rating).less_than 4
end

I find the new syntax to be more intuitive, and my colleagues whom I polled unanimously agreed. Sometimes Ruby developers get perhaps a bit too excited about how cleverly one can construct English-like DSLs in the language, and I would plead guilty to that charge where the earlier version is concerned.

Negative scoping

The search DSL now provides without, which is a counterpart to the with method that negates the restriction. So, you can do the following:

Sunspot.search(Post) do
  without :blog_id, 2
end

That would exclude all posts whose blog_id is 2.

Exclusion by identity

A special use of the without method is excluding specific objects from the search results. This is done by just passing the objects you want to exclude to without. For example, if you have an instance current_post that you don’t want in the search results:

Sunspot.search(Post) do
  without current_post
end

Restrict by empty values

You can now pass nil to an equality restriction, which will restrict the results to documents for which the given field has no value. For example:

Sunspot.search(Post) do
  with :category_id, nil
end

The above would return only documents that do not have a category_id. without works as expected, returning only documents that do have a value for the field in question. Passing nil to other restriction types is not allowed.

Faceting

One of Solr’s most powerful features is faceting, which returns all of the values stored for a given field, and the number of documents that have each value. It’s perfect for building drill-down search interfaces. Sunspot now supports it:

search = Sunspot.search(Post) do
  with :blog_id, 2
  facet :category_ids
end

category_ids_facet = search.facet(:category_ids)
category_ids_facet.rows.map { |row| [row.value, row.count] }
  #=> [[25, 3], [13, 1]]

The above results indicate that there are 3 documents with blog_id 2 and category_id 25, and 1 document with blog_id 2 and category_id 13. Note that facet results are for the total number of documents that match the search conditions, not just the current result set (which is paginated). Note also that Sunspot casts facets into the appropriate Ruby object for their type - thus a time field’s facet values will be Time objects, etc.

Solr also provides even more powerful query-based faceting (to facet by ranges of values, for instance), which I plan to tackle in a future version of Sunspot.

Explicit commits

Changing data in Solr is a two-step process - first, data is added, updated, or removed; then the changes are committed. When a commit is called, all pending changes are written to disk, and Solr instantiates a new searcher object with the updated index. In order for changes to appear in search, they must be committed, but the commit is a fairly expensive operation; thus, if you are making multiple updates as part of one operation, it’s highly advisable to commit once, after making all of the changes. Earlier versions of Sunspot automatically committed after each change; now a commit method is exposed, as well as bang!-versions of the update methods, which perform a commit immediately. So:

Sunspot.index(my_document)
Sunspot.commit
# does the same thing as
Sunspot.index!(my_document)

Boolean field type

boolean is now an available field type. It works pretty much as you’d expect. Note that in order for a false value to be indexed, it has to explicitly be false — nil will not be indexed at all. Anything else is indexed as true.

Attribute field flexibility

You can now tell an attribute field to pull data from an attribute other than the one named by the field. For example:

Sunspot.setup(Post) do
  float :average_rating, :using => :ratings_average
end

This would index a field called average_rating, pulling data from the ratings_average method.

Virtual field evaluation flexibility

If the block specified for a virtual field takes an argument, the block will be passed the instance, rather than being evaluated in its context:

Sunspot.setup(Post) do
  string :sort_title do
    title.downcase
  end
  # is the same as
  string :sort_title do |post|
    post.title.downcase
  end
end

Use whichever one feels more natural for the given object - for instance, I would use the first form for a model I had written, but the second form for File objects.

Order by multiple fields

You can now call order inside the search DSL more than once. Earlier calls get higher precedence.

New adapter API

Sunspot is intended to be flexible in what it can index and search; to that end, it provides a pluggable adapter architecture. Sunspot 0.7 makes several changes to the adapter API; this should be the final API that goes into the 1.0 release.

Instead of building a single adapter module that contains two classes with preset names, the new API allows (and indeed requires) the two classes to be registered seperately. An adapter should consist of two classes: a subclass of Sunspot::Adapters::InstanceAdapter, and a subclass of Sunspot::Adapters::DataAccessor. Check out the Rdoc for information on what methods each should and can implement. Here’s an example of how one might build an adapter for File objects:

class FileInstanceAdapter < Sunspot::Adapters::InstanceAdapter
  def id
    File.expand_path(@instance.path)
  end
end

class FileDataAccessor < Sunspot::Adapters::DataAccessor
  def load(id)
    @clazz.open(id)
  end
end

Sunspot::Adapters::InstanceAdapter.register(FileInstanceAdapter, File)
Sunspot::Adapters::DataAccessor.register(FileDataAccessor, File)

Goodbye Builder API

The last release of Sunspot attempted to provide a framework for using the Builder pattern to convert external parameters into searches. Upon further reflection, I found the API and implementation rather awkward, and it wasn’t really a core part of what Sunspot was trying to do. The search method still can accept a hash of parameters, but you really shouldn’t use that because the DSL is way better.

This also means that, for the moment, Search objects don’t provide access to the parameters passed in. That’s probably not a good thing, so I’ll try and find a clean, intuitive way to provide that access in a new version.

Goodbye extlib

Extlib is cool, but Sunspot was only using three methods from it, and it was causing some sort of weird error when I tried to run the tests on the installed gem. So I implemented the three methods myself and got rid of the extlib dependency.

That’s all, folks

That should just about cover all of the external-facing changes in the new version. There’s also lots of internal refactoring and simplification that should make the code leaner, faster, and more maintainable. This is the first version that I would feel comfortable putting into a production environment - if you’re using it, I’d love to know.

RabbitMQ and Ruby

Thu, 09 Apr 2009 14:58:10 -0400

RabbitMQ and Ruby:

The day when we ditch Starling for RabbitMQ will be a good day.

Git Tip of the Day: git cherry-tree

Thu, 09 Apr 2009 11:20:31 -0400

The problem

Recently I came across a situation in which I needed to make some fairly major changes to our codebase, and those changes needed to apply to two different branches - both a version branch that’s currently in QA, and the master branch. The changes involved reverting a few commits as well as making new changes that were too big to comfortably fit into one commit. One option would be to simply make all of the commits in one branch, and then cherry-pick them by hand into the other, but that seemed awfully manual for such a powerful tool as git.

The Solution

Enter git cherry-tree, a little alias I came up with that basically creates a series of patches based on the commit diff between two branches, and pipes the result directly into a third branch. Before I explain how to use it, here’s how to add the alias to your config:

git config --global alias.cherry-tree \!sh\ -c\ \'git-format-patch\ --stdout\ \$0..\$1\ \|\ \git\ am\ -3\'

How to use it

For the sake of this example, we’ll call our branches qa and master. My goal is to branch qa, make a series of commits, and then apply the changes in those commits to master as well. The result should be equivalent to individually cherry-picking the commits that I’ve made, but easier and more reliable. Here we go:

git checkout qa
git checkout -b qa-big-changes
# make and commit the changes
git checkout master
git checkout -b master-big-changes
git cherry-tree qa qa-big-changes

That last command says, essentially, “Sequentially apply each commit that is in qa-big-changes, but not qa, to the current branch.” Unless you’re a naturally lucky person, you’ll probably have conflicts - when this happens, the process will stop midstream, telling you which files are in conflict. Let’s say config/environment.rb is in conflict:

# open config/environment.rb and fix the conflict
git add config/environment.rb
git am -3 --resolved

Note that, unlike with a normal merge conflict, you don’t want to commit after fixing the conflicts - just add the conflicted files to the index. The last command just says, “OK, problem solved, pick up where you left off.” Note also that, since each commit from your other branch is applied individually, this can happen more than once (of course, with different conflicts, in different commits).

Once the entire patch has run cleanly, master-big-changes will have a series of commits equivalent to the commits that you made to qa-big-changes (they won’t be the same commits, though - just like a cherry-pick). Then you can merge your changes into the respective branches:

git checkout qa
git merge qa-big-changes
git branch -d qa-big-changes
git checkout master
git merge master-big-changes
git branch -d master-big-changes

As a final note, the use of master-big-changes isn’t strictly necessary - you can just do it directly in master. I just feel safer doing this process to a separate branch and then merging it in. However, making your original set of changes in qa-big-changes, rather than directly in qa, is required for this process.

ShellElf runs shell commands from a Starling queue

Thu, 02 Apr 2009 16:44:19 -0400

ShellElf runs shell commands from a Starling queue:

ShellElf is a small daemon that reads shell commands out of a Starling queue and runs them. Great if you need to do non-trivial processing tasks in the background. It’s lightweight but has a few neat features:

Post back to a specified HTTP service on success/failure for each command batch
Exit gracefully on TERM/INT - either finish the current batch, or reenqueue it, before exiting.
Retry Starling connection indefinitely if unable to connect on the first try - never exit silently.

Check it out.

rake db:rollback_to_common

Thu, 26 Mar 2009 10:28:00 -0400

rake db:rollback_to_common:

If you find yourself having to switch locally between different git branches that have different database schemata, (for instance, a master branch and a production branch), one of the biggest hassles is keeping the database in the right state. We wrote this rake task to make the process painless: just specify the branch you’re about to switch to, and it will automatically roll back to the last common database version between that and your current branch. Usage:

rake db:rollback_to_common branch=production
git checkout -m production

It will raise an exception if nothing needs to be done, which doesn’t really bother me, but if it bothers you, feel free to edit - it’s a gist.

Using Git Submodules for Shared Rails Components

Wed, 25 Mar 2009 14:38:00 -0400

Using Git Submodules for Shared Rails Components:

I’ve been thinking for a while about how nice it would be to share the model layer of our Rails app with other codebases that don’t have to load up all of Rails. In particular, this would neatly allow one to avoid using Workling (which essentially runs the entire Rails stack as a Daemon), instead building lightweight queue-reading daemon processes that still have access to the model layer (since this is, in pretty much any case I can think of, the only part of the Rails stack that a background-job processor would need access to).

Where are the REST frameworks?

Mon, 23 Mar 2009 14:28:00 -0400

Where are the REST frameworks?:

I’ve been thinking a lot about REST lately - what it means for Rails developers, and what it means in the broader scheme of things (the two are scarcely related). This article is an interesting addition to the discussion, and I’m especially intrigued by the treatment of hyperlinking and its relationship with the traditional Rails approach of polymorphic associations.

Using Ruby's Autoload Method To Configure Your App Just-in-Time

Thu, 19 Mar 2009 13:19:07 -0400

Using Ruby's Autoload Method To Configure Your App Just-in-Time:

Good primer on Ruby’s autoload keyword, which makes it easy to only load pieces of a library or application when you need them

"Debugging 2x as hard as writing code. If you write as cleverly as possible, you are, by definition,..."

Mon, 16 Mar 2009 13:44:53 -0400

“Debugging 2x as hard as writing code. If you write as cleverly as possible, you are, by definition, not smart enough to debug it.”

- wycats

blowmage: anyone remember seeing a screenshot of _why's computer about a year ago? he was using a tiled window manager. can't find the url...

Tue, 10 Mar 2009 19:29:23 -0400

blowmage: anyone remember seeing a screenshot of _why's computer about a year ago? he was using a tiled window manager. can't find the url...
JamesKilton: ion?
JamesKilton: wmii?
Mikoangelo: _why uses a window manager?
Mikoangelo: I thought he just used eight switches and an enter key

Three changes I'd like to see to DataMapper before 1.0

Fri, 27 Feb 2009 11:51:41 -0500

DataMapper is an excellent Ruby ORM, brought to you by (roughly) the same rock stars who built Merb and are now working on building the first production-ready version of Rails. DataMapper boasts some obvious advantages over ActiveRecord - a small core with a gem-based plugin framework, an object store (to enforce one-object-per-row within a session), and a very clever strategy for implicitly avoiding n+1 query inefficiency. However, there are a few places where DM doesn’t shine - most of them would be easy enough to fix, but would involve changes in the API, so pre-1.0’s the time to do it. Here’s my wish list:

1. Get rid of `has 1` / `has n` for declaring associations

ActiveRecord defines the class methods has_one and has_many for declaring 1:1 and 1:n assocations respectively. This works fine. DataMapper, on the other hand, defines has, which is written like this:

has 1, :user
has n, :users

There are a couple of gripes I have here. First, the has method is doing completely different things depending on the first argument passed. When a method’s logic works like that, it’s usually a candidate to be two separate methods. It reminds me of ActiveRecord::Base#find, a method which does three different things, depending on the first argument passed. Mercifully, DataMapper defines a different method for each of those three things - #get, #first, and #all, respectively. They shouldn’t then go and make the same mistake with the ::has method.

Second gripe: that n there is major code smell. As it turns out, the n class method on DataMapper::Resource returns the Float value Infinity. That does make a sort of semantic sense, but declaring this class method just to provide some minor syntactic sugar for one declaration is a Ruby antipattern. This seems to me to be a case of someone getting a little too excited about how English-like Ruby can look.

My last, minor, gripe is that has_one :user is slightly more readable than has 1, :user. That comma ruins it for me a little.

2. Don’t extend Symbol for non-general use

ActiveRecord’s ability to construct SQL queries using Ruby is woefully limited - if you want to test for any conditions other than equality or IS NULL, you’ve got to write SQL into your conditions (which, as far as I’m concerned, qualifies as string programming and is thus a Bad Thing). That’s clearly a weakness - most ORMs I’ve come across have provided a full criteria API allowing the construction of complex, robust queries in the ORM’s language. DataMapper doesn’t have that, but it does provide what at first seems to be an improvement on the situation in ActiveRecord. An example from DM’s API documentation:

Person.all(:age.gt => 30)

Oh, hey, we can query for conditions other than equality without writing SQL! Here’s the problem: DataMapper defines the methods Symbol#gt and its friends in order to do this. Those methods on Symbol, a core class, have absolutely no meaning outside of that one use inside the DataMapper conditions, but they’re defined for every Symbol in the object space, regardless of its use, once you include dm-core. That’s stinky. Ruby is almost unparalleled in the ease it provides for writing beautiful DSLs - someone should take advantage of that for writing criteria APIs for Ruby’s main ORMs.

3. Take migrations seriously

One of the most exciting features brought to the table by Rails is migrations. While I’m sure the Rails team didn’t invent the idea of database migrations, Rails introduced me to the idea; I suspect that’s true for many people. Migrations act like version control for your database; they work more or less as a collection of diffs, just like a revision history in git. The fundamental idea behind migrations is that any given version of your codebase corresponds to a particular database schema; by keeping migrations in a set of files inside version control, you guarantee that any time a new revision of the codebase is checked out, the database can be updated to correspond to it. This is absolutely crucial for production applications that are backed by databases.

However, the DataMapper team seems to be ambivalent about the idea of migrations. DataMapper provides three mechanisms for making modifications to your database schema: automigrate, which simply blows away your database tables and recreates them from the property information declared in your model; autoupgrade, which attempts to modify existing tables to match the property information declared in your model; and regular migrations, which work like their counterpart in ActiveRecord. automigrate is a nice toy for playing around in development, but is obviously useless in production applications, since you can’t just blow away your production data every time you want to modify your schema. autoupgrade is a nice idea but can’t possibly cover all of the possible changes you want to make. dm-migrations, a plugin in dm-more that provides equivalent functionality to ActiveRecord’s migrations, with a couple of nice enhancements - in particular, uniqueness based on name, rather than index/timestamp; and a nicer-looking DSL than what ActiveRecord does. The problem with dm-migrations is that it’s so feature-poor it’s practically unusable. You can’t even set column lengths.

In an ideal world, we’d have a nice standalone migrations tool for Ruby - after all, migrations are about schema definition in relational databases, and they’re conceptually distinct from your ORM. Indeed, DataMapper is flexible enough to work with schema-free databases; the idea of a schema isn’t even inherint in DM’s architecture. There’s really no reason that migrations should be bundled with it. In the short term, though, dm-migrations is there; it’s just not where it should be. Before 1.0, I hope the DataMapper team gives migrations the attention they deserve.

I’m entirely convinced that before long, DataMapper will be the de facto Ruby persistence layer. It’s fast, well-engineered, and feature-rich. Here’s hoping the DM team is able to erase these few blemishes from what is otherwise an excellent library.

MongoDB

Wed, 25 Feb 2009 15:46:47 -0500

MongoDB:

MongoDB is an object-oriented database, providing yet another alternative to the old relational database

Google Data on Rails

Tue, 24 Feb 2009 10:44:11 -0500

Google Data on Rails:

Ruby API for Google Data

Why you see hundreds of "SHOW TABLES" queries when running ruby-prof in Rails

Wed, 18 Feb 2009 17:41:00 -0500

As part of my continuning quest to make ruby-prof a useful part of my optimization toolkit, I decided to finally investigate why, when running the profiler, I was seeing the SQL query “SHOW TABLES” output over and over, hundreds of times, in the log. And it was super-slow - over a minute to finish a request that otherwise clocked in around a second.

After digging around in ruby-prof and ActiveRecord, I discovered the following interesting facts:

For each method call that it logs, ruby-prof calls #inspect() on the receiving class. It does this from within its C layer, so it’s not immediately obvious, but the call is received in the Ruby layer.
When you call #inspect() on a class that extends ActiveRecord::Base, it queries the database with “SHOW TABLES”
Every time.
Seriously.

This happens when you instantiate one of RubyProf’s Printer instances, so it doesn’t actually affect profiling data, but it still slows things down quite considerably. So, I threw in this monkey patch:

(class <



As it turns out, the process of building the printer object is still quite slow, but it is, as you’d expect, a lot faster without all of those SQL queries.



jmettraux's rufus-scheduler at master - GitHub
Tue, 17 Feb 2009 15:32:21 -0500
jmettraux's rufus-scheduler at master - GitHub: A Ruby task scheduling implementation

out of time (mat brown on programming)

ruote

redis is a key-value data store

Sunspot 0.8 is out

Direct access to the Query API

Dynamic Fields

Dirty Sessions

That’s all for now

Installing alternate Ruby versions as optional packages

Download the packages

Install YARV

Install JRuby

Installing Gems

A 15-line alternative to multiruby

Sunspot 0.7 is out

Documentation!

Less magic in the search DSL

Negative scoping

Exclusion by identity

Restrict by empty values

Faceting

Explicit commits

Boolean field type

Attribute field flexibility

Virtual field evaluation flexibility

Order by multiple fields

New adapter API

Goodbye Builder API

Goodbye extlib

That’s all, folks

RabbitMQ and Ruby

Git Tip of the Day: git cherry-tree

The problem

The Solution

How to use it

ShellElf runs shell commands from a Starling queue

rake db:rollback_to_common

Using Git Submodules for Shared Rails Components

Where are the REST frameworks?

Using Ruby's Autoload Method To Configure Your App Just-in-Time

"Debugging 2x as hard as writing code. If you write as cleverly as possible, you are, by definition,..."

blowmage: anyone remember seeing a screenshot of _why's computer about a year ago? he was using a tiled window manager. can't find the url...

Three changes I'd like to see to DataMapper before 1.0

1. Get rid of has 1 / has n for declaring associations

2. Don’t extend Symbol for non-general use

3. Take migrations seriously

MongoDB

Google Data on Rails

Why you see hundreds of "SHOW TABLES" queries when running ruby-prof in Rails

jmettraux's rufus-scheduler at master - GitHub

1. Get rid of `has 1` / `has n` for declaring associations