out of time
mat brown on programming, politics, cooking, and assorted nerdiness
-
Use backup gem with Heroku’s pgbackups
CommentsThe backup library is a powerful, flexible, easy-to-use way to configure and execute data backups for your Ruby app. It doesn’t support Heroku’s pgbackups out of the box, but the library has an excellent modular design, making it dead simple to code up a module for pgbackups.
Here’s my quick-and-dirty crack at it. It works, but has basically no configuration options (not that pgbackups itself has a great deal anyway). Hope you find it useful!
To configure it, just set the
nameto the name of your Heroku app: -
Custom error pages in Rails 3
CommentsUpdated: It’s
public, notapp/views.Just so that posterity may have more luck Googling for this than I did: to create a custom error (5xx) page in Rails 3, create a static HTML file with the error code in your base
publicdirectory, e.g.:public/500.htmlIf you’re internationalized (and you are, right?), you can add an I18n extension:
public/500.en.htmlThese paths are looked up by the ActionDispatch::ShowExceptions middleware when rendering an error page for a non-local request.
This may or may not already be obvious to everyone but me.
-
Use observers with Mongoid 2 and Rails 3
CommentsMongoid 2 doesn’t have built-in support for ActiveModel observers, but thanks to the modular design of Rails 3, it’s trivial to tack on:
Drop that somewhere into your load path, include
Mongoid::Observingin the models you’d like to observe, and it’ll work just like ActiveRecord observers. -
http://xkcd.com/844/
Comments -
Spork, Mongoid 2, Rails 3
CommentsIn one of my (to date) signal accomplishments of 2011, I recently upgraded the Clique application to Rails 3. One thing I couldn’t get working right away, though, was the excellent Spork library, which lets you run RSpec without having to reload your application dependencies on every run. In particular, I couldn’t get Spork to pick up changes to my model classes. Not a deal-breaker, but TDD is pretty painful on an underpowered netbook when you have to load 50 gems each time you want to run a spec.
After some digging, I determined that Mongoid 2 has a railtie that eagerly loads the contents of the
app/modelsdirectory when the application loads. Game over, I thought — the railtie isn’t configurable, and I couldn’t even monkey-patch the method, because no application is run between loading Mongoid and running the railtie.But after a few painful days and some more digging, I noticed that Mongoid’s code uses
require_dependency, an ActiveSupport method that requires a file but makes the modules and classes defined therein reloadable (assuming you’re following the usual naming conventions for source files). Which means the problem was solved very simply by adding the following to myspec_helper.rb:Spork.each_run do ActiveSupport::Dependencies.clear endAnd that’s it. Usually this isn’t necessary because Spork’s master process never loads your application code, so it doesn’t have to unload it, but in this case it’s needed and it works.
-
Sunspot 1.2 released (finally)
CommentsAfter a firmly ridiculous amount of time in release candidate status (mainly owing to lack of time on my part), Sunspot 1.2 final is out. Here’s the inside scoop.
Upgrading
First, if you’re using Sunspot::Rails, you no longer need to explicitly load the ‘sunspot/rails’ source file (in fact, if you do, things won’t work right). So if you’re using Rails 3 (or bundler with Rails 2), your Gemfile just needs:
gem 'sunspot_rails'And if you’re using Rails 2 without Bundler, it’s just:
config.gem 'sunspot_rails'The other major change is in spatial search: Sunspot 1.2 has a complete rewrite of spatial search functionality, and both the API and the underlying implementation are quite different. I’ll go in to quite a bit of depth a little later in this post, but for now, here’s a quick before-and-after on the API.
Previously, you configured a (the) spatial field like this:
coordinates :coordinatesIn this case,
:coordinatesis just a method that’s used to return the coordinate information, which could be either a two-element array, or an object that responded to#latand#lng, or some other variants on those attribute names. Each document got exactly one set of coordinates, so there was no explicit field name associated with the information.Now, you set it up like this:
location :coordinatesSeems pretty similar, but there are a couple of crucial differences. First,
:coordinatesis an actual field name, with aLocationtype. You can think of it like any other field, and you can pass all the usual options in. You can also specify more than one location field:location :hq_coordinates location :field_office_coordinates, :multiple => trueAlso, Sunspot 1.2 is stricter about the data that’s used to populate location fields: It has to be an object (or array of objects, if it’s a multi-valued field) that responds to #lat and #lng. You can use
Sunspot::Util::Coordinatesif you’re not working with objects that already fit the bill.OK, now on to performing geo search. Before, you’d do this:
near [40.0, -70.0], :distance => 5Now you’ll do something like:
with(:hq_coordinates).near 40.0, -70.0, :precision => 8Don’t worry too much about what that
:precisionoption means right now; we’ll get into that later.What’s new in Sunspot 1.2
Spatial search with GeoHash
By far the biggest change in Sunspot 1.2 is a complete rewrite of the spatial search component. Instead of relying on solr-spatial-light, a Solr plugin I wrote, to perform spatial search, Sunspot now uses a geohash-based spatial search strategy that is implemented completely in Sunspot itself; no special functionality is needed from Solr. This has some major advantages, but it also has some disadvantages.
The good:
- Performs well at large scale, since under the hood it is executing what amounts to a relatively simple fulltext search in Solr. Contrast with solr-spatial-light, which has severe performance problems at scale.
- Allows multiple location fields in a single document, and also multi-valued location fields.
- Allows searches to incorporate both fulltext relevance and spatial proximity when calculating result score, resulting in a very “natural” default result ordering when the search contains both fulltext and spatial components.
The bad:
- Control over search “radius” is severely constrained — only provide nine precision levels, ranging from 389 miles (precision 3) to 8 feet (precision 12).
- Proximity search matches locations that inhabit the same square on a fixed grid on the globe as the search origin; if the origin is near the edge of the square, then nearby documents will be missed, whereas more distant documents will be matched.
How it works:
When locations are indexed, they are converted into GeoHash values. GeoHash is a clever algorithm that encodes coordinates on the globe into a single string which has the property that, on average, the shared prefix of two geohashes increases as the distance between the locations decreases. Thus, proximity search can be performed very efficiently by simply searching for documents which share a prefix with the origin point, and closer documents can be given higher relevance by boosting matches with longer shared prefixes.
For complete documentation on using Sunspot’s new GeoHash spatial search, see the API documentation.
Rails 3 Compatibility
Sunspot::Rails 1.2 is fully compatible with Rails 3. There’s not much more to say about that — just include it in your Gemfile, and it’ll work. Sunspot::Rails is still tied to ActiveRecord, though; I’d like to make it more ORM-agnostic in the future, much as Rails 3 is today.
Support legacy Solr schemas with :as
If you’ve got a legacy Solr schema where the field names don’t follow Sunspot’s naming conventions, you can now explicitly tell Sunspot what a field’s name in Solr is using the
:asoption:string :title, :as => :legacy_titleAs well as supporting legacy schemas, this option can be useful if you want to set up new field types in your Solr schema.
Other enhancements
- The
SilentFailSessionProxywill swallow errors that occur during write operations, useful if you’ve got an unreliable Solr instance and don’t want to throw application errors when a non-critical Solr write fails. - You can now include documents by identity, e.g.
with(some_instance). Presumably the primary use for this would be inside a disjunction. - You can call
Sunspot.optimizeto trigger a Solr optimize from inside your application.
That’s all, folks!
Well, that’s all for this release, friends. But we’ve got some big, big plans for Sunspot 1.3, due out in January 2055:
- Support for [http://wiki.apache.org/solr/FieldCollapsing](Field Collapsing) (group results by field value)
- Add NGram and EdgeNGram field types for easy prefix/substring search
- Improve edge-case spatial matching by searching proximate n+1-precision geohash boxes
- More stuff that you want!
Thank you!
Each release of Sunspot has been less of an individual effort and more of a community effort than the last, and 1.2 is by far the biggest example of that yet. From now on, it’s my official policy to give committer rights to anyone who submits a good, robust patch; I’d like for ongoing Sunspot development to be entirely community-driven, and we’re already a lot of the way there. Big thanks to everyone who has contributed to the library so far and thanks in advance to those whose patches are still to come.
