Rails Misapprehensions: The Hardest Thing about REST is Working with Representations!

{{{
The REST architectural style is still intriguing me. After having discussed “how PUT and POST do differ”:http://nicksda.apotomo.de/2010/12/rails-misapprehensions-understanding-restful-put-and-post/ let’s move forward and *learn a bit about representations*, one of the key principles in REST.

h3. What’s a Representation?

It is inevitable to understand that when accessing a REST service, say @http://users.example.com/apotonick@, you get a _representation_ of the resource in its current state, *and not the resource _itself_*.

By specifying the @Content-type@ header we can even instruct the representation type. While a JPG picture is a decent representation we usually want *a XML document keeping all the important attributes* of the resource (a representation can be just anything, but I’d like to focus on XML for this post).

To illustrate the requirements for *proper representation handling* I will use an imaginary order resource for processing orders (orders contain items).

Let the following snippet be subject to further discussion.

  1  
  
    
    1
  
  
    
    3
  
        
  
  

Notice the contained @@ blocks and the embedded hyperlinks in terms of good HATEOAS.

h3. Representations can be pain.

Now what’s the problem with representations? Why not start with the *server, the REST service*. It has two duties to handle.

* *Serializing the order* with its items is one task. This might lead to an XML document that is sent back to the client in a GET request.
* *Deserializing a representation* of an order is the other tedious task. When a client POSTs/PUTs we usually need to *process the sent representation and map it to our data models* in order to save the changed world in the database.

Both tasks can cause headache. Rails has a couple of built-in – but limited – mechanics to help you with this work.

h3. Generating XML representations in Rails

ActiveRecord makes serializing a model to XML very easy (at first sight) – *you can simply call @#to_xml@.*

order = Order.find(1)
puts order.to_xml

This returns an opinionated XML document.

  1
  2011-04-03
  1
  2011-04-03
  ...

Additionally, @#to_xml@ allows some tweaking by passing in option parameters. However, *this approach is pretty limited*. For instance, it is quite difficult to render @@ tags with attributes.

*UPDATE:* Also, it is _impossible_ to render the @@ list without a wrapping @@ tag.

h3. Using builder for rendering

Another possibility is *using an XML template with builder*, in our case this is in @app/views/orders/show.xml@.

xml.instruct!
xml.order do
  xml.id @order.id
  xml.item do
    xml.link :rel => :article, :href => item_url(item)
#...

This provides a decent way for rendering complex XML documents that represent our resources.

h3. Using Restfulie’s tokamak

The “Restfulie gem”:https://github.com/caelum/restfulie ships with its own template language for rendering representations. This is called tokamak and works similar to a builder view.

member(@order) do
  id @order.id
  collection(@order.items) do |m, item|
    link :rel => :article, :href => item_url(item)
    amount item.amount
  end

  link :rel => self, :href => order_url(@order)
end

The cool thing here is that *tokamak allows rendering this abstract representation to multiple targets*, like XML or JSON.

h3. Working with representations

Now that we covered the outbound part of rendering representations let’s see how we can *work with incoming representations*. This usually happens in POST and PUT processing actions in your controller.

*Working with a representation means parsing* – or deserializing – it and then accessing its attributes to map those to your data models.

h3. How does a real POST look like?

In Rails you typically use HTML forms, POST them to a controller action, and then process the incoming data.

For a long time *Rails made me think that POSTs are all about “POST parameters”* – incoming data is made available in a nested @params[..]@ data structure.

However, this is something I call a _Rails misapprehension_. A POST might also have a request body containing the resource representation you want to create.

A REST client written with Restfulie as HTTP helper might do the following to create a new order.

res = Restfulie.at("http://localhost:3000/photos/party")

res.as("application/jpg").post(%{

  
    
    1
  

})

Note that we’re *sending a document directly*, without any form usage.

Assuming our resource controller does understand that we will create a new order. Wow.

h3. What’s wrong with form POSTs?

Nothing! Using the @#form_for@ helper and Rails params is absolutely ok. However, I want to point out that *POSTs are not limited to HTML forms* – they can also send other representations.

So, what Rails does behind the scenes is
* *parsing* the incoming @application/x-www-form-urlencoded@ representation of your resource
* and make the deserialized representation *available in @#params@.*

This automatical behaviour is all possible *if you stick to Rails’ conventions*, meaning very limited XML formats and thus representations.

h3. Working with _parsed_ representations

In the simplest case it looks as if the Rails approach would strike.

class OrdersController < ApplicationController
  def create
    @order = Order.create(params[:order])
    # ...
  end

However, in our case, this won’t work as we have “non-standard” @@ tags in our incoming representation. *The internal parser in Rails* doesn’t recognize that and things will explode.

h3. The internal parser?

Yeah, Rails has a hard-wired internal parser to deserialize a couple of mime- types (“POST parameters”, XML and JSON). This code is found in @ParamsParser@ “here”:https://github.com/rails/rails/blob/v3.0.6.rc2/actionpack/lib/action_dispatch/middleware/params_parser.rb.

I’m sorry to say that but that _middleware_ is a debatable concept in Rails. While it blindly parses incoming request data it also urges you to use *Rails’ limited representation convention*.

If you don’t go the Rails way and use very simple XML representations, it will simply not work.

h3. Parsing D.I.Y.

Now that Rails doesn’t give us what we need we have to do it ourself.

class OrdersController < ApplicationController
  def create
    item_urls = Nokogiri::XML(request.raw_post).
      xpath("//item/link").
      collect { |i| i[:href] }
  
    @order = Order.new
    # ...
  end

Looks horrible.

h3. The dilemma – where is my representation?

A severe problem emerges – *where is my representation*? Or, in other words: where are transformation rules stored?
}}}
{{{
* *Rendering the representation* involves either an XML template or an adjusted @#to_xml@ which *knows _a lot_ about the structure and semantics* of our representation.
* *Parsing the representation* implies a hand-made XML parser that itself *knows about the very structure and semantics* of the representation, again.

The dilemma: Knowledge about the representation is *cluttered over the entire MVC-framework*.

h3. A new abstraction layer: Representers

What might sound over-engineered to many developers used to the Rails convention is my new approach which abstracts “working with representations” to a new layer.

A so called *Representer handles both the in- and outs* of representations, not sharing the internal syntax and semantics with the outer world (the best we can).

The @Item@ representer looks simple.

class ItemXmlRepresenter < Roar::Representer::Xml
  name :item
  link

  property :amount
end

The @Order@ representer already contains a composition – the ordered items.

class OrderXmlRepresenter  :item, 
    :as => [ItemXmlRepresenter]
  
  link do |controller|
    :rel => :self, :href => controller.order_url(self)
  end

  link do |controller|
    :rel => :checkout, 
    :href => controller.order_checkout_url(self)
  end
end

This is a part of my “REST framework roar (Resource-oriented Architectures in Ruby)”:https://github.com/apotonick/roar which tries to help with writing real RESTful apps in Rails, Sinatra and Padrino.

h3. How does roar help me?

Now that we defined the representers we can use them in controllers.

class OrdersController < ApplicationController
  def show
    @order = Order.find(params[:id])
    
    xml = OrderXmlRepresenter.serialize_model(@order)
    # ...

The call to @#serialize_model@ will render the XML representation for the @order object using the proper representers, *including real HATEOAS hyperlinks, nested items*, and so on.

What comes out is the XML document I introduced in the beginning paragraph.

Now, how do we process input?

class OrdersController < ApplicationController
  def create
    rep = OrderXmlRepresenter.
      deserialize(request.raw_post)

    @order = Order.create(rep.to_nested_attributes)
    
    # ...

When handling incoming data the @#to_nested_attributes@ method from *the representer helps transforming the data into a hash* complying to nested-attributes requirements used with @#update_attributes@.

h3. Benefits?

Representers try to improve the architecture. They…
* *embody the transformation* of a representation in one class while allowing serialization _and_ deserialization.
* allow *compositions* of representations. Order @has_many@ items.
* make *hypermedia support* easy (we will learn more about that in the next post)
* are inheritable and extendable and *bring back OOP*. For instance, you might write convenience accessors into your representer class.
* are *usable in REST clients* as well, which allows decoupled, solid systems (see next post).
* make *testing* easier.

h3. Hey, this is a proposal!

The cool thing here is: that stuff is already working. *It’s still under heavy development, but it works.* I’m maintaining a “sample application at github”:https://github.com/apotonick/restful_order which I use in my upcoming thesis about Ruby, REST architectures and web GUIs.

My goal with roar is to make it easier and cleaner to develop RESTful, decoupled systems in Ruby while using all the great existing gems like “Restfulie”:https://github.com/caelum/restfulie.

Looking forward to your feedback, now.

In the next post I will write about how Representers help keeping your REST _clients_ clean and dumb.
}}}