Ruby On REST: Introducing the Representer Pattern

{{{
This post introduces the new “Roar gem”:https://github.com/apotonick/roar, a framework-agnostic REST framework for Ruby that puts focus on object-oriented REST documents.

*More object-orientation?*

The calls for more object-orientation in Ruby frameworks are growing ever louder. Most web frameworks, like Rails, come with a nice database abstraction, a Rack-like HTTP endpoint, routing, and a template engine. *It’s the lack of additional abstraction layers* that make many programmers ask for “more OOP”.

Already yawning? Yeah, I wrote “about that earlier this year”:http://nicksda.apotomo.de/2011/04/rails-misapprehensions-rails-needs-more-abstraction-layers/. Let’s talk about _working_ solutions now. Let’s talk about REST.

h3. REST is about Documents!

REST is not only about pretty URLs. The problem ain’t URLs at all. Most frameworks have a decent routing layer to connect resource logic (aka controllers) to URLs. However, the key of *REST is pushing documents between clients and servers*, so why don’t we finally focus on documents?

It’s the document parsing, processing and rendering that makes our heads spinning.

*Note:* I here use the terms _representation_ and _document_ interchangable just because I can!

h4. Parsing and Processing

Both clients and services have to parse incoming documents. Consider this JSON document.

{"title":  "Lemon",
 "colors": ["green"]}

This could be sent to a resource to create a new fruit, or be retrieved by some client which issued a GET request.

In Rails, and many HTTP gems, this is automatically parsed using a JSON gem.

params = JSON['{"title":  "Lemon",
                "colors": ["green"]}']

params["title"] #=> "Lemon"
params["colors"][0] #=> "green"

We’re using a hash access to read properties from that document. This is handy! Now, assume the sender forgot to include the colors.

params = JSON['{"title":  "Lemon"}']

params["colors"][0]
#=>NoMethodError: undefined method `[]' for nil:NilClass

An exception, shit! While processing the parsed document (aka hash), _we_ have to take care of missing keys, wrong types etc. We as the consumer have to know about all the pitfalls with this particular document.

h4. Rendering

Now that our parsing and processing code is finished, how do we render outgoing representations? We could use @#to_json@.

@lemon.to_json(:only => [:title, :colors])

Being a quick solution for rendering small models this approach hits the wall when we have nested models but still want a plain JSON representation, or when we want to embed hypermedia. Also, we put “REST” logic into our database layer. It’s up to you to judge.

What about template languages? Let’s try “jbuilder”:https://github.com/rails/jbuilder for JSON rendering.

Jbuilder.encode do |json|
  json.title  @lemon.title
  json.colors @lemon.colors
end

Nice! But, wait. How do I test this? Can I test this separately? What if I want different representations for the same model? Do I need different templates? And… is it cool to have format knowledge both in our parsing layer and in our rendering layer? I mean, both need to know about the @title@ property and the @colors@ array.

h3. Representers: OOP to our Documents!

I won’t further discuss the shown concepts here. Let me just criticize that they are not object-oriented and encourage the distribution of document internals about the entire framework.

Here’s how “representers”:https://github.com/apotonick/roar can handle all that and make you a happy man at the same time!.

require 'roar/representer/json'

module FruitRepresenter
  include Roar::Representer::Base
  
  property :title
  collection :colors
end

This is all we need to *declare the basic syntax* of the fruit document. Note that it resides in a module which can be used anywhere in your app.

h3. Rendering a Lemon

Now let’s say our database layer provides us a class @Fruit@ with ActiveRecord-like behaviour.

lemon = Fruit.new(:title => "Lemon")
puts lemon.title #=> "Lemon"

Since the representer knows everything about the document format it can be used for rendering JSON. We just have to include the @FruitRepresenter@ module into the class and new methods will be available.

Fruit.class_eval do # DB layer.
  include Roar::Representer::JSON
  include FruitRepresenter
end

lemon.to_json
#=> "{"title":"Lemon","colors":[]}"

Awesome, the @FruitRepresenter@ module gives us a @#to_json@ method. Internally, this method compiles the JSON document by asking the represented object (aka @lemon@) for its properties and values. Remember, the representer _knows_ about those properties, since we declared them. It does _not know_ anything about @ActiveRecord@ and the like.

Now, what about parsing?

h3. Parsing a Lemon

The great thing about representers is: *they work bidirectional.* That alone is one of the main advantages to many other gems out there. Bidirectional? Let’s just see what that is.

orange = Fruit.from_json("{"title":"Orange"}")
puts orange.inspect
#=> #

We can parse representations and create objects from it! That’s *bidirectional: rendering and parsing* is combined in one representer.

Again, the representer does not know anything about the underlying database – it just creates an object it was mixed into, reads from the document and assigns properties using polite setters. Nothing more, bro.

Don’t like the class method? If you already have an instance, you can use @#from_json@ on the instance.

orange.from_json("{"title":"Yummy Orange"}")
orange.title #=> "Yummy Orange"

h3. Nesting Representers

Now that we have fruits, lets put ’em in a bowl. A fruit bowl contains a list of fruits.

bowl = Bowl.new
bowl.items << lemon
bowl.items << orange

In order to bind the fruit bowl to a REST resource we write another representer module.

module BowlRepresenter
  include Roar::Representer::Base
  
  property :location
  collection :items, :as => Fruit
end

Bowl.class_eval do # DB layer.
  include Roar::Representer::JSON
  include BowlRepresenter
end

Properties can be typed using the @:as@ option. The target class must be a representer, too. That’s the only requirement.

bowl.to_json
#=> {"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}
]}

Typed properties allow us to build more complex representations without breaking with OOP. The @collection@ assignment makes building lists as easy as possible. And if you still need to tweak things, there are some helpful configuration options around – more on that in one of the next posts.

h3. Representers love HATEOAS

HATEOAS? Check out “this talk I gave”:http://www.eventials.com/rubyconfbr/recorded/M2UzZTJkMzY2MzdiNTg2NTUxNWM1MzI3NWY1YjRhMzYjIzM3OQ_3D_3D if you want to learn about Hypermedia and HATEOAS.

Hypermedia helps clients to operate the API without having to compute URLs. We want hypermedia in our documents.

module BowlRepresenter
  include Roar::Representer::Feature::Hypermedia
  
  link :self do
    "http://bowls/#{location}"
  end
end

}}}
{{{
Including the @Hypermedia@ feature dynamically adds a new method @#link@ to our representer module. What does it do?

bowl.location = :desk
bowl.to_json
#=> {
"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}],
"links":[{"rel":"self","href":"http://bowls/desk"}]
}

Seems like an easy way to embed links in our documents. In @roar-rails@ the Rails URL helpers work like a charm within the @link@ blocks – making it super simple to provide hypermedia to consumers and still being _railsy_.

h3. Representers on the Client Side

Ok, representers basically are plain modules. Why not use ’em on the client side as well? The client doesn’t have access to the model layer – if it does, something’s wrong! However, it’s absolutely ok to share the representer modules.

class Bowl
  include Roar::Representer::JSON
  include Roar::Representer::Feature::HttpVerbs
  include BowlRepresenter # from a gem.
end

See how we simply use an empty class with the mixed-in representer on the client side? Remember, we don’t want to have access to the database here.

The next step is creating a new fruit bowl.

bowl = Bowl.new
bowl.location = "kitchen"
bowl.post!("http://bowls")

This code is enough to send a POST request to the specified URL. The @bowl@ instance will serialize a JSON document, send it to the URL using the @HttpVerbs@ feature, receive the new bowl, deserialize it and update its properties accordingly. You may know this behaviour from REST client gems like @ActiveResource@ – here, with less code and manageable complexity.

h3. Consume Hypermedia, it’s gooood.

The representer also makes it easy to extract hypermedia links from documents.

bowl.links[:self]
#=> "http://bowls/kitchen"

No parsing needed, the representer knows where to find your hypermedia. How cool is that?

h3. XML is not a Crime!

Roar comes with both JSON and XML representers.

module BowlRepresenter
  include Roar::Representer::XML
end

bowl.to_xml
#=> '
#  
#'

Now choose if you want JSON or XML – it’s up to you! I still don’t understand the XML hating – did I miss something or is it just “too java”?

h3. Roar – and more?

The “Roar project”:https://github.com/apotonick/roar is still evolving. However, the current API is almost stable. New features, new concepts, testing helpers, etc will be added over time just like in every framework. Roar is framework-agnostic and can be used anywhere, as long as it’s Ruby.

Its modulare feature architecture makes it easy to extend your documents. Currently, we have some exciting on-going brainstorming and hacking on features like client-side caching in the representer instance, generic error handling, validations, DSLs, … *We need you* to try it out, complain, propose, help, and spread the word. Thank you for reading until here, you rock!
}}}

30 thoughts on “Ruby On REST: Introducing the Representer Pattern

  1. Also I like to have messagepack format (for huge objects less transferred data). Just an idea.

    If it’s almost stable now I will give it a try in a testing branch of a project. Currently playing around with rabl (https://github.com/nesquena/rabl) which also seems very useful to build representations. What do you think?

    Like

  2. @asaaki: The messagepack would be another feature to include – cool idea!

    rabl is fine but it’s one-way only (just rendering) and it doesn’t provide any OOP notation like a representer instance does (like a #post! on a document instance, that’s simply cool).

    Like

  3. I like that your modules are bi-directional but by mixing the presenter as a module you forgo the first reason of the presenter pattern: to allow multiple representations of the same object.

    Like

  4. @zimbatm: Good point! You’re still free to include the representer module in an additional class which is responsible for wrapping a model for a particular representation format.

    Beside that, in the next version you can extend models at runtime and inject only the representer module you just need. This way, you can have different representations – how you like? Sexy time?

    Like

  5. Hi Nick! Great post… it sums up a lot of nice ideas and problems that occur in the presentation layer (not only in the Rails/Ruby world).

    As someone pointed out, it could be nice to be able to add the module to a single object so we can render it in different ways (I bet you already though of this… its just that you didnt write the example.).

    Configuration should be bidirectional although I still fight against the idea that noone should be aiming at integrating systems with the same language. If the idea is to split a system into two, its probably because both have different strengths and it might be that different languages solve those different problems. In other words, aim for bidirectional config at once, but do not rely on it.

    I like the API and I used to love all the magic we did on hypertemplate (and therefore tokamak) such as:

    module FruitRepresenter < Representer {
    title
    colors
    }

    module Bowl Fruit
    }

    It’s a method missing layer on top of your API. Not the fanciest thing ever. And not slow if you read config once.

    The presentation layer has always seen a lot of abuse (in every language), and it’s always nice to see it improving.

    keep up the good work

    cheers

    Like

  6. I version my API (and thus the representations of my models). Using your example above, let’s assume I have v1 and v2 of my Fruit representation. I would have something like this:

    require 'roar/representer/json'
     
    module V1::FruitRepresenter
      include Roar::Representer::Base
     
      property :title
      collection :colors
    end
    
    require 'roar/representer/json'
     
    module V2::FruitRepresenter
      include Roar::Representer::Base
     
      property :name
      collection :varieties
    end
    

    We can’t include both of these modules into our Fruit model class, as a call to #to_json would result in calling which ever module got included last. Instead, we would need to dynamically extend whichever module we need for the version requested by the client, i.e.

    @fruit = Fruit.find_by_id(1)
    @fruit.extend(V1::FruitRepresenter)
    

    Correct? Just want to make sure I’m not missing something regarding different representations of the same object.

    Like

  7. It doesn’t look like the module approach you use in your examples above actually works with roar 0.8.2. Looks like the underlying representable gem expects a class:

    ruby-1.9.2-p0 :001 > require 'roar/representer/json'
     => true 
    ruby-1.9.2-p0 :002 > module FruitRepresenter
    ruby-1.9.2-p0 :003?>   include Roar::Representer::Base
    ruby-1.9.2-p0 :004?>   property :title
    ruby-1.9.2-p0 :005?>   end
    NameError: undefined local variable or method `superclass' for FruitRepresenter:Module
    	from (eval):7:in `representable_attrs'
    	from /Users/bploetz/.rvm/gems/ruby-1.9.2-p0@api/gems/representable-0.9.3/lib/representable.rb:82:in `block in add_representable_property'
    	from /Users/bploetz/.rvm/gems/ruby-1.9.2-p0@api/gems/representable-0.9.3/lib/representable.rb:81:in `tap'
    	from /Users/bploetz/.rvm/gems/ruby-1.9.2-p0@api/gems/representable-0.9.3/lib/representable.rb:81:in `add_representable_property'
    	from /Users/bploetz/.rvm/gems/ruby-1.9.2-p0@api/gems/representable-0.9.3/lib/representable.rb:62:in `representable_property'
    	from (irb):4:in `'
    	from (irb):2
    	from /Users/bploetz/.rvm/rubies/ruby-1.9.2-p0/bin/irb:16:in `'
    

    Like

  8. @Brian: Yupp, if you want multiple representations for the same object/class, you need to use extend. I’m working on this right now. The problem is when you have nested setups, those sub items need to be extended as well. Will blog/tweet when it’s running. Thanks for the code samples!

    @Evan: Thanks! Please note that JBuilder is just another template engine – we don’t need another template engine! Representers take the OOP approach and also provide parsing, which a simple template engine can’t.

    @Guilherme: Hey bro, good to have you here! I really like something like tokamak in Restfulie and I guess it’s way faster to write a representation, but then, again, the parsing is missing and that was my initial concern why I came up with roar…

    Like

  9. Holy crap, long overdue comment. My bad. ❤

    I fear this project is going down the same-ish road as ActiveResource; it feels too 'serialize'ish. SOAP in REST clothes.

    XML and JSON are _not_ hypermedia, so that's another huge problem.

    Do you know what I mean?

    Like

  10. @keruilin: Hmm… Representers are Ruby so they won’t be of much help in a JS environment. I can see them working inside a Backbone.js/Apotomo frontend app, though.

    @Steve: Couldn’t disagree more and don’t know what you mean 😉 ActiveResource is focused on models, Representers on documents that may feel like models. The hypermedia support works out-of-the-box (did you miss those paragraphs above?) and representers can be used on client and server. That will make a good blog post, thanks! 😀

    Like

  11. > The hypermedia support works out-of-the-box (did you miss those paragraphs above?)

    No, I did not. You’re not doing hypermedia; you’re doing JSON and XML.

    > representers can be used on client and server.

    Right, this is why I make the ActiveResource comparison; you’re using HTTP to transfer some model objects over the wire. All your examples show a 1-1 model -> representer connection.

    Like

  12. > You’re not doing hypermedia; you’re doing JSON and XML.

    Uuuuuuuhm… explain! In my idea, hypermedia is embedding links into documents (i.e. JSON, XML, MessagePack, …) that point the client to possible actions he could take from this state. What’s hypermedia in your understanding?

    > you’re using HTTP to transfer some model objects over the wire.

    Nope, representers are not necessarily objects, they are documents. Just documents. That’s a big difference to ActiveResource which misapprehends people that every resource is a model.

    > All your examples show a 1-1 model -> representer connection.

    For the sake of simplicity. Let’s write a follow-up post on how representers can help with “virtual” resources, like a workflow.

    Models can have different representers, and representers don’t need models necessarily.

    Like

  13. > What’s hypermedia in your understanding?

    You’re adding semantics on top of JSON and XML that don’t exist in the media type itself. This reliance on out-of-band information violates the ‘self-descriptive messages’ constraint.

    > For the sake of simplicity.

    Fair enough. I’d like to see more of this kind of thing, though, because there’s far too much 1-1 going on in the real Rails world.

    Like

  14. @TS: I personally don’t care about the format but I like your gzip idea. People asked for msgpack, so it’s my job as a gem maintainer to provide it 😉

    Like

  15. @steve klabnik: you seem to be focusing on what hypermedia isn’t but not on what it is. Can you please give a straight answer to “What’s hypermedia in your understanding?” ? This will hopefully help us understand what you mean.

    Like

  16. @foofam Hypertext is text displayed on a computer or other electronic device with references (hyperlinks) to other text that the reader can immediately access, usually by a mouse click or keypress sequence. Hypermedia is used as a logical extension of the term hypertext in which graphics, audio, video, plain text and hyperlinks intertwine to create a generally non-linear medium of information.

    Like

  17. Actually Steve Klabnik seems to have a valid point. From “Building Hypermedia APIs with HTML5 and Node” (published by O’Reilly, see http://shop.oreilly.com/product/0636920020530.do):

    “A Hypermedia Type is a media type that contains native hyperlinking elements that can be used to control application flow.”

    Same book, Chapter 1, page 6: “Unlike plain text, XML, JSON, and other common formats, HTML has native support for hypermedia. HTML is, in effect, a Hypermedia Type. There are other media types that share this distinction; ones that contain native support for some form of hypermedia linking. Other well-known types include SVG (read-only), VoiceXML, and Atom.”

    I don’t know where that leaves us in regards to REST in Rails, roar, JSON responses and hypermedia…

    @steve, @nick thoughts?

    Like

  18. By the way, it is probably worth adding that the use of Node.js and HTML5 in “Building Hypermedia APIs with HTML5 and Node” is mostly incidental, the book is all about building hypermedia APIs (not necessarily REST ones) and therefore relevant to what’s being discussed here.

    I can see that Chapter 3 is titled “JSON Hypermedia” but I haven’t yet read that far in the book; I hope it will provide some answers to questions raised here.

    Disclaimer: I have no affiliation with O’Reilly or the book author. I just happen to be interested in the subject of creating REST APIs.

    Like

  19. I do have an affiliation with the author; he’s awesome, and while I haven’t read the book yet, my understanding is that your characterization of node as ‘incidental’ is correct.

    Like

  20. @steve the book is an eye-opener so far. I think we’re just scratching the surface here.

    @nick in my humble opinion, roar will benefit a lot if you read this book.

    Like

Leave a comment