The Sun is Setting on Rails-style MVC Frameworks
Lately I've been thinking a lot about the impact of the move to a thick client architecture for web applications, and I'm becoming more and more certain that this means that Rails-style MVC frameworks on the server-side are going to end up being phased out in favour of leaner and meaner frameworks that better address the new needs of thick-client architecture.
There are a few major reasons for this:
The server is no place for view logic
The devices available for running a web app are vastly different from when these web frameworks first sprung up. The slide towards thicker / richer clients has been proceeding on pace with increases in processing power since the Web 2.0 days. It's much simpler to handle views and view logic in only one place, and that place is slowly moving away from the server side. MVC has always been a strained pattern for a server-side, non-gui application and it is been a confusing and complicated trade-off to have the backend generating front-end logic. Front-end frameworks like backbone.js, as well as advances in web technologies like HTML5's history.pushState are now making server-free views a realistic quality of cutting-edge front-ends. Rendering on the client-side also gives us the opportunity to create responsive designs based on device capability rather than having the server try to somehow figure out what the capabilities are without actually running on that device.
The kinks aren't all the way out yet, but I do think the trend is clear.
Server-side Templating and .to_json are both underpowered and overpowered for the actual requirements of JSON APIs
There's no need for templating on the serverside (or view helpers, or any view-related cruft) to generate simple JSON documents, but there are a tonne of problems left unsolved when we fail to see that generating a JSON API is more than just a serialization problem.
How should dates look? (RFC3339 / ISO8601 of course!). What should the JSON error document look like when you provide a 400 and want to tell the client why? How should links to other resources in the API look? How does a collection look? How does pagination look?
These aren't just serialization concerns, and they have nothing to do with templating.
HATEOAS is not just an academic pursuit
A thick client does not want to maintain a vast list of static strings representing all the crazy URLs that it will have to call in a non standard API. As an API designer, you don't want them doing this anyway, because hard-coded URLs and URL structures make it a real pain for you to change your API.
The AtomPub protocol, if you ignore its XMLiness, gets this right -- The thick client just knows the root URL (which serves up a simple service document) and is aware of a bunch of relationship types (the 'rel' attribute on 'link' and 'a' tags) that it can follow as necessary. If a game client needs to access a list of players, it just goes to the root url, and follows the link with the rel property called 'players' (and probably remembers that link until the server says it has moved via 301 status code). JSON has no concept of links or rel, but this is still easy to imagine and implement, and while it's a teeny bit of extra work up front, the standardization buys you the ability to start writing smarter HTTP/REST clients for your frontend that take care of much more for you automatically, so you can spend time on real business logic and actually do something more productive than fiddle with the javascript version of a routes.rb file.
(A really great API framework might generate some or all of these links on its own and automatically put them in the JSON documents. It's pretty easy to imagine pagination links like next/back being generated automatically, or even links amoung resources in some cases, possibly based on some internally declared relationship.)
Rails-style MVC frameworks have a horrible routing solution for RESTFul JSON APIs
In a resource-oriented API, the router need not be concerned with the http methods that are or are not allowed for the resource. That's the concern of the resource and the resource alone. When the router tries to manage that, you get the unnecessary verbosity of a route for every method supported by the resource, and you get the app incorrectly throwing 404s instead of 405s when a method is not supported. This probably means that 'controllers' need to go away in favor of 'resources', and routes can be vastly simplified, if not completely derived/inferred by the framework. Because we keep thinking in this conventional MVC style though, we miss the possibility and potential of vastly more simple applications that actually do a lot more more for us.
The Application Developer shouldn't have to Deal with these Details
There's no reason for us to all separately think about these problems and solve them in a million different ways every time we're confronted with them. Aside from the years of wasted time this involves, we've also got a bunch of non-standard and sub-standard APIs to interact with, so all the client code needs to be custom as well and nothing is reusable. This is why being RESTful is not just academic and this is why being concerned with the details is not pedantic.
As I said earlier, AtomPub gets a lot of this right. A lighter-weight JSON equivalent would be a huge improvement to what people are doing today, because the conventions would mean that the framework can take care of most of these API details that we reimplement ad nauseum -- or worse, not at all. It also means that frontends can start to abstract away more of the HTTP details as well. This is already starting to happen in the new frontend MVC frameworks but in almost every case, the HTTP end of things still needs to be handled in a custom way for every API endpoint. There is still way too much work left to the application developer, and we're silly to continue to do it over and over without coming up with a better abstraction.
CouchApps and WebMachine are just starting to touch on this style of architecture. Backend-as-a-service platforms like Parse certainly understand how far this simple architecture can go, but ultimately there's a huge need for a framework that can create more complex RESTful APIs (in a language that's more general purpose and "blue collar" than Erlang).
Rails-style MVC frameworks are both too much, and not enough at the same time. It really is time for a new framework to support this new architecture.
February 24th, 2012 - 08:50
If we focus on thick clients, will support for the thin clients fall by the wayside? Do they matter anymore?
February 24th, 2012 - 09:11
You’ll find this interesting:
http://knockoutjs.com/
February 24th, 2012 - 10:20
I disagree with you, While some of your points are valid, I don’t think any sort of paridim shift to going to occur. The separation of logic between views and server in frontend MVC frameworks is simply to large for its own good. This separation if only abstracting where it isn’t helping.
February 24th, 2012 - 10:59
@colin: People are already talking about the browser-as-the-OS, and if everyone were already on today’s latest browsers, I think it’d be safe to say thin clients have already fallen by the wayside. They just have so much more capability today than they used to.
February 24th, 2012 - 11:03
I’ve been thinking many of the same things. Have you heard of Persevere? It is a javascript server that was designed to be a very thin server.
February 24th, 2012 - 11:04
@Anonymous: MVC was invented for front-ends specifically, so I have a hard time thinking it’s not appropriate there. It’s harder for me to consider it appropriate as a server framework. While I agree that on the server it can give you a good standard for separation-of-concerns, I think it coerces you into separating incorrectly.
February 24th, 2012 - 12:51
Right. There’s no reason the “V” component isn’t the bit that’s implemented on the client; in fact, it makes more sense that way.
February 24th, 2012 - 13:06
What about code duplication? Let’s say your server is just the data source and your logic and views are on the client-platforms themselves. Wouldn’t that imply a lot of logic duplication?
Aside from the point above, I think this is trully the trend set by platforms like Parse.
February 24th, 2012 - 13:41
Happy to hear somebody spell it out. I’ve been working on PureMVC http://puremvc.org thick clients in JavaScript and ActionScript for years now for the sole reason of achieving responsive designs that I consider a must. The milliseconds that I see a white screen between page loads of an app makes my soul die a little every time.
I’ve powered them mostly by using Rails but dropping the “V” from its MVC and creating RESTful APIs that only deal with storing the “state” on the server. It’s hard to beat Rails at the convenience of delivering RESTful models on the web.
Up side of the thick MVC client is that one can switch the API engine without having to rewrite the client. Also, multiple clients (iOS, Android, JS, AS) can subscribe to the same API. Point being, radically decoupling presentation and data layers is a very good thing.
To be one’s own devil’s advocate, I have heard of one good solution for thin client architecture that involves some clever caching from 37signals:
http://37signals.com/svn/posts/3112-how-basecamp-next-got-to-be-so-damn-fast-without-using-much-client-side-ui
@Anonymous I haven’t found the separation between the model on server and client a big hurdle in my experience. A lot of apps can load the required model from the server in a second. I approach huge and complex models through lazy loading. I’m very happy with how it keeps working out good.
February 24th, 2012 - 13:57
@Sasha: Yep… That’s exactly what I’m getting at: “radically decoupling presentation and data layers”. My only disagreement is that I don’t think Rails is actually tough to beat for delivering RESTFul models on the web. I don’t think anyone has beaten it, that’s for sure, but I do think it’s pretty easy to beat. I should probably be shutting up and writing that next framework though.
February 24th, 2012 - 14:02
@Tudor: I do see the possibility of code duplication in a lot of document validation scenarios (eg. The client should know not to try to POST a new user that doesn’t have an email address, and the server should know not to accept a POST of a new user without an email address). I haven’t got an idea for removing that duplication unless the server-side language you’re writing in can target the browser (either its javascript or compiles to javascript). Is that the duplication you meant?
February 24th, 2012 - 14:56
I really would LIKE to make this kind of shift, and i did something like what Sasha did with rails and a flex app. The main problem CURRENTLY is that it is hard to get Google to crawl the data properly. So much javascript makes things faster, but kills search-ability currently.
February 24th, 2012 - 15:26
I agree with @Sasha as well. Model and controller (and sensitive business logic) can’t be on the client where a simple “view source” can reveal all your secret sauce.
February 24th, 2012 - 15:46
@Gregg Or could validation information be passed in a similar manner as the link rel information, ie. as some form of structured meta data that would allow the framework to autogenerate the necessary validation logic on both the server and the client?
February 24th, 2012 - 17:43
I’m writing my web applications in a “resource like way” since 2009. This small framework supports resources represented as FOOP objects. Look at the wings example over here: http://rundragonfly.com/dragonfly_routes
February 24th, 2012 - 17:52
Of course server side isn’t the place for all view logic, but it never has been. That said, the browser (a wildly inconsistent, and rule-less environment) isn’t the place for all view logic either. There’s a happy medium which many sites have found, and which Rails fits into quite easily.
February 24th, 2012 - 19:31
Checkout my posts about a future web development framework. They were written back in 2007/2008 http://goo.gl/iqQFP
Also see http://www.thinserverarchitecture.com/
February 24th, 2012 - 19:38
Excellent post. I agree we need to make the clients work harder, be smarter, but I think the SEO/spider issue is still signifiant for web sites (not web applications). The Google Bot (and the semantics it expects) is holding us back.
February 24th, 2012 - 22:18
Just today I was thinking how amazing it is that it’s 2012 and we are still rendering HTML on the server. Yet as long as I’ve been developing there have been attempts to push it to the client and they’ve all pretty much fizzled. Java applets, Flash… they all made a go at it. One company I worked for in 2001 used JavaScript with XSLT to do browser-based rendering. Their rationale was pretty much the same as what you advocate here. So definitely not a new idea!
Great post though, and I generally agree with your logic. I guess we’ll keep seeing the early adopters are the “walled garden” sites that don’t care about crawlers and have to scale really huge.
February 25th, 2012 - 01:34
Linkbait. Bullshit. Won’t happen.
Next.
February 25th, 2012 - 02:05
Wrong. Wrong. Wrong.
If you’re writing (much) code in your view, you’re doing it wrong.
If your model doesn’t contain most of your application logic, you’re doing it wrong.
If your controller isn’t the gatekeeper between those two, you’re doing it wrong.
finally: to_json does not encapsulate rails. And if you aren’t page caching in rails, or any framework (preferably writing before you expire the cache), you’re probably doing that wrong, too.
Listen, I don’t really care if you screw it up. But I’m happy to pit 2 clowns and a parakeet against a team of idiots trying to write everything themselves. Good luck.
February 25th, 2012 - 03:26
@c ya: I don’t know if any of your arguments are actually with anything I’ve said. I’m actually 100% for MVC… I just don’t think it makes sense on the server once the view is moved to the browser. Client-side MVC makes a heck of a lot of sense though, especially since that is exactly what the MVC pattern was originally designed for.
@Mario Valente: Thanks… I enjoyed http://mvalente.eu/2009/11/25/requirements-for-a-modern-web-development-framework/ . That’s basically exactly how I imagine it too. I think I’m going to build it.
February 25th, 2012 - 07:00
Maybe you will be interested to take a look at OData ( http://www.odata.org/ ). Microsoft are not the most popular company in the Ruby community but they already have a solution that does prett much what you describe. OData producers can return Atom, JSON or plain XML, the URLs are in the service response and there is support for paging. The only downside is that in practice it is useless without an ORM framework under it. However with an ORM framework the clients can compose actual queries which provides great flexibility.
February 25th, 2012 - 08:21
@gregg: I agree 100%. Re validation duplication, this is already a problem. You should have constraints at the db level, since that is the most reliable way to maintain data integrity. You should have validations on the server side, since you often want to short circuit units of work if they don’t make sense. And the only way to deliver a non terrible user experience is with client side validations.
@people calling BS: things have been trending this way for at least the last ten years. In 2000 10% of your app being in js was a lot of client side code. Nowadays, 50% is completely normal. I do tsee it slowing down any time soon. The server is about presenting data, the client is about consuming data, that is the only way to make a responsive app with complex interactions.
February 25th, 2012 - 10:09
OData is an interesting protocol for what you’re proposing. It’s based on AtomPub, but it can also serve JSON. That a look! http://www.odata.org
February 25th, 2012 - 10:25
The benefit of any framework is that it limits the domain of possible solutions. Solution providers choose frameworks to create efficiencies for their operation that protect profitability. It has little to do with the clients. Developers can target locations on the continua connecting architectural qualities, but these optimizations are minor concerns. No executive team really cares about your responsive design initiatives, for example. These discussions are dumb. If IT would realize they’re a second-class business function, these conversations would focus more on the fact that we choose frameworks for personal reasons. They’re easy, they’re fun, or they’re new and exciting. It has little to do with any inherent superiority of the framework/style itself. You can successfully deploy rails-style-mvc frameworks for as long as your development team wants to.
February 25th, 2012 - 11:21
@Stilgar/Jordão : Yep… well OData is based on AtomPub, so that’s actually sort of what I mean. I haven’t seen any examples of the JSON support, but it would have to be pretty lean and mean for people to want to use it. People use JSON because it’s tough to imagine anything simpler, so they won’t just tolerate JSON translated from XML, or JSON with extra nodes that add no obvious value. That’s where a lot of the bigger companies get JSON wrong, but I haven’t seen the MS version, so that’s not a comment on OData.
@ecomware: It sounds like you think there’s no more need for innovation in computing. But there’s nothing in Rails that PHP couldn’t do, and there’s nothing in PHP that Perl CGI scripts couldn’t do. Each shift just did the same thing, but in better, simpler ways. This is how innovation happens, and it has never tried to get the permission of the executive teams of the world first.
February 25th, 2012 - 11:42
> “MVC has always been a strained pattern for a server-side, non-gui application…”. I disagree. See the history of MVC.
If you/developers choose a thick-client approach, a RESTful architectural style isn’t necessarily the ultimate solution. It’s not the default solution for every purpose. If a webapplication can’t be realized solely with true hypermedia formats, then why trying to force it by hook or by crook?
# Routes # In essence, routes are just a proven and wide spread solution for determining the parts of an application that is responsible for an incoming request. I agree that this reasoning logic is concern of Resources; delegating these “responsiblity checks” to them is trivial. But I think even then, we still need kind of routing mechanism. Resources can be exposed via more than a single URI, and in order to make “responsibility checks” solely the concern of Resources, they must be aware of each and every URI they are exposed through. Is this a desired behaviour? Asking each Resource for a responsibility-check once for every single incoming request? Wouldn’t it be more useful to just let a Resoure decide if it “accepts” an incoming request based on its properties (HTTP method, data supplied, asf.) but leaving the actual Routing (literally) to another application part?
> A thick client does not want to maintain a vast list of static strings representing all the crazy URLs that it will have to call in a non standard API
Roy Fielding and some other smart brains already have a suggestion for that issue in their petto: http://tools.ietf.org/html/draft-gregorio-uritemplate-04 (Following @mamund sometimes pays off
February 25th, 2012 - 11:49
Sorry, the url points to an outdated version of the “URI Template” IETF draft. Here is the most recent one: http://tools.ietf.org/html/draft-gregorio-uritemplate-08
February 25th, 2012 - 12:03
@Gregg
I may agree with almost everything you said in reply. I hope to escape any nuanced conversation about what constitutes innovation, but following the precedent you’ve set for generalities, I can happily provide confirmation that I’m a fan of it.
I appreciate the opportunity, too, to make use of your example of languages supplanting each other. Amazon still runs a significant amount of perl in their web systems. There are still profitable PHP systems, I’m told. This makes me believe a rails-mvc-framework can provide good, profitable service for as long as your team is content with it.
No need to seek permission from the c-level teams on these matters, either, I agree. Solve their primary problems effectively, and they couldn’t care less how you solve yours.
February 25th, 2012 - 12:23
@Peter: Do check the history… http://c2.com/cgi/wiki?ModelViewControllerHistory . MVC predates the web and even HTTP, and originally had nothing to do with networked applications at all.
My beef isn’t with routes… It’s with the factoring of routes in Rails-Style frameworks. A route is necessary to tie a URI to a resource, but the resource can decide whether or not it supports the request method. That’s the best separation of concerns for reasons I’ve already explained. And while the concept of routes is obviously necessary, I’m honestly starting to think that a routes *file* isn’t.
I’ve seen that URI template draft before, and I think it’s a fantastic idea. It’s meant to supplement HATEOAS though, not supplant it. It solves only a subset of HATEOAS, which is how the client can determine URLs that are not most simply represented as just a simple link. This is exactly the kind of stuff that I don’t think traditional server-side frameworks have cared to support.
February 25th, 2012 - 19:14
Well this seems about as far from thick client as you can get…
http://www.nytimes.com/2012/02/23/technology/personaltech/onlive-desktop-plus-puts-windows-7-on-the-ipad-in-blazing-speed-state-of-the-art.html?_r=1&ref=technology
February 25th, 2012 - 19:52
Spring MVC supports both URI templates (http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/mvc.html#mvc-ann-requestmapping-uri-templates) and no-routes-file mapping to resources (using @RequestMapping)
February 25th, 2012 - 20:51
@Eric P: I’m not sure Spring MVC is the lightweight non-MVC alternative that I’m looking for.
Also, @RequestMapping seems to include the HTTP method in the route, which I personally think is a problematic factoring because an application can’t tell the difference between a 404 (Resource Not Found) and 405 (Method not Implemented) that way.
February 25th, 2012 - 21:48
This is a great post, thanks. I know its supposed to be dying out, but isn’t Silverlight what you are after? I think it requires all communication with the server to be done using WCF (another Microsoft technology) which supports various communication bindings from plain a web service through to binary TCP.
February 25th, 2012 - 22:21
@dodgy_coder: Well not really… I’m just talking about using existing open, non-proprietary standards and non-plugin web technologies. There are good real-world reasons that Silverlight and Flash/Flex are dying out (even though they’re architecturally pretty sound).
February 27th, 2012 - 15:07
s/RFC339/RFC3339/
February 27th, 2012 - 16:08
Thanks Leif. Fixed.
February 29th, 2012 - 08:13
Thick clients sound lovely for all the reasons you mention. They will not work, however, for one simple reason — Google. If google cannot crawl my site and see the data in nicely parsable views, created server side, people will not be able to find my site. Meaning no money coming in to pay for the lovely thick client.
Thanks, but no thanks.
February 29th, 2012 - 09:52
So in a few words, you are saying the new trend is single page apps that uses a RESTful API.
February 29th, 2012 - 12:50
@shawn mclean: I think so. But I’m really saying that since that’s the way it’s going, we can create better server-side frameworks than what we currently have.
@Michael Watson: Google is already adapting. Googlebot already knows javascript! I agree that it’s not ready today, but the apps leading the way will be the ones that don’t rely as much on SEO.
February 29th, 2012 - 13:29
There are some interesting thoughts in the article and in the comments too. I can’t help but have a concern about this behaviour will move us more away from the document oriented model of the web. It really bugged me when Twitter’s website only hit their API and if you had Javascript turned off you’d get an empty page. They’ve fixed that now, and I know it’s an ever diminishing argument but we really shouldn’t have to have Javascript enabled to use the web. Accessibility is one issue here, but as mentioned, so is crawling the web. Google has created Ajax Crawler (https://developers.google.com/webmasters/ajax-crawling/docs/getting-started) but will that work for non Google search indexes?
Sorry, got sidelined a bit there.
There are some inititives to try and get some standardisation on JSON APIs, and once again Google is heavily involved: http://code.google.com/apis/discovery. They are using http://json-schema.org, although seeing words like SOAP put me off a little. Their Discovery process powers the API Explorer (http://code.google.com/apis/explorer/) and they automatically generate client libraries from this JSON generated. I’m not sure if that goes far enough as you’re suggesting, but I guess it shows some people are thinking about it.
I certainly liked some of the links you added, particular to HATEOAS and 37 Signals. The 37 Signals post made me think about AHAH (http://microformats.org/wiki/rest/ahah) where HTML snippets are returned to the browser rather than JSON. This keeps the view logic on the server side and the client logic about what to get on the client. I used this on a recent project and it certainly helped having the views only in one place, and making use of partial templates to render snippets to the browser.
Finally, I hope this post encourages people to think about their APIs they are building, you certainly highlighted some of the benefits for that over people serialising out their models.
Cheers
February 29th, 2012 - 14:46
I don’t disagree with you however but it requires a balanced look at why server side generation is used. From experience it’s server side allows scalability of the solution In a way that client side doesn’t and client side even when using frameworks is too error prone by developers in coding up performance problems or browser specific code. I believe that both solutions can exist and need to be weighed up based on the site and client needs
February 29th, 2012 - 20:46
@Robbie Clutton: I really bet this shift will continue to the point where not having javascript enabled will sound as crazy as browsing without HTML enabled (Not to knock any Lynx ( http://lynx.isc.org/ ) fans out there!). I personally don’t think server-side view rendering (especially in the partial rendering scenario) is the simplest way to do things though — we’ve just optimized that so much that it really seems like it must be the easiest way.
@Richard: I don’t see anything inherent in server-side view generation that aids in any sort of scalability in any way. I can only imagine that moving view processing to the client would be more scalable. What exactly were your concerns?
March 5th, 2012 - 16:27
Talking about web applications don’t forget that web is not only API. Still it has to have a searchable HTML content presentation.
March 13th, 2012 - 09:45
I think that Sinatra is just as easy to use for RESTful services as Rails, if not easier. I tend toward Sinatra on the server and KnockOut or Backbone on the browser client.
March 16th, 2012 - 17:03
Hmm, i wouldn t really care wether google can find my server… I should only bother to sell my thick client in all those markets (apple store, microsoft store, google store).
Fact is, is it wise to move the view part into the client? Which and how many platform should i support?
A view on the server might be useful ’cause there are too many client platform but they all have a browser, while writing the thick client in native platform would imply to write code in objective c (apple), java (android), html5 c# visual basic managed c++ XAML (windows 8). Plus you have to subscribe to their market (and charge your customer).
HATEOAS can be delivered in html / xhtml format, add stylesheet and you have an interface good enough for thin and thick client.
April 25th, 2012 - 21:47
We tried the approach you advocate for an app we built back in 2008 called Go. It was all RESTful APIs exposing JSON, and all the view logic was written in JavaScript. The problem was – it was a massive pain in the ass to write automated tests, and the tests we wrote were slow and brittle. Also it slowed down development because you had to change things at multiple layers in the stack, which makes it harder when you’re playing around with ideas during development. We actually moved _to_ Rails because it was easier to test and faster to code.
April 26th, 2012 - 13:07
@Jez: There are a couple of truths in what you’ve said that I can’t deny: (1) It’s a real pain to test highly interactive clients and (2) The separation of concerns slows you down at development-time. To (1), I agree it’d be best to avoid creating a highly interactive client if it’s not necessary to please your users (like Wikipedia). In those cases, it would be silly to recommend a thick client architecture. For applications where users are expecting much more than just the interactivity that comes with links and forms, I think the thick client architecture starts to make more sense. Testing tools are lacking for that style of architecture, but they’re improving at a really impressive rate. Now assuming that you are writing a highly-interactive application, I think that any speed you might get from not separating concerns properly (for my definition of ‘properly’ at least) is really just technical debt being added to a ball of mud. I especially don’t think pjax-style applications are easy to test or maintain, regardless of how fast they might be to write (PHP scripts with SQL next to HTML are fast to write as well). I also think that “API frameworks” and front end frameworks are very immature and as they improve will offset any development-time speed gains you might get from a more muddled architecture.
BTW, your book is great. It’s a game-changer that everyone should read.
April 27th, 2012 - 08:07
yes, but is not slowly moving to the client side.
It’s happening faster than most people want!