Why I’m Sold on Meteor and Node.js

It has been almost two weeks since my last blog post.  During this time, I’ve been working hands-on with Meteor, Node.js, MongoDB and Twitter’s Bootstrap framework.  One of my main objectives was to find gotchas that might disqualify Meteor as a viable solution for upcoming projects.

During this time, I spent quite a bit of time working with HTML5 and CSS3 in the context of Bootstrap.  Meteor,  Node.js and MongoDB are so expeditious at dealing with the core issues such as data storage and retrieval that a larger percentage of time can be spent working on aesthetics, for example a responsive layout that works equally well on both hand-held and desktop clients, and also ensure that the application simply looks good and professional.

I broke down and purchased the fine book Discover Meteor: Building Real-Time JavaScript Web Apps:

This book helped me resolve a key mystery about Meteor, namely how the system determines which templates need to be refreshed when back-end data changes.  Now that I understand it, I have to say that it is awesome, but please bear with me because you’ll need a little background to understand.

A meteor application is comprised of two “things” that must be developed in tandem:

1.  Templates, which are HTML plus special Handlebars tags.
2.  JavaScript template functions, helpers and event handlers.

Meteor templates are cleaner than traditional template-based development approaches like JSP, because there is never any JavaScript code in the template, and the interplay between the template and JavaScript code follows well-defined rules.  Microsoft has gone in this direction with ASP.NET using the so-called code behind philosophy, where most C# or VB code can be in a separate file.

When Meteor processes your template, it will call into your JavaScript code at key control points to retrieve necessary data.   For data-centric applications, the JavaScript code will typically call MongoDB to retrieve data, and that data will be cached in the browser-side Mini-Mongo as JavaScript objects.  The Mini-Mongo data is directly accessible to the templates and is in the necessary format to feed the templates.  There is no conversion overhead.

Now here is the magic: after Meteor processes your template, the system is smart enough to detect whether any of the data referenced by your JavaScript code has been changed, and to dynamically re-render the corresponding template(s).   This automatic update occurs without a single additional line of coding, and is the key to Meteor’s so-called reactive behavior.  Please take note of the word reactive, because I suspect that you’ll be hearing that more frequently in coming years.  This is because Meteor, and similar platforms such as Derby, are poised to change user’s expectations regarding how web apps should behave, and there is no turning back.

But there was one central mystery: how can Meteor know if any of the data elements that happen to be referenced by my JavaScript code might have been changed, requiring that my template(s) be re-rendered?  Meteor solves this problem in an elegant way that is transparent to the developer; moreover, by mastering a few simple concepts, developers can control the underlying machinery to connect to new data sources that are not known to Meteor.

Whenever Meteor calls into your JavaScript code, it first constructs an object called a Computation.     The Computation contains many Dependency instances.  Each Dependency instance represents a single data source that might dynamically change after your template has rendered.  In practice, most of these Dependency instances correspond to MongoDB collections.  The relationship between Computation and Dependency is many-to-many, so a given Dependency may be contained inside any number of Computation instances.

At any point in time, there is a so-called current Computation which is “active” at the time your JavaScript function is executing.

Meteor getter functions, particularly those that involve retrieval of MongoDB data, will automatically add Dependency instances to the current Computation.   These Dependency instances are listeners for any changes in the underlying data source, and each Dependency knows the set of Computations that might be affected my underlying data changes.

Together, the Computation and its Dependency instances comprise a complete record of all reactive data sources that your template references, both directly and indirectly.  In other words, as a byproduct of executing your JavaScript template-rendering function the first time, the system constructs a record of all reactive data sources that are inputs into your template.

Since the relationship between Dependency and Computation is many-to-many, a given Dependency knows the set of all Computations (and associated templates) that should be re-rendered if the underlying data changes.  With the help of Websockets, Meteor listens to data state changes, and selectively re-renders only the affected templates.  This is in contrast to standard JSP, ASP or PHP where the entire page would have to be refreshed on demand by the user, or via AJAX and jQuery calls to patch HTML after the initial rendering, requiring two separate code paths that do essentially the same thing.

Meteor unifies the code paths: the same code that handles the initial rendering handles dynamic refresh, so making a real-time web application carries no additional cost.  Since real-time behaviors are inexpensive, you’ll see them more and more frequently, and static approaches such as JSP will seem clunky and “legacy” by comparison.

I’m particularly excited about the possibilities using Meteor together with HTML5/CSS3 animations, Google Charts and RIA frameworks such as Sencha ExtJS.   Consider these possibilities:

1.  Smooth-scrolling tables of data that dynamically roll as data is added similar to the ending credits of a movie
2.  Google Charts that dynamically update as data is changing
3.  Table and tree controls that are dynamically updated as user collaborate on a shared data model

Such reactive behaviors will likely become hallmarks of modern web apps.  Start-ups can take advantage of these technologies to differentiate themselves from well-established competitors, at least in the near term.

So, if Grails is a carrier battle group, and Ruby on Rails is a rickety pirate ship, I would say that Meteor/Node.js are the Oracle catamaran: minimal, streamlined, modern and super fast to develop.