On jQuery & Large Applications

Posted

Update: I’ve written a separate post on the wisdom of rolling your own large application toolkit that incorporates jQuery.

I’ve been thinking a lot lately about JavaScript applications. As my skills have evolved, I’ve had the privilege of working on more actual applications, and I’ve gotten further and further from clients who want to add a bit of Ajax or bling to an otherwise fairly traditional web site.

The most interesting applications I work on are client-side intensive: the server is responsible for providing data as JSON to the client, and most everything else — templating, state management, data management, site navigation, and of course user interaction — is left to the client side.

It’s a lovely way of writing an application. There’s no need for me to touch server-side code; in some cases I work with a server-side developer to decide what the data they send will look like, but in others I just take what an API already provides and make it work. I get to use the same templating framework across projects, regardless of server-side technology, and I can prototype complex interactions before the server side even exists.

This is a land where HTML, CSS, and JavaScript are almost all you need, and I like it. I’ve become a firm believer in moving giant hunks of functionality that used to belong to the server over to the client. For a variety of reasons, I think it’s clear that this is where most interesting web development is headed, to the extent it’s not already there.


This style of building an application changes the front-end development game. In fact, “development” may no longer be an adequate description; we’re moving into the realm of engineering, here. We’re not using JavaScript to add a bit of bling to our sites — a slideshow here, some Ajax there — we’re architecting an application, damnit. We can’t just write some procedural code that binds a bunch of anonymous functions to some events and call it a day; if we do, I can tell you from experience that we’re going to end up with a steaming pile of unmaintainable crap.

Among a host of questions presented by these sorts of applications, some of the most interesting to me are:

  • What are the units of functionality that will make up the application?
  • How will those pieces be organized into units of code?
  • How will those pieces communicate with each other?
  • How will dependencies between components be expressed and managed while adhering to the principle of loose coupling?
  • How will components manifest themselves in the DOM? Do they need to?
  • How will we persist data across URL and page loads?
  • How will we manage communication with the server?
  • How will we make sure users only see the data they’re allowed to see?

At the risk of making a broad generalization, this isn’t the way today’s average JavaScripter learned to think. The mantra of jQuery, the most popular JavaScript library on the internets, is “get some elements, do something with them” — perfectly terrible preparation for analyzing an application from a perspective other than the DOM. And, IMHO, therein lies a tremendous problem.


As more and more application logic moves to the browser, I’m eager to see the JavaScript community rise to the challenge, but instead it feels like the opposite is happening. People with little understanding or appreciation of these questions are taking on projects that demand these questions be answered. The result is a land of fragile code that gets the job done while giving the finger to the next developer; a land of code so tightly coupled, so deeply beholden to the DOM, so blatantly not reusable or extensible or maintainable as to render every subsequent commit a complete crapshoot, as liable to cripple the application as not. The viability of the project is threatened, and so is the reputation of JavaScript.

We are better than this. JavaScript, even that old-fashioned browser kind, is a language worthy of respect, not a thing to be dreaded. But — and here’s the sentence I have struggled 10 months to realize and an hour to write: in order to prove that we are better than this, we must make abundantly clear to the budding developers, to the project managers, to the enterprises, to anyone intending to build a remotely complex JavaScript application, that there’s more to JavaScript than jQuery. The questions are bigger, the answers more complex, and the relevant skills, alas, a bit harder to come by.

We have to make clear that, in fact, jQuery is but a hammer. When it comes to building these intensively client-side applications, we’re talking about building skyscrapers, for god’s sake. The problems solved by a hammer are the least of our concerns.


It was just a few months ago that I gave a presentation on building large jQuery applications. I emphasized jQuery’s role as strictly a DOM and Ajax tool, and demonstrated a few other tools — John Resig’s simple inheritance, James Burke’s RequireJS dependency management and build tool, Jan Lenhardt’s mustache.js — that one would want to bring to the table for such an undertaking.

But to what end do we assemble said hodgepodge of tools? Is it just so we can continue to “use jQuery”?

jQuery’s API is, indeed, dead-simple, but we are smart people! We are building skyscrapers! When it’s time to write a complex application, and we need all of these things that jQuery doesn’t offer, can we not learn to use another hammer — learn that dojo.place('<div>I am new!</div>', oldDomElement, 'last') means the same thing as $('<div>I am new!</div>').appendTo(oldDomElement) — if learning it gives us access to legions more functionality than jQuery even aspires to provide?

Do we assemble this hodgepodge because finding jQuery developers is perceived as an easier task than finding practitioners of another library, even though someone saying they “know jQuery” is little indication that they will know how to work with the assembled solution?

Do we do it for the plugin ecosystem — full of code of varying quality and maintenance — even though many of the large application needs addressed by those plugins are addressed by other libraries as well, and sometimes better?

And when we do it, when we assemble this collection of tools ourselves, what risks are we accepting? What price will we pay down the road to maintain three or five or 10 different pieces from three or five or 10 different authors, with different release cycles, no guarantee of compatibility or maintenance, and no central project thoughtfully considering their future?


I’ve wrestled with these questions for months, agonizing during sleepless early-morning hours over how to advise clients on the answers. I’m the co-host of yayQuery, a contributor to the jQuery Cookbook, and, I’ll venture to say, a decently respected member of the jQuery community. I did not arrive at this conclusion lightly, and I have few illusions it will be well-received, or even heeded.

But I’ve grown weary of people championing a tool that simply does not answer the big questions I see in project after intensively client-side project. I’ve grown weary of those same people dismissing tools that answer those questions handily and have been answering them for a while now. I cringe when clients tell me they’ve chosen jQuery because it was “easy,” and then watch them predictably struggle with all of the questions it does not answer. And I’ve found I can’t continue to bite my tongue when people recommend jQuery as an enterprise-grade solution while failing to acknowledge these questions, let alone answer them*.


I do not want to see jQuery go away. The simplicity of its API was undeniably instrumental in the rise of JavaScript as a language these last few years. It is a perfect gateway drug, and I greatly enjoy watching people transition from “get some elements, do something with them” to the elegant patterns of JavaScript itself.

jQuery is an entirely appropriate answer to so many questions, but it falls so short for large applications, forcing you to assemble such a tenuous toolkit of your own, that it simply isn’t a viable answer — or, in my opinion, part of an answer — for large applications. If we hope to continue to gain respect as a community, we ought to admire jQuery’s immense contributions, but we must not be afraid to accept and make very clear its limitations. We do otherwise at our peril.


*An aside: To its credit, JupiterIT has put forward JavaScriptMVC, the only substantive attempt I’ve seen at answering these large application questions using jQuery. I applaud them, but fear their efforts will continue to be somewhat isolated without the support and endorsement of the wider jQuery community. If you have read this far and still have your heart set on a jQuery-centric large application solution, you should by all means take a look at JavaScriptMVC.

Posted

70 comments

Aug 09, 2010
nelsonmenezes said...
Great article; I couldn't agree more having developed sites with both YUI and jQuery. I know there are plenty of "grey areas" in between these two extremes, but in simple terms: if you are enhancing a "web page" with a few UI niceties then jQuery is, hands-down, the most productive way to work. If you are developing a "web application" then a "proper" framework is essential (I'd vouch for YUI given the large numbers of very clever people working on it).

This separation fits neatly in my head in the "hypertext delivery system vs software interface" duality of the web model by Jesse James Garrett's.

Aug 09, 2010
 said...
As the developer of a large web app myself I have to say that I love jQuery and use it as it was intended to be used: as a utility where needed. JavaScript is powerful, but it does take careful engineering, architecting, and careful planning to execute something that is safe, fun to code for, and extensible and isn't a bloated dump on the browser download process. I always hate that I learn so many things along a project lifecycle that I wish I had known when I started the web app.
Aug 09, 2010
leebriggs said...
The integration of jQuery into ASP.NET development is a growing problem. For the complexity of applications jQuery is often a poor choice, but is now seen as the Microsoft supported solution.
Aug 09, 2010
PosAbsolute said...
Doing a bit of application myself, and having done a big application with jquery at my workplace, I also doubt about jQuery competency in that matter.

But there is also a factor, I think, you re forgetting. Its the javascript programmers pool in your region. If you re not in the valley or in a smaller city than new york the pool of javascript programmers is greatly diminished and finding talents that want to work with dojo(for example) in comparison to jQuery is really harder.

That is not to say that this will be better for your app. But im "mostly" sure manager think that way. They also think jquery will make them save money in development time. Which is mostly true for website, not so much for applications.

Because there is not a lot of front-end expert in large javascript applications, forming an experienced team is also really hard.

Also if you take .net teams (from what i saw) where they always used .net tools to generate front-end in general, your in for a good headache. When you enter that territory as a front-end dev and want to change things, you are down for a big fight.

...

With thay said, at my workplace we have a system similar to javascript MVC (but a lot less advanced and complicated). We use zend for the backend, and zend load on demand the javascript files associated to modules, so we moved the logic a bit. On top of that we use the module pattern in mostly all our javascript files to keep things organized, but thats as far as we can go for now...

Aug 09, 2010
Excellent article and I concur with the thought behind it. I still worry though that some of the computationally intense may not be feasible on every platform, vis-a-vis mobile browser. For that reason, some emphasis on the back end is still necessary and even preferred.
Aug 09, 2010
josephcorcoran said...
I do wonder what the future holds for JavaScript applications. I think we may well be stuck in wondering mode for some time actually, since the web as we know it in 2010 is almost entirely built on relational databases -- so far not a good fit for js apps at all. NoSQL (as a blanket term) is creeping in...

I'd put forward Sammy as a potential way of structuring JavaScript apps: http://code.quirkey.com/sammy/

I've been using it a lot recently and can't praise it enough. Combined with something like CouchDB it's very powerful indeed. I understand Aaron's long-term goal for Sammy is no longer depend on jQuery, which would be excellent.

Aug 09, 2010
Encosia said...
While I've definitely seen a few debacles that resulted from accumulating too much "loose" code, I have to disagree that jQuery itself is the problem.

My first contact with jQuery was in the context of improving enterprisey, internal ASP.NET development, so I've been wrestling with this from "application" PoV for years. Ultimately, I've found that as long as you approach client-side development with good software engineering practices in mind, jQuery doesn't fall short at all. Things like custom events to decouple your code and custom plugins/selectors to improve DRYness make a world of difference, for example.

Aug 09, 2010
Rebecca Murphey said...
@Encosia I don't think jQuery itself is the problem, necessarily -- it's the perception among people who don't know better that jQuery is the default answer to all questions JavaScript. The approach promoted by jQuery -- so appropriate to plenty of questions -- is wholly inappropriate when it comes to large apps, and as a community I think we have an obligation to do some education on this.
Aug 09, 2010
fractastical said...
Seems to me the problems you mention would be enhanced by combining multiple Javascript libraries. And I think ultimately what you should say is that Javascript itself is not ready to be the default language for large app dev. I think the common usage of jQuery (I'll just use a plugin to enhance such and such) is actually right on the mark -- like it or not.
Aug 09, 2010
Rebecca Murphey said...
@Joel Dietz See, there are actually libraries out there (Dojo and YUI come to mind) that do solve these problems, and elegantly. There's no need to combine; there's a need to learn the right tool for the task. jQuery is that tool for simple enhancements, but there are other mature, robust tools for application development. The problem isn't JavaScript; the problem is people who don't know how to use it.
Aug 09, 2010
Kevin Dangoor said...
I wrote a longish reply in which I more or less agreed with you wholesale: jQuery does nothing for "programming in the large" which is potentially a barrier for some folks who need to do so.

I just thought I'd mention that the JavaScript language itself seems likely to get modules in the next edition. That will help reduce fragmentation somewhat and the browser can do some nice things with module loading that a library cannot. So, one part of the "programming in the large" puzzle seems likely to be dealt with along the way.

Aug 09, 2010
PosAbsolute said...
if I can add another thought,

jQuery, even with the UI add-on is not solving as much "application problems" as dojo or YUI

Dojox charting and mobile are good examples

and outside of UI, the plugins quality, in general, is greatly diminish and do not always follow the same standards (giving the DOM caller back comes to mind)

Aug 09, 2010
iammerrick said...
I saw your presentation at jConf a year or so ago about using object literals to organize your jQuery and I could tell your approach to writing jQuery was entirely different then the majority of "jQuery users".

I couldn't agree more with this article. I am personally a MooTools fan focusing primarily on applications. I get frustrated each time I here a client demand jQuery for their application because its the latest buzzword for "bling".

The JS community is so much larger then jQuery. And the talent thereof is overwhelming. I applaud this post.

Aug 09, 2010
mrDarcyMurphy said...
I sympathize, and run into these questions daily. What are your thoughts on a total departure like Cappuccino?
Aug 09, 2010
 said...
I love that the modular aspects of javascript - not being addressed with jQuery (as a community) - are finally being addressed. I choose to work in Mootools for the Modular aspects - yet am forced to work in a jQuery world. Moo4Q.com helps in making using jQuery bearable.

Thank you for bringing up this point.

Aug 09, 2010
Great article and good points.  jQuery was my gateway into JavaScript application engineering.  I am using jQuery more for the tool it is, and understand how to build large clientside applications with JavaScript as a whole.  Microsoft's adoption of jQuery as opposed to YUI, Dojo tells me how they view js still, and it becomes a double edged sword.  I have a large asp.net project now that is moving towards being a more  clientside app.  Would a framework like Dojo,YUI be a better fit from an organizational point of view? Possibly.  Will those be accepted against a Microsoft supported library for an asp.net app? Probably not?  So, My plan is to be aware and use and promote the engineering principles(like prototypal inheritance and event driven programming) that jQuery does not include (handrolled or using tools beforementioned).  I would suggest the same for jQuery community leaders.  Show the community howto organize/test code in the light of the right thing to do, not as how to do it in jQuery.  Then show how other frameworks have this built in.  Serious is app devs will learn this and have a knowledge of which tools/techniques are best for each type of app. With articles like this and other leaders talking about it, this is happening.
You guys keep up the great work.  You and the js community(you too Pete Higgins- :) ) have been a tremendous help to me being a better js dev.
Aug 09, 2010
mgkimsal said...
I've said for years now that jQuery is the PHP of the JavaScript world, with all the pros and cons that brings. The article here seems to illustrate that point even more. The PHP learning curve makes it very easy to get started and building stuff without understanding the underlying architecture that will be needed for larger apps.

Like jQuery, there are a host of problems that PHP is good at solving. There's another set of problems that PHP is not great at addressing. I also say this as someone who's made my living from PHP for the better part of 14 years. I don't disparage it, nor do I disparage jQuery - I'd simply like to see more people using these tools have a better understanding of what they're doing.

In the short term, when people develop poorly designed apps, it rather guarantees more work for me down the road when they run in to problems. However, I've grown tired of being a digital garbage man, whose primary purpose is to come in an clean up someone else's mess. I suspect Rebecca has grown tired of this (to whatever extent she's engaged in that type of work), probably much faster that I did.

Aug 09, 2010
Rebecca Murphey said...
I want to add a few things after some conversations I've had about this post. One thing I keep hearing is that people don't mind assembling their own toolkit, choosing individual tools that do individual things well, and jQuery is one of those tools.

This may be an intractable difference of opinion, but I really do not believe this approach makes sense from a maintenance point of view. It is simply too fragile. If jQuery itself wants to say "here's the jQuery large applications toolkit, and we're going to support it and maintain it," then good! But individuals hacking together their own solution consisting of disparate pieces maintained by (admittedly very, very smart) individuals ... I can't see how that's a good idea, not when carefully considered, well-supported solutions exist.

I *wish* jQuery would answer this question, or get behind JavaScriptMVC, or something! But asking people to bring their own templating, inheritance, data abstractions, dependency mangement, build tools, and more -- and leaving those decisions in the hands of one developer after another after another -- it doesn't make sense to me. Smart people have thought about these questions and put together really good, really well-thought-out answers. We should use them. And maybe jQuery would do well to offer a solution of its own.

Aug 09, 2010
davemcfarland said...
Thanks for the post. Could you go into more detail on what YUI or DOJO are doing that jQuery doesn't (when it comes to supporting large scale apps). From the post, it sounds like the lack of templating support, and ways to manage dependencies are serious problems with jQuery. What else would you like to see added to jQuery so that it is suitable for large-scale app dev? Or, is there some fundamental problems with jQuery that can't be overcome?
Aug 09, 2010
mgkimsal said...
You're so spot on with that last comment Rebecca. I won't even begin to tell you how spot on. The issue of combined toolkits to address large app development have been addressed numerous times by other communities, and long term this approach doesn't work very well for the community as a whole.
Aug 09, 2010
Rebecca Murphey said...
@davemcfarland A short list of things Dojo has that jQuery does not: dojo.declare, dojo.provide/require, per-spec array utils, dijit._Templated, dojo.data, dojox.rpc.Service, string caching, dojox.Grid, dijit.layout, and on ...

But more to the point, Dojo is an integrated toolkit built for developing applications. jQuery is a DOM and Ajax API. It's apples and oranges, which is kind of my whole point.

Aug 09, 2010
andrewdupont said...
I agree that the community would be well-served to look beyond jQuery for solutions to the problems faced by large applications. But you say, "I wish jQuery would answer this question." I think that's a bad, bad idea.

First: the jQuery API is fantastically suited to DOM-type stuff, but gets ridiculous very quickly when applied to other sorts of things. (See `$.proxy`, `$.inArray`, and so on.) I suspect any efforts to shoehorn even more stuff into this API would be met with resistance.

Second: even if the jQuery team were to say something like, "By the way, we like JavaScriptMVC for large-scale app architecture," it would feel more like an _ordaining_ than a _recommendation_, and would only encourage adoption of the ordained toolkit for cargo-cultish reasons. Better for users to assess those tools on their own and pick what works best for them.

The optimist in me would hope that anyone who _needs_ sophisticated stuff out of her JavaScript is also savvy enough to do her own research. In the real world, not everyone is that savvy — hell, JavaScript is everyone's second language — but all they need is a push in the right direction. They don't need jQuery to make the decision for them.

Aug 09, 2010
 said...
To wax poetic, as JavaScript engineers we are visionaries who realize the potential of the materials we work with, see where others have gone, and seek to go further. Classically trained or not, realizing these goals will require technique and discipline greater than that of those before us, but we also have their success and failures to learn and start from.

Additionally, the excellence of jQuery as a tool, like any tool, provides no commentary on the design and architecture of its creations. Hammers may or may not have been necessary to build the Eiffel Tower, but it is in the realm of the engineer to realize these limitations. As engineers, we will envision creations beyond the capabilities of the tools available to us. If necessary, we must create these tools to realize our dreams.

For a little more practical perspective on the issue, recently a member of Dallas.js, Matt Shannon, asked me about JavaScript scaling and architecture. I've included my response here:

"Matt,

There appears to be a disconnect in JavaScript as compared to other programming language with regards to layers. I think the concept of a JavaScript BLO (Business Layer Object), DAO, and Presentation Layer (Object or not) is something that quite frankly does not exist within the mind of most JavaScript developers. The ramifications of this are obvious; data storage, business logic and DOM interaction logic are all muddled together making decoupling into appropriately organized layers extremely difficult. Further complicating this is the additional tendency to create highly coupled JS/CSS/HTML code to the extent that much JavaScript is organized within the scope of its respective HTML and not with respect to its logic or functionality.

In summary, the JavaScript community suffers from a lack of software engineers and an abundance of hobbyist and web designers. This isn't all bad as it is this unencumbered lack of industry opinion on the "right" way to do things that lends so much freedom to innovation. However, that being said, there are many "right" ways to do things that are just that b/c they are proven to work well, like what I stated above. However again though, as I wrote about BLOs and DAOs in JavaScript I felt myself cringing at this reliance upon paradigms rather than embracing the full potential that an unencumbered scripting language like JavaScript holds. I'd like it to the move from theatre to motion pictures. Nothing prevented a one-to-one relationship and transposition, but time soon proved a much greater potential in the latter."

Aug 09, 2010
ryanflorence said...
> but I really do not believe [the mixing-frameworks] approach makes sense from a maintenance point of view. It is simply too fragile

As the father of the red-headed script moo4q I agree (for those that don't know, check out moo4q.com: combines MooTool's inheritance API with jQuery's DOM API, and automatically extends the jQuery prototype with a MooTools class, much like extending Element.)

I actually call moo4q my javascript hot-sauce, making any jQuery application I inherit palpable, but I always feel a little bad for the next guy and try to document my madness as much as I can.

Case in point: MooTools' Array.each and jQuery.each

foo.each( fn (item, index) ) // Array.each
bar.each( fn(index, item) ) // jQuery.each

Wha? ...

Also, `bind`. MooTools has the object oriented approach to change the context of a function with `Function.bind(context)` which is equivalent to the jQuery functional approach, `jQuery.proxy(context, fn)`. Oh wait, jQuery does in fact have `jQuery.bind` but it's equivalent to MooTools `addEvent` and the DOM's `addEventListener`!

I would never expect somebody to choose moo4q for a sophisticated javascript application. You have to use just one library/framework or stick to Vanilla JavaScript. Anything else is whacky.

jQuery's DOM tunnel vision is why I rarely choose it for a website--if you're going to help me with JavaScript, help me everywhere so that my application can scale.

Aug 09, 2010
paul_irish said...
> The optimist in me would hope that anyone who _needs_ sophisticated stuff
> out of her JavaScript is also savvy enough to do her own research

Me too, but you and I know that's too optimistic. Most developers (and their bosses) need recommendations and blessed paths. At the very least, they need to know what the possible options are.

The jQuery project could spell out those options.. "So you want to build a large app with jQuery? Awesome. Here are some solutions to consider..."

* the Widget factory, resig's simple inheritance, pub/sub, mustache templates, requireJS, sammy - choose your application components a la cart.
* JavascriptMVC - add a more defined architecture style with plenty of tools and library code to support that
* Dojo with Plugd - Start off in Dojo with a jQuery-ish API and the ability to scale up to the whole framework.
* fuck it all and go use sencha, cappuccino, sproutcore, etc.

Coming up with the *right* recommendation for this audience is probably impossible. But I also hate to hear my colleagues and folks like Dustin Diaz essentially hitting a brick wall with seeing how to scale up after you've reached the bounds of the jQuery API.

I want to see this options at least published so we can begin to discuss how to scale up jQuery developers sense of larger app development.

Aug 09, 2010
 said...
If you want to read a bit about building ridiculously maintainable large-scale apps with MooTools, Cloudera really brings it home FTW: http://is.gd/eaoFB
Aug 09, 2010
justinribeiro said...
> I *wish* jQuery would answer this question, or get behind JavaScriptMVC, or something!

John Resig has commented on design patterns before: http://groups.google.com/group/jquery-dev/msg/b4a91084a1299714 If you've got time, the entire thread is a fairly interesting read.

Aug 09, 2010
unscriptable said...
I doubt I'm the first one to predict this, but it seems to me that 2010 is the year that javascript graduates from ajax frameworks to web app frameworks. So many of us are working on the same problem. Something good has to come of it.

It's a good time to be a javascript coder. :)

Btw, for the record, javascript is my primary language (ahem Andrew), my favorite language, and the basis of my career.

Rock on, Javascript.

Aug 09, 2010
yeah_its_me said...
This is easier now that Sizzle won.

Given the wide adoption of Sizzle, many other libraries have now incorporated jQuery-like functionality, in addition to their own organizational and structural features. So you can have things like the jQuery-YUI Rosetta Stone (http://carlos.bueno.org/jq-yui.html), creating a fairly easy path to switch from one library to another. When it comes to core DOM manipulation tasks, it seems like the differences are more syntactical than conceptual at this point.

I think this is great, since it means that developers can jump into a new library and get started with what they already know, learning the other features over time. Hopefully it will lead to more developers branching out and picking the right tool for the job.

Aug 09, 2010
paul_irish said...
Ditto on reading the thread justin just linked to. The thread is a long, but really good, discussion of these issues.
Aug 09, 2010
ryanflorence said...
@unscriptable

> "but it seems to me that 2010 is the year that javascript graduates from ajax frameworks to web app frameworks"

Because MooTools, Dojo, YUI, Ext and Prototype aren't already general purpose application frameworks? jQuery is the black sheep here.

From Rebecca:

> When it’s time to write a complex application, and we need all of these things that jQuery doesn’t offer, can we not learn to use another hammer

Hammers that already exist long before 2010.

Aug 09, 2010
justinbmeyer said...
Rebecca, I couldn't agree more. I spent the last few hours putting together a response

http://jupiterjs.com/news/why-jquery-needs-javascriptmvc

Let me know what you think.

Essentially, I am begging the jQuery team to look further into Enterprise needs.

John mentions writing JavaScript for a long time, and I have no doubts on his talent, but has he (or anyone on jQuery's core) written an enterprise JavaScript app supervising a large team of mostly untrained JS developers?

The conversation got side-tracked into problems with classical inheritance. I would be happy removing it for another way of easily extending a plugin.

In fact, I tell almost everyone to avoid inheritance, and instead focus on mixing multiple controllers on an element and having them communicate with custom events. This provides a much better separation of concerns.

However, this is very difficult with something like jQueryUI's widget factory which doesn't handle cleaning up events handlers.

What concerns me is that in the year and half since I made that post, jQuery has done almost nothing to address the vast majority of issues I posted and Rebecca enumerates.

I don't think that this is anyones fault, it's just that jQuery and people who use jQuery have focused on different problems

But this is changing!

What's important now is to work together instead of fragment. I really like some of sammyjs's stuff and am planning on adding it to JavaScriptMVC 3.1. I'd happily drop JavaScriptMVC for a more collaborative effort; but unfortunately, no one has talked to me. I feel like the guys from dojo - slowly watching everyone in jQuery implement everything that JavaScriptMVC has already done.

Aug 10, 2010
codylindley said...
I just don't think there are easy answers here. Should we use the jQuery library with XY&Z or go with a Dojo/Qooxdoo/ExtJS/Backbase/Closure complete framework solution?

Like most things, its comes down to the variables associated with each solution/project. I think jQuery with XY&Z can work!

The question is what sort of problems do you want to have. And who have you trusted to answer these problems. After all how this question is answered comes down to what sort of engineers are solving the problems. jQuery with XY&Z can do everything a framework can do. Peter Higgins could create Gmail with jQuery and his "its just JavaScript" slogan...and it could be brilliant.

At the end of the day I gravitate towards jQuery because of what I know of the reality of web application development. Change is always the same!

My observation is that large JavaScript frameworks are great at what they do and down right horrible and what they don't do. The more we organize and plan a framework of solutions the more ridged things actually become.

Sure, jQuery + XY&Z has its issues but its these issue I want to wrestle with as opposed to moving a mountain. I'd rather move little mountains around to create a work of art and forgo the paint by numbers. To me jQuery application development is more like a game of chess rather than a game of checkers.

Its change I fear the most and from what I have be able to determine a framework of solutions only offers a skin deep security as it pertains to maintainability. Try and draw outside of the lines and all hell breaks loose. For me, I lean towards a blank canvas to spin a work of art. I like mixing the colors and all the headaches that this entails.

Everything you are suggesting is great. In fact I think you should just scratch that itch and recommend/use a Dojo/Qooxdoo/ExtJS/Backbase/Closure when you see fit. I see nothing wrong with that. But I also see nothing wrong with using jQuery with XY&Z. At the end of the day all solutions still suffer from the same major issue. Was the hammer swung correctly.

Also based on your comments, I'm going to assume that you are a big fan of AppendTo. This company is working on providing solutions to the exact issues you are raising. This company, from what I can tell, is trying to forge better XY&Z tools.

Aug 10, 2010
andrewdupont said...
@paul_irish:

"Most developers (and their bosses) need recommendations and blessed paths."

I don't agree with this.

"At the very least, they need to know what the possible options are."

Aha! But I do agree with this. There are many in the jQuery community — there are many in _this here comment thread_ — who have slipped the surly bonds of jQuery and are wiser for the experience. It would be nice to read blog posts that elaborate: "This combination of libraries worked well for us on this project."

For some reason, this conversation doesn't seem to be happening. Maybe people are moving to YUI or Dojo, instead of arranging other specialty libraries around their existing jQuery. Maybe they're quite successful at building large-scale apps that feature jQuery, but their bosses won't let them talk about it on blogs and whatnot.

But (and here we might need to agree to disagree) I can't imagine it's because the jQuery project hasn't spoken with a single voice to tell all its users which specialty libraries get the jQuery seal of approval.

So, in summation: _listing options_ is good, and triggers conversations about about pros/cons, what works best with what, etc. But _an endorsement_ is bad because it would bypass those conversations entirely. I can't tell which Rebecca is arguing for here.

P.S.: Dustin and I have discussed this very topic before, but I don't know if I'd use the phrase "brick wall." After all, he's written a book on JavaScript design patterns.

Aug 10, 2010
schwoortz said...
The fact that jQuery is so easy to use has led to an explosion of new sites (even very large projects) that use simple dhtml-ish effects as well as ajax and other stuff extensively. Now everbody needs "js-developers". What happens? Just find someone "who programs is jQuery" get started. The first couple of things work fine, but as things get more complex, that person can't handle it. We really need more developers who understand javascript much more thouroughly. BTW there are more and more of these people, and they will always find good work. But there are also very many "jQuery users", who are involved in projects way beyond their capabilities.
Aug 10, 2010
Mark said...
It's great to hear someone of your caliber share their opinion on this issue. I recently built a reasonably large app (my first) using jQuery only to find that my code became quickly disorganised and as a result the application became slower with each new addition.

After reviewing some of the larger libraries out there I decided to go with YUI 3 and pitched it to my employer. At first the learning curve great, not only from the perspective of learning a new library but also in learning how to build an application properly. I lost many hours of sleep wondering if had made the right recommendation.

I'm glad to say it was a rewarding experience. I learned a lot and ended up with an application that performs significantly better than it's jQuery predecessor, is better structured and is far easier to maintain.

I did however migrate a couple of jQuery plugins over the new app, namely jqTransform and Flot. At first I had reservations about it but given time constraints had little choice. In the end I found that YUI 3 and jQuery can actually complement each other. YUI takes care of the core application architecture while jQuery provides some tactical value adds through the miriade plugins developed by a large developer base.

Aug 10, 2010
Rebecca Murphey said...
@andrewdupont I'm really hoping this post is going spur good conversations about what does work. I know lots of people have differing opinions on what toolkit makes the most sense, whether it's a home-grown framework that does exactly what a project needs while still including jQuery as a component of it, or a larger off-the-shelf framework that has the tools to answer (almost) every question under the sun, at the possible (but I think this is arguable) cost of having to buy into doing things a certain way.

I think the first part of that conversation, though, is establishing the baseline fact that the questions are bigger than the ones answered by jQuery, and that's what I set out to do in this post. This fact may be stunningly obvious to people who have been thinking about it for a while, but I promise you it's not stunningly obvious to plenty of others, who answer "jQuery!" to every front-end development question under the sun. Getting that premise out there, and getting a raft of smart people to nod their heads, is the first step in getting people to realize they need to be thinking about these questions when they're taking on non-trivial projects.

Aug 10, 2010
 said...
In case you haven't seen it, you should go checkout an excellent video on this topic, though not on jQuery specifically:

Video: Scalable JavaScript Application Architecture
Nicholas C. Zakas

http://developer.yahoo.com/yui/theater/video.php?v=zakas-architecture

Aug 10, 2010
Rebecca Murphey liked this post.
Aug 10, 2010
chiborg said...
I have given Claypool.js (http://www.claypooljs.com/) a shot and it worked quite nice. Clypool is a jQuery-based MVC framework that works ond the client and the server. Whenever I do more than simple DOM manipulation on the client, I use Claypool.
Aug 10, 2010
Rebecca Murphey said...
A tangential but related conversation (and my response): http://www.garann.com/dev/?p=186.
Aug 10, 2010
jeremyckahn said...
I think that a part of the problem lies in how a lot of us were taught to program. A lot of web developers didn't major in Computer Science, which isn't necessarily a negative. However, because of that, they may not have had formal software development practices beaten into them like a C++ programmer might.

Since jQuery makes everything so easy, planning for something to be robust, maintainable and scalable often becomes secondary. I think as the nature of "serious" JavaScript development evolves and becomes more obvious, this problem will be alleviated.

Aug 10, 2010
writeson said...
I'm fairly new to both JavaScript and writing larger client side applications, and I agree with Rebecca's statement that using jQuery alone is not a good solution to building these applications. My first foray into jQuery yielded something that resembled Visual Basic spaghetti code FAR to much. Since then I've developed much better code, I wouldn't say it's a framework, more convention and style than anything else, but it has been working for me as my applications have grown larger.

There are quite a few reference samples that demonstrate how to use jQuery well, but I think there is nothing comparable for building larger applications. I know that personally I learn best by example, but I have yet to see an example of a larger application written in JavaScript. At least not an example that I'd like to emulate. I've looked at JavaScriptMVC, but I guess I'm thick, I'm just not getting it. Rebecca, you mentioned other tools like Dojo being better alternatives, or providing better tools, than jQuery. Can you say why you think that, or point to examples that show that?

Thanks!

Aug 10, 2010
markstickley said...
Great article, thanks and so, so true. I don't know if you've come across the BBC's own JavaScript library, Glow. It's really rather good and with version 2 in Beta it looked like it had a bright future ahead of it, until until it got targeted in the latest round of budget cuts and it was announced that we are to use jQuery instead.

Virtually no-one is happy about the move aside from those who don't really know JavaScript anyway, they only knew jQuery.

It just goes to illustrate how deep set the opinion is that jQuery is the awesomest thing that was ever awesome, certainly in the world of JS frameworks. That the BBC are willing to halt development on a genuinely great framework in favour of another which is outperformed by Glow 2 in virtually every respect is a baffling concept.

Glow on GitHub: http://github.com/glow

Aug 11, 2010
Rebecca Murphey said...
I want to also call attention to the related Reddit comment thread: http://www.reddit.com/r/javascript/comments/cz4sy/on_jquery_and_large_applica...
Aug 11, 2010
metaskills said...
This reminds me of a pattern I coined called the AJAX Head Design Pattern for websites. It's also one of the reasons I like to structure my large "classes" of JS objects (YES I SAID CLASSES, don't freak out) using Prototype. Link here http://metaskills.net/2008/5/24/the-ajax-head-br-design-pattern
Aug 11, 2010
bolinfest said...
As I argue in the Preface of Closure: The Definitive Guide, this is exactly the type of development that Closure is designed for: http://my.safaribooksonline.com/9781449381882/preface. As you can see from the Closure Library, it has quite a track record as the toolkit for many serious web applications: http://code.google.com/closure/library/
Aug 12, 2010
andrewcacioppo said...
Great article!

I've spent the last 10 years or so building rich client applications in Flash/Flex/AS3. Currently, I'm working on an js/ajax client app and have been frustrated at jQuery's limitations. I don't consider it javascript "framework" at all, but merely a toolkit for dom element selection/manipulation. It provides only a small portion of what an application engineer needs from a framework.

I would much rather use YUI3 which is a much richer frameworks and incredibly well put together. I love the fact all their code is sandboxed and modular. Very pro.

Aug 13, 2010
i.m. ruzz said...
I agree.. I think a good many of the intangibles were very well laid out in this article: http://blog.supermatter.com/post/867724597/what-i-want-from-a-javascript-fram...

Right now, I'm writing a largish app with a php/symfony backend and a jquery/sammyjs frontend: http://code.quirkey.com/sammy/

it adds really decent routing on page based hashtag ("ala gmail") and several other handy larger app features, but it fails to cohesively tie everything together, so I've been slowly building up a core app that integrates sammy and jquery as well as more generic app stuff. but it's still a long way away from supermatter's list.

one of the really badly needed (from my view) parts of a framework is CRUD generators that tie to DB/ORM layers and generate js templates. There's just no way i can afford the time it takes to write all my forms and master/detail pages by hand--customization already takes too long without the initial build--and if i go to straight js templates, i lose symfony giving me a jumpstart on those pages--a loss that would be the difference between making money on a job, and losing.

the addition of sammy hasn't solved all the problems, far from it, but i think its a big step forward, and i'm really surprised it's not more popular, or widespread. I think you should give it a look ;)

Aug 13, 2010
danielknell said...
I have recently been voicing similar concerns and I find the single minded resistance of many non-engineers in the javascript world to consider technologies other than jQuery quite frightening.

But of greater concern is the amount of people who work in our industry who do not actually understand javascript as a language, and who's skills are confined to the scope of a single framework.

Unfortunately javascript seems to have many more buzz word yielding marketing types meddling in its affairs than other programming communities.

It has been long established that there are no silver bullets, so can we all get back to using the right tool for the job now please?

Aug 16, 2010
sminnee said...
I'm curious to hear what you think of the approach that jQuery Entwine takes to organisation of larger codebases - http://github.com/hafriedlander/jquery.entwine

Essentially, it maintains the jQuery style of using CSS selectors as its primarily way of specifying what goes where, but introduces inheritance, methods and properties. It uses selector precedence as an alternative to inheritance. It's based on the Concrete UI concept.

Aug 20, 2010
angusTweets said...
Thanks for elegantly articulating the thought at the back of the minds of all seasoned front end developers. Frameworks are not magic, just tools. Good JavaScript architects/coders are not dispensable, they're essential.
Sep 01, 2010
yozefff said...
Wow .. thanks for this great article! Being a YUI developer myself and build webapps al the time, I wanted to try out jQuery. At first I loved it, very nice API and neat plugins. But then I wanted to try the application approach. Then I realized, jQuery is more for eyecandy framework, instead as an application framework like YUI. For example, I would LOVE to have custom events just like in YUI to see in jQuery. Or the class based like PrototypeJS (although I found a nice class plugin Digg uses for jQuery). I also hope jQuery can find a answer for this "problem".
Sep 01, 2010
justinbmeyer said...
@yozefff jQuery has custom events. See trigger. Also, check out http://jupiterjs.com/news/a-simple-powerful-lightweight-class-for-jquery
Sep 02, 2010
yozefff said...
@justin ... I know the trigger function. But maybe it's my inexperience (just new to jquery), I only see triggers originating from DOM elements.

But I want triggers from Class object for example

Class X has a variable wich subscribes to a custom event in Class Y (http://developer.yahoo.com/yui/examples/event/custom-event.html)

I'll give the jupiterjs a try .. for now I'm using the Class from Digg (http://code.google.com/p/digg/wiki/Class)

Sep 02, 2010
justinbmeyer said...
What about OpenAjax.hub?

Sent from my iPhone

Sep 27, 2010
goleftcom said...
Hi Rebecca,

I was going to go the jQuery route for developing JS based apps but after reading your posting I'm going to have another look around. That is very helpful before going off the wrong route.

I've been writing software for more than 20 years now and one thing that struck me odd in your posting is that you make it sound as if a framework has less limitations than a toolkit like jQuery. I think they just have different limitations and applications are no longer built for eternity.

The thing I like about the jQuery approach is not that it is simple but that I do not have to learn a framework because every framework I have worked with has limitations of its own and at some point you have to start working around them.

The more complex a framework gets the steeper the learning curve and the more complex gets the workaround as you have to guess the developers intentions how problems are to be solved within their framework.

I've spent a lot of time trying to work around problems in frameworks and given the explosion and the mind-boggeling pace at which the JS framework development happens I have my doubts that any of them can call itself mature. Building good frameworks is difficult.

Let alone to decide whether the framework you chose won't be superseded by something even cooler and your choice will no longer be supported.

So with jQuery at least I can get back in the drivers seat and I'm in control as there is little to learn.

Just my $0.02
Frank

Sep 27, 2010
Rebecca Murphey said...
@goleftcom I think it's important to differentiate between "frameworks" and "toolkits." Toolkits don't prescribe patterns; rather, they provide a set of tools. Frameworks, on the other hand, do enforce patterns, or at least make them difficult to work around.

Solutions like Dojo and YUI, I think, are accurately described as toolkits, as they provide a set of tools that you can choose to incorporate into your project or not -- they don't provide a ton of guidance about how best to use those tools to actually create an application. On the other hand, a solution like JavaScriptMVC is probably more accurately described as a framework, as it prescribes a fairly particular approach.

I agree that there are times you may not want a prescriptive framework; on the other hand, such things can be invaluable when they are appropriate to the problem being solved, especially when larger team -- often including people with varying skill levels -- is working on the project.

Sep 27, 2010
justinbmeyer said...
Those are pretty good definitions. I'd like to say that JavaScriptMVC has a foot in both camps.

It's MVC name, I am realizing 3 years later, is something of a misnomer. It's more accurately described as:

Ajax / Service wrapper
jQuery widget Factory
Client Side Templates

It also provides tools for testing, code cleaning, documentation, etc.

All of these parts work stand-alone (without the other part).

JavaScriptMVC doesn't proscribe traditional MVC style binding (Model to View), although you can build that way if you want.

If you are trying to use everything together, it does try to force you into a particular folder / file structure. You don't have to use it, but it's code generators make it easy.

JavaScriptMVC is more "framework" than Dojo, but you can still pick the parts you want. It doesn't have nearly the extensive toolkit that dojo has (it contains nothing with a graphical component), but it's really good at what it does.

Sep 27, 2010
Rebecca Murphey said...
@justinbmeyer You are right, when I call JavaScriptMVC more of a framework, I guess I'm referring more to the framework-y pieces of it :) That it encourages a certain folder structure or a certain means of defining event handlers within a controller, for example, is a *good* thing when a project can benefit from that sort of guidance.

These same (basic) constructs could be achieved using toolkits, as well, but they aren't baked in -- they have to be created. Contemplating whether a team is up to that task -- that is, whether they have the knowledge required to make good decisions about those sorts of architectural questions, and then to implement those decisions -- is an important part of deciding which route/approach/solution is best for a project.

Often when people say that they don't want to use a prescriptive solution, what I hear is that either they think the prescriptive solution is more complex than their actual needs, or they have confidence in their ability to implement a custom solution that will suit their needs better. Either or both of these may be true, but I'd encourage them to thoroughly investigate a solution before deciding it's too complex -- nothing says a project has to use every single piece of a solution from the get-go! -- and to honestly assess whether the project will be best served by them trying to craft a well-considered solution themselves.

Oct 06, 2010
Vladimir Galajda liked this post.
Oct 13, 2010
xMartin said...
Interesting and inspiring article. At my company we're using the Dojo Toolkit for our large webservice-based Ajax app and sometimes wonder if development wouldn't be easier and faster and it would be easier to find developers if we used jQuery. I don't have any experience with using jQuery for large apps and I do see Dojo's strengths and after reading this I'm more confident may daily work is heading into a good direction.

I like this comparison jQuery = PHP in a way. I thought and talked a lot about PHP and why it's popular and what it's good for and it kind of fits this discussion. Still, jQuery is much more beautiful and nice, though.

I think it's really funny that people here agree on the point of view in the article and then tell there favorite additions to jQuery to make it better fit their needs ;)

I really only know the Dojo Toolkit (just did the bling thing with jQuery before) so I can't really compare, but I can say that Dojo is very good in letting you choose which part of it you wanna use and which not.

Nov 18, 2010
Johan Steenkamp said...
This sounds a lot like the numerous discussions that happen/ed in the Flex RIA community.

Without delving into the history I think its fair to say that the Swiz framework has rapidly become the most popular.

Can we benefit from the approaches Swiz has taken to develop a similar micro-architecture for JavaScript?

http://swizframework.org/

Jan 12, 2011
themikerobi said...
Followers of the jQuery Mantra: "There's a plugin for that", cease to be programmers.

I was never a true believer, but was ostracized by my peers, and was forced to drink the cool-aid.

After two years in the cult of Resig I am no longer a programmer. I am a quilter; I take an assortment of patches and stitch them together, resulting in a flammable cloak that only a circus performer would be proud to wear.

Jan 13, 2011
goleftcom said...
Four months down the road developing a more or less desktop style application in JS I would disagree with @themikerobi and also with the general direction of the original post. I had a look at YUI, Dojo, Ajax.com as well as JavaScriptMVC. I've started out building the app like Nicholas Zakas wrote here (http://www.slideshare.net/nzakas/scalable-javascript-application-architecture) but abandoned the approach after a while as it is a good idea on paper but in reality you just end up duplicating APIs multiple times that are already there.

jQuery certainly isn't the one to rule them all but neither are the others. In fact jQuery brings a lot to the table and in my case I had to write only two additional things:

1. A Core object which handles the bootstrap and some global things
2. A simple jQuery plugin which implements application wide events which are not related to a DOM node. (like $.trigger())
3. Dependencies are managed with requireJS
4. All UI parts are written as a jQueryUI widget packaged as a requireJS module

The key really is the app events as they allow for the cross plugin communication loose-coupling style. There are no hard-wired dependencies and the code is nicely separated into separate modules which all deal with their own concern. Nothing to this approach is specific to my app.

It takes a programmer to build something like this but this isn't rocket science and saying that by using jQuery you're no longer a programmer means that you've given up.

My stance with the other frameworks are:

YUI 2.8: good documentation, complex api
YUI 3.0: nice approach but not there yet
JavaScriptMVC: Website broken half of the time and the examples didn't work. They also really focus on the MVC stuff which is not necessarily what complex webapps are all about
Dojo: Same api problem as YUI 2.8. Too much desktop style
Ajax.com: Even worse

Building JS apps requires dealing with the DOM and that what's making apps so fast. Developers love adding layers on top of layers to abstract things away until they don't understand anymore how it all works. I'm just being reminded by that by writing the server part using Spring - which I love, once I got through the initial setup nightmare.

My take is that you can already write maintainable desktop apps with JS and jQuery today and the very clean and lean API of jQuery makes it fun. There isn't so much missing as you might believe after reading this thread. However, you are free to shoot yourself in the foot as well. But I guess you can do that with any of the frameworks.

Also don't underestimate the power of the plugin ecosystem as it shows that people care and devote energy to this. This also means that other people are going to work on improving the situation. This is better than having a potentially brilliant solution that hardly anyone uses.

Yes, there really is always a plugin for this and I must say I love that.

Just some more random thoughts... :)
Frank

Jan 13, 2011
justinbmeyer said...
On JMVC, I've not had a downtime report for 6 months. Which examples don't work?  And jmvc is not at all about the mvc stuff. And it's mvc stuff is not at all traditional mvc. It's a traditional model, but it's controller is just a jquery widget factory, and it's views are just client side templates. 

If you are going to evaluate a framework, please evaluate it.

Jan 13, 2011
goleftcom said...
@justinbmeyer: Hi Justin, let me rephrase.

When I've tried JavaScriptMVC in Sep/Oct 2010 the website was up but sometimes the JS on the page had issues for a prolonged time which made it impossible to get to the documentation. It also meant that it wasn't tested before being released.

I've given both the 2.0 and the 3.x examples a fair shot but forgive me if I don't remember exactly which one I've tried. AFAIR the example was a multipage CRUD application which was not what I was going to build and the documentation was not deep enough to understand how to deviate from this to build other kinds of apps.

I've sent an email to the mailing list which was answered promptly. This was something about formatting and code quality.

I believe that you and the team have spent a great deal of time working on JavascriptMVC and to be honest your approach was the most appealing to me as it seemed to have the whole tool chain covered. Especially the browser testing. However, my impression was that the documentation lacked sufficient depth for someone who is new to your framework (not new to programming) and who is not going to build exactly the kind of app you were demonstrating. It was just too thin and simple but the framework was not so easy to use.

There are a lot of frameworks out there and I am writing code for a living. My time to evaluate things is limited. So I've picked the 5 frameworks above and tried to go through all of their respective tutorials and build their sample apps to get a feel for the tool chain and the documentation. If things go smooth I can take it from there. If not, then I'm off to the next one.

So what I've looked at are the tutorials, the documentation, the code, comments in the code, formatting and attention to detail, how well does it all work, what does the community look like to get an idea whether it was worth spending more time to evaluate it. To that respect I've given JavascriptMVC much more time than others since I really wanted to use it. In the end I gave up because I couldn't get the browser testing to work. There may have been some issues with MacOS build scripts but I'm not sure anymore.

So the framework maybe good and the website showed the really interesting features but in terms of how to build applications I've found that exercise had been left to the reader.

What I've seen on jQuery and OpenLayers for example looks a whole lot different.

IMO, you need to work on the documentation.

Just my $0.02
Frank

Feb 05, 2011
Angelo Selvini liked this post.
Apr 13, 2011
Marlin said...
Nice write up. I'm at the beginning of this journey and am now treading lightly. BTW please $('.posterous_comments_list.').toggle(); Went and got myself a coffee for this essay and turned out to be a page or 2 with 20 pages of comments. lol
May 04, 2011
Aaron Fay said...
There were parts of the article where I was thinking, "you could swap jQuery for PHP and make the exact same point..."

Great article.

Cheers,
Aaron

Leave a comment...