On Rolling Your Own
There’s been a lot of activity around my last post, On jQuery & Large Applications. A number of people have asked me why, exactly, I’m so opposed to using jQuery as part of a roll-your-own solution.
To answer that, let’s start by looking at (some of) the pieces my ideal large application solution might include:
- DOM manipulation tools
- Ajax tools
- A dependency management and build system
- Clear patterns for code organization, such as namespaced modules
- An inheritance system, preferably one that offers multiple inheritance, for sharing code across modules and staying DRY
- A non-DOM-centric, loosely coupled API for communication between modules
- A widget system that makes use of the inheritance system, with lifecycle management (setup/teardown) and templating
- A system for maintaining templates separate from JavaScript while interning them into the build to eliminate HTTP requests
- A system for abstracting RESTful server communication
- For a UI-intensive project, a rich widget system pluggable to arbitrary data sources and designed with an easily-extended API
- For an enterprise project, a11y and i18n provisions, as well as clear patterns for file organization
To all of you who have said you can do this yourself, you win. I can’t argue with you without sounding like I’m saying you’re too dumb, and you’re probably not.
But here’s my question: why? What, exactly — exactly — do you gain by putting all of this together for yourself, rather than using the pieces you need of a toolkit that provides all of this out of the box? Because here are a few things I think you lose:
- Integration. Is your abstracted RESTful data API designed to talk to your widget system that’s designed to talk to your template system? That’s hott. Now what about when a new version of one of those components comes out that violates an assumption you made?
- Maintenance. I heart the good folks who have put together individual answers to these individual questions, but they have no obligation to continue being the good folks they are, and they certainly have no obligation to do it on any sort of schedule. Remember all those plugins that broke with jQuery 1.4? That sure was fun.
- Documentation. I’m going to grant you, right now, that jQuery is one of the best-documented JavaScript libraries out there, hands down. But what about all these other pieces you’re putting together? Especially the ones you really are rolling on your own, like that templated widget thing that communicates so nicely with your abstract data API. There are a wealth of resources for understanding, troubleshooting, and using these pieces in established toolkits. Where will the next developer turn when they have questions about yours?
- Experience. Like I said, I get that you’re smart. Possibly smarter than me. That’s cool. But are you smarter than the combined wisdom of a team of people that has been thinking about these questions for years? Are you sure your solution has thought through all the questions they have?
I’ve noticed that, in the conversations I’ve had the last few days, it seems to fall to me to “prove” that a roll-your-own solution that includes jQuery isn’t advisable. Perhaps that’s fair — “you started it!”, you might say, and that I did. But simultaneously, others argue that jQuery never set out to answer these questions, and so it’s not jQuery’s fault that people are trying to use it in ways it wasn’t intended to be used. I have waited in vain to hear a compelling reason why jQuery should be part of a large application solution, to hear why I should recommend a roll-your-own solution that includes jQuery to my clients. The extent of the argument seems to be “because I like it, and it doesn’t force me to think a certain way.”
No one puts baby in a corner. Got it. But the straw man-ness of this argument has me, literally, chuckling right now. Let’s not confuse a mythical one-size-fits-all solution with a toolkit that provides, well, tools. Tools to do all sorts of things, tools meant to work together, tools developed and tested and maintained by a whole big team of smart people, tools that are, actively, being used in really frigging big, really frigging enterprisey applications.
I very purposefully didn’t propose a particular alternate solution in the original post, but it’s hardly a secret that my personal favorite, of late, has been Dojo. Not because it purports to solve every problem or prescribes how to solve them, but because it gives me so many tools to use to solve a given problem. Time and again I find that “Dojo already did that” — they already wrote the tool I’m wishing I had. Now I don’t have to write it, and, perhaps more importantly, I know it was written to work with all of the pieces I’m already using, and when I use it I’m not risking duplication of code or a lack of testing, maintenance or support. Win.
But let’s be very clear: no one’s forcing me to use that component! No one is forcing me to do things a certain way, any more than jQuery is “forcing” me to think of my application entirely in terms of the DOM. I can write my own component if I want, or use someone else’s if I want, or change it a bit if I want! For example, on a current project I pulled in mustache.js because the project had a lot of templates that had already been written to use it. The brilliant thing, though, was that integrating mustache.js into dijit._Templated instead of the standard templating system was trivial. That component, and all the others in Dojo and Dijit, are architected explicitly not to be one-size-fits-all. They provide a rock-solid base for large application development, for getting up and running quickly using a bevy of ready-made solutions, but also provide so many extension points that you can turn those solutions on their head if you want or need.
Garann Means, whose blog you should be reading, took a bit of issue with my original post in model-view-controller and comfy clothes.
I do agree that it benefits everyone to be working in the same setup and making use of tools that have been vetted by geniuses whose entire job is to create such things. But I’m dubious about any approach which comes too close to promising one size fits all. If you’ve ever sewn a dress, you understand that one size fits all is technically possible, but some people are going be left with a lot of excess while others will scarcely be able to breathe.
Carrying on that metaphor, these pieces provided by Dojo — or any other comprehensive toolkit, for that matter — are but starter patterns, and thread, and scissors, and pins, and a sewing machine, and OK I’m stretching the metaphor, now, but my point is they’re definitely not finished one-size-fits-nobody garments. On the other hand, if I decide to use jQuery in a large application, it can feel like I’ve been given a black marker and some of that crinkly brown paper, and now it’s up to me to draw a pattern and then come up with all those other pieces, too. Intellectually interesting and pleasingly crafty, perhaps, but not particularly efficient, sustainable, repeatable, or maintainable.
So again I ask, in all seriousness and in hopes of fostering a good discussion: Why? jQuery provides you with DOM, Ajax, and event management tools, but little else. There are tools designed for building large applications, designed to provide all of the pieces I want and so many more it’s not even funny, and they provide you with DOM, Ajax, and event management tools, too. What’s the compelling case for rolling your own solution that includes jQuery instead?
20 comments
This application I'm speaking of was a business-to-business financial support tool (really, bank-to-business). It had features for accountants, for people carrying company credit cards in their wallets, for purchasing managers, for decision makers, etc etc. Really a gigantic number of features, so many that bugs could often be closed when it was realized that an enhancement request was already supported by an application feature that everybody had forgotten about. The site involved literally hundreds of separate pages.
Across all that, however, the number of different things that had to go on in the view end of the system was really not that big. There were dialogs, and thing-pickers, and table/queue views, and search tools, and pull-down menus, overall a healthy but not excessive array of typical RIA widgets. Once that had grown to a certain size, new application features - even sprawling, vast new features - did not really necessitate more Javascript. (Sure, we wrote it anyway, but that brings us back to inexperience, ignorance, and poor planning.)
I guess what I'm saying is that the fundamental argument baffles me a little. I want to *avoid* having to worry about anything like an Enterprise coding framework for client-side site infrastructure because I don't want to worry about that much code. I want to avoid it not only because I want to keep the code base under control and maintainable, but also because I think that an excess of client-side features and behaviors seems like a potential problem for application users.
Of course, it could be that I'm still ignorant of what it is that would go on in an Enterprise-scale mountain of client-side code. I can think of a lot of stuff to do on the client, but the list doesn't get bigger just because the hypothetical application gets bigger - it gets bigger when I want client-side interaction behaviors to get richer and more varied. It just seems like there are some natural limits to that growth, and in my imagination it never gets that big.
For instance, I'm working on a project where we're actually just using the GET part of Sammy.js as we don't actually need the other parts, because we set them up using Event binding.
The second part is: Often when working on webapps, I've had to use jQuery, to lower the barrier of entry to the other developers on the team. These other developers often don't do front-end programming often, but they can easily understand what they need based of the jQuery documentation. So, consequentially, I've become quite good at writing frameworks on top of other libraries, or just frameworks to do things with a better API (better being a contextual variable).
I'm not saying that jQuery is the best tool for the job, often I use things from both Dojo, YUI, etc and roll them into just a set of methods in a single API. I have actually been known for saying that people learning jQuery don't often learn javascript, so, it's not necessarily that jQuery is the best tool, but it's a tool.
To extend on that, We can make jQuery more like a toolkit by using parts from other frameworks and writing small modules to be able to do what we need to do.
(probably remind me to follow up with a blog post if you want more on the thoughts above)
Also, to me, GMail doesn't seem like a typical "Enterprise" application; it's far, far too small and limited. That however is just a matter of semantics, and as such it's only important in that it helps explain some of my difficulty in seeing your point of view. That GMail's appealing and elaborate client-side interactions necessarily require a framework, well, again that's an assertion you can make but I'm personally not convinced it's true. It probably wouldn't hurt, but I would imagine that most of the headaches there come from the implementation of the features themselves, and not the "code wrangling" involved. Not having implemented GMail I clearly can't be sure of that; maybe I'll do a quick "view source" :-)
I do enjoy the topic, and it may be that my thinking is clouded by a subconscious fear of the unknown, or possibly subconscious laziness.
Most of the unanswered questions that I have about client-side code are pretty much the same as the unanswered questions that I have about the server as well... with the exception of things based on URL changes.
YUI 3 fulfills every single bullet point in your wish list, AND world-class unit testing, profiling, documentation, and compression tools, incredibly thorough examples and tutorials, and a thriving community. Plus, YUI 3 is lightweight and lightening-fast to load out of the box. Cheers!
Have you had a chance to check out YUI lately? We've come a long way since the days of YUI 2 and have a lot to offer to novices, hackers, and application developers alike. We are currently revamping our website to make it easier to navigate, but in the meantime I encourage you to check us out at http://developer.yahoo.com/yui/3/.>
I have worked on a variety of sites in the past, but my favorite projects are always the rich application type web apps than the simple scripty tweaks to existing sites (as you said in your earlier posts, simply selection a couple elements, adding some anonymous methods to them gets old quick).
My library of choice is MooTools because of its fantastic set of coding patterns that emphasize code re-use and modularity.
Living in the Bay Area, I have had the opportunity to debate these discussions (jQuery vs X) over a beer, many times. More often than not, I always get the response "that's not javascript", "you're trying to make javascript a language it's not", etc and an outright dismissal of anything but. Many of these responses have even come from some very talented developers, which surprises and saddens me as I really wish there was more room (and believe, like you say, there is a real need) for other frameworks/libraries.
Thanks for your thoughtful and well informed posts on the matter!
@miksago would you let someone who doesn't understand Java write the architecture of the Java-based back-end of your "enterprise" application? If you have people with little knowledge of FE development dealing with *complex* FE tasks you will end up with an unmaintainable mess.
@rmurphey Again, couldn't agree more. JQuery is very well suited for lots of tasks, and it is very good to get those things done quickly; but if you need a complex UI with layers of abstraction then "rolling your own" won't cut it in the long term.
We have great solutions for some of those things in .NET, which I'm very happy with. At the same time, Ben probably has a different approach that works great for him in ColdFusion. A "universal" client-side framework to integrate with both our server-side workflows (and the many others) would almost certainly be a watered-down, overly-abstracted disaster.
For that matter, even libraries targeted at a single framework can be overwrought. Just look at how ASP.NET developers trended away from the enterprisey MicrosoftAjax.js/ASP.NET AJAX (which provided most of what you're asking for), right into the arms of jQuery. Even when they bring a lot to the table, complex frameworks drive most developers away.
On a more specific note, I'm curious about your point against DOM-based messaging. $(document).trigger/bind('CustomEvent') works well as a light-weight "message bus". Or, $(JSObject).trigger/bind('CustomEvent'), which is even more handy if your event revolves around a client-side object, since "this" will refer to the object itself within the event handler.
Have you run into specific trouble using that approach, or is that personal preference?
- Building a RESTful server is, obviously, the domain of, well, the server. Connecting to it from your JavaScript application is, well, the domain of JavaScript. Dojo's dojo.data and related stores provide a pleasing way to make this connection.
- How can the server make a widget accessible, so that, for example, arrow keys can be used to change the width of a content pane or tab/arrow from one grid cell to another? Sure, it's up to the server to make sure a form works without Ajax, but there's more to accessibility than that.
Regarding server-side builds, I have yet to meet a build system that isn't server-side. What I'm advocating is a self-contained way to express dependencies inside my JavaScript files, a way that can then be parsed by a build tool in order to generate production-ready files, without depending on integration with the server-side application. On my current project, I don't even have *access* to the server-side code! My HTML/CSS/JS app connects through an nginx proxy to ... I don't even know! And I don't have to care! I'd hate to have to rely on a server-side dev to build my code; I don't even know who the server-side dev is.
As far as DOM-based messaging, I don't dispute that custom events work and that they can be used for this, but it's slow and unnecessary if a more generic messaging system exists: http://skitch.com/dantehicks/dthfj/mozilla-firefox.>
Personally, I dislike that approach, but it's basically what you're asking for (driven from the server-side). We've had that in .NET for years. I'd imagine other server-side frameworks that expose endpoints have similar?
There are enough subtle differences between server-side endpoints that I would definitely not want to include a client-side framework to abstract them all away when I'm usually only using a single type of endpoint.
- Widgets definitely should be client-side. I do prefer the ThemeRoller approach to being locked into a certain UI library's widgets though. Some of my earlier experience with standardizing on a UI "framework" was the AJAX Control Toolkit, years ago. Do. Not. Miss. That.
- The app I'm working on today has the combination/minification built into the build process, which runs on a continuous integration server. So, when I check code into source control, it automatically combines files, runs JSLint on them(, hurts my feelings), and then minifies them into production-ready versions.
The way each page's script is referenced (through an ASP.NET MVC HTML helper), it knows to either reference all of the individual scripts or the single, minified script depending on whether the app is running in debug or release mode.
Currently, that build is controlled by a simple, standalone XML file located with the scripts that comtains named groupings of files. There's room for improvement (each "master" script already has a /// <reference /> to the files that will be combined into it; we should be leveraging that instead of a separate config file), but it would be in that server-side build process.
- I'd be curious about the performance impact of a simple $(jsobject).bind/trigger, vs. whatever wrapper that is in the screenshot. I use the former pretty heavily and have never had any performance trouble in practice.
- My experience and opinions are probably different because I come from a server-side background. The idea of working on a project where I can't even see the server-side code is completely foreign to me.
I think that fundamental difference really illustrates how significantly company-specific practices drive these things too. Much of what you've described is (or should be) covered by the company-specific coding standards dictated by most large companies. At the same time, companies vary so widely on those standards, it would be very difficult getting them all to adopt a single client-side coding standard as far reaching as how the project should be structured and files combined.
That doesn't mean it's uninteresting to talk about these issues though. I definitely enjoy reading about everyone else's approaches. I just don't think an overreaching "enterprise" JavaScript library is a viable solution. It's a problem of people and process (IMO).
> that includes jQuery instead?
For me it has always been that there is no standard way of doing it, so i'd rather do it my own way with the knowledge that i can later switch to the standard instead of doing it the dojo way.
I really hope commonjs will soon be more useful for spec'ing all those things for clientside javasript. but until then i try to keep it light on all the big problems you mention, even roll my own solution - modularization, widgets, MVC - so i can later mor easily switch to whatever then new standard will be.
YQL and nodejs seem to be nice at hiding details without forcing an object model on you.
JSON takes the (semi) structured content out of the code it was hidden in.
It would be nice to have MVC less components that you can wire with a simple bit of JSON.
One thing that I think gets overlooked in your analysis is aesthetics.
For example, Dojo seems well designed and thought out, but its data grids look like Microsoft Access. I think sometimes people think "I don't care how well thought out their data model is; their grids are way too complex for my need and they look funky." I know everything is theme-able, but there are basic aesthetic decisions baked into every framework that many may not like.
This discussion also does not address the tortured designer/developer, front-end/back-end relationship. Most developers confine themselves to the back-end, leaving most of the front-end to others. I'm convinced this is why jQuery is so prevalent; it's something that designers can pickup pretty readily and it is, by virtue of its DOM-centric approach, practically guaranteed to keep the designers out of the kitchen.
As someone who is concerned with both engineering and visual aesthetics, I am looking for approaches that tightly couple server and client side data models and leave view-layer presentation details more up to designers — all while not requiring them to reinvent the wheel.
So, I think it's interesting to consider how much visual aesthetics plays a role in peoples' choice of frameworks. I think many look at the visuals first and then go from there, rightly or wrongly; and jQuery, with its popularity amongst designers and flexible visual effects, can always score well on that front.
It doesn't work for everyone, but more and more developers are realizing that it is entirely possible that as long as you can accept the dependency on javascript (well beyond progressive enhancement), you don't need a web framework on the server. The server can basically become a place for business logic exposing json web services. When you do that and you have dozens or hundreds of screens with dozens or hundreds of different data types and tables interacting in complex ways that you realize jQuery really is "a hammer and you're building a skyscraper". If dom manipulation is still your highest level of abstraction, then you are basically writing assembly code and I weep for you.
When I started writing the large web application I'm currently maintaining (3 years ago) the landscape was a little different than it was today. I did a lot of work trying to find a suitable web framework and quite frankly didn't find anything that I considered met my needs. Even dojo - to me - didn't really seem to have a good story for large single page applications. A single page webmail client or RSS reader maybe, but not a loan automation system with workflows and different roles, and system integrations, and well, you can imagine.
Before embarking on a javascript version, we had actually done a Flex version and wanted something similar (but better). In the end I did roll my own, and I actually did use jQuery, but it was really only in there as a low level dom library. It is completely abstracted over by the template/component system and has actually since been replaced by the YUI library, but doesn't really matter because it is only used by a tiny portion of my framework's code and none of the application code. After 3 years, the amount of framework building has been pretty intense, but I haven't seen anything out there I'd really prefer to work with.
I'm not really sure where this comment is headed any more, but I just thought I'd mention that even though I did roll my own, it was a major undertaking, and I pretty much agree with everything you wrote.
