Keeping jQuery in Check

Segregated DOM makes for a maintainable JavaScript codebase

jQuery makes it really easy to work with the DOM and other browser APIs. Almost too easy. Having the almighty $ available to you at all times can lead to an architectural style that I refer to as “jQuery soup.”

JQuery soup muddies application code

A jQuery soup codebase is one where adhoc references to $ appear everywhere. AJAX calls and DOM manipulations are intermingled alongside application logic and business rules.

Taking this approach in a JavaScript app of any real significance will cause a lot of pain. Any part of your app that is reaching out to the DOM via $ is essentially mutating a big bag of shared global state. That means that whenever you want to modify or extend that part of the app you need to carefully maintain a large mental model of every DOM interaction in your head. That’s really hard to do, and very much prone to error.

Testing with the DOM is no fun

It’s interesting that jQuery soup tends to be a frustrating thing to unit test. I believe that TDD plays a significant role in providing feedback on the design of your code—a focused, loosely coupled design tends to lend itself to testing. So I don’t find it particularly surprising that jQuery soup—which is hard to unit test—tends to correlate to a poorly structured codebase that is hard to maintain and evolve.

As discussed, the $ global variable (a.k.a. the Almighty Dollar) is always within arm’s reach, and its convenience makes it quite tempting to use. However, implicit dependencies such as global variables make for code that is hard to test. It’s also well known that UI code is hard to test. jQuery soup code uses implicit dependencies to modify UI. This double-whammy leads to code that is extremely tricky to test. Most developers when confronted with the task of writing automated tests for a jQuery soup codebase will quite rightly feel a lot of pain.

You can’t fake the $

Imagine you have an app that does basic validation of a phone number. The easiest way to get that working is to use $ to wire up some DOM events to some validation functions (or a jQuery plugin) that in turn use $ to update the UI if there are validation issues. The problem with this approach is that your custom validation logic will be coupled directly to the DOM, via jQuery. Code like this that is using $ directly can’t really be tested in isolation.

So if we want isolated unit tests that aren’t coupled to an actual DOM maybe we should isolate what we’re testing by mocking out the calls to $ being made by our test subject (in this case the validation logic). A nice idea, but anyone who has tried to mock out jQuery’s awesome fluent API will tell you what a bear that can be. For a start, almost all use of jQuery starts with the same function call: $(...). Trying to mock out multiple distinct calls to that same $(...) function that differ only by the specific string passed in is tedious, overly verbose, and highly fragile. In addition, the fluent method-chaining API style that jQuery exemplifies means that you have to do an extraordinary amount of gymnastics to mock the right function call at the right time. Again this is tedious, really hard to read, and very fragile. Oh, and any changes to the way that sequence of chained methods are made in your test subject will tend to cause a unit test to break, and to break in very non-obvious ways.

The only reasonable response to all this is to give up on behavior-based testing of how jQuery’s is using the DOM, and instead use a state-based approach (read this for more on the distinction between behavior-based and state-based). The state-based approach would be to run tests in the context of a DOM, that jQuery can then interact with directly. To simulate user input you trigger actual DOM events. To check how your test subject is using jQuery, you have it actually modify the DOM and then verify those modifications.

The net result of this is that any tests for code that use $ directly will require a DOM to be present in order to work, that for a jQuery soup app means you have to run all your unit tests in a browser. You can use tools like phantom to make these tests faster and more automatable, but it’s still a lot less preferable than running most of your tests in a more isolated setup within a runtime such as node.js.

Is it really such a big deal that we run our unit tests in a browser? After all, that’s where the code will run in the real app. I’d argue that yes, it does have a serious negative impact on the type of testing we do, and therefore an impact on how well our tests drive our design. If we’re testing our app logic by inspecting how our code uses jQuery to interact with the browser then by necessity we have coupled testing of our app logic to our UI. This means that our tests are brittle—changes in our UI might break our app logic tests for no good reason. For example if we change where in the DOM our validation messages appear then that might cause our validation unit tests to break for no good reason. It also means our tests are going to run slowly—we probably need to run them in a browser or at the very least we need some sort of DOM available when running our unit tests. Finally—and most importantly—our tests are not focused. It’s hard to see what a test is really verifying when you have UI concerns muddying the waters. And again, when that test breaks it’s not immediately clear whether it broke because of a UI change or because some application logic is no longer working correctly. This is what we mean when we talk about isolated tests. Are they testing an isolated chunk of our app such that when the test breaks we know exactly that small chunk of our app is no longer working as expected.

A Solution: Segregated DOM

This isn’t the first time our industry has come up against this problem of domain logic intermingling with UI code. Anyone who did thick-client development in 90s will probably have encountered the joy of a button click handler that merrily accessed the database directly. That’s good news for us, as it means there are well established solutions that we can adapt to our present problem.

Segregating jQuery

The solution is to aggressively segregate our low-level UI code into a small area of our codebase that then exposes a clear API to the rest of the app—creating a presentation layer, essentially. This API goes two ways. It allows your app to update your UI, by manipulating the DOM. It also allows your app to respond to interactions with your UI, by subscribing to DOM events and re-publishing them to your application as domain events.

Keeping the presentation layer separated solves jQuery soup

With this well-defined presentation layer in place we can now ban all use of $ from anywhere else in our code. In this new segregated world our codebase can only use jQuery to interact with the DOM from within that skinny presentation layer, effectively segregating the DOM from the rest of our application.

Segregation in practice

Here’s how an an application of this Segregated DOM pattern might look, using the same validation example as before. The example is simple and the implementation quite naive, but it should serve to illustrate what a presentation layer does and how it’s used by the rest of the application.

DOM events

When it comes to events the purpose of our presentation layer is to turn low-level DOM events into abstract ‘domain events’:

In this example we’re simply wrapping a call to a $(...).on(...), but in a ‘real’ presentation layer you would see more complex mapping. You’d also see selectors like form.userInfo turned into shared, re-usable jQuery objects:

The goal with these domain events is to capture the high-level semantics of the event (e.g a user selecting which color they want) rather than some low-level DOM event (such as a user clicked on an item in a dropdown). Once you’ve done that in your presentation layer the rest of your application code can be written in terms of your domain (users selecting colors), not in terms of technical implementation (clicking on a DOM element). Additionally you now have the flexibility to tweak the UI structure and/or which lower level events you use without affecting the rest of your codebase. Both of those are big wins.

DOM inspection

Similarly to how we abstract over low-level DOM events, we also use the presentation layer to abstract over DOM lookups. Instead of our validation code getting current form input values directly via $ calls we would add something like this to our presentation layer:

We’ve wrapped up all low-level inspection into one method call that returns a simple object representing the current contents of the form fields.

DOM changes

Lastly, we use the presentation layer to abstract over the low-level details of updating the UI. Our form validator wants to update the UI when it detects invalid input. Rather than reaching directly into the DOM via $ it can use a function in the presentation layer such as:

This low-level function does the boring jQuery work to remove any existing contents from a list of validation messages, and then add a new <li> for each message passed in. Not particularly interesting, and also not the sort of code that you want to have to reason about in the middle of your validation logic while you’re trying to figure out how to add or modify validation functionality. With this function in hand we can keep our validation code focused on validation. All it needs to do once it’s detected any validation issues is call updateValidationMessages with an array of strings describing the issues.

You might wonder whether using a templating system would be better here, or perhaps you disagree in some other way with how updateValidationMessages is implemented. You could be right. And the great thing about segregating the DOM manipulation code in this way is that it’s really simple to improve the way we do this low-level DOM manipulation, because it’s all done in a single contained area of the codebase.

Putting it together

Here’s how our validation code would work as a client of this presentation layer:

In setupUserFormValidation we register a handler for the high-level “user form changed” domain event. Whenever that event occurs we get the current value of the form fields, check them for any validation issues, and update the UI to reflect whatever issues we have found. Succinct, easy to understand, and focused on a single thing, input validation. We don’t see any low-level technical details about interacting with the DOM. We could easily test this code in complete isolation. If the structure of our UI changed this code wouldn’t be affected at all unless the change was absolutely relevant to this code’s responsibility.

Segregation == cleaner code

So we see that a Segregated DOM leads to some nice benefits. The presentation layer is essentially an abstraction layer over the DOM, adapting boring technical details like CSS selectors and DOM events and converting them into richer higher-level concepts. As the rest of your codebase refers to the UI via these higher-level abstractions, your code becomes easier to understand. For example, which of the following would you prefer to digest while skimming through a high-level function in your application code:

$('.userInfo input.name').val() or getUserForm().name?

The benefits extend way beyond more readable code. By keeping UI-centric code in one place you’re in a much better position for your HTML to grow and flex over time. By focusing on isolated components you now have the ability to test the majority of your application code using incredibly fast isolated unit tests. On a large JavaScript codebase that followed this approach we were able to run our entire unit test suite in a few seconds. They ran so fast that our feedback loop was to just run all the unit tests whenever we changed a file! Finally, this approach also applies equally well to the other side of the house where jQuery often plays—AJAX calls. Similar techniques can yield similar benefits when applied in that context too.

For further reading, check out Martin Fowler’s post on Segregated DOM, published the same day as this article.

Related

Sign up for the O'Reilly Programming Newsletter to get weekly insight from industry insiders.
topic: Web Platform
  • http://blog.mitemitreski.com/ Mite Mitreski

    To some extend MVC frameworks like angular solve some of the issues but even there you can still couple the DOM with the rest of the very easely.

    • ikari

      Actually I believe the problem isn’t whether frameworks solve or create issues. It’s always a matter of your coding style in the end. These aren’t actually jQuery issues, they are usage and code style issues.

      • http://blog.mitemitreski.com/ Mite Mitreski

        Yes I agree it is always the code style or lack of it that creates the problem.
        But certain frameworks lead to certain style. One can always mess up in the best of libraries.

        JQuery in as sense removed the use of global variables by having document.ready section but that does not mean that we don’t have global variables in jQuery code.

        The same goes for Angular, we have the module pattern there but we can always mess with it.

  • Shripad

    Excellent post. Will be using some of the ideas in our current project.

  • ClayShentrup

    You used == instead of ===. Fail.

    • ClayShutup

      And you added nothing of value to the discussion, other than a demonstration of your haughty, snide arrogance. Fail.

  • johndurbinn

    Sooooo, basically use Backbone/Angular/Ember is what you’re saying

    • stephenkamenar

      Don’t forget React!

  • Christian Meilke

    I noticed that you used a lot of ‘That causes pain.’ or ‘That is error prone.’ You are absolutely right. I worked with so many other developers that really have a problem understanding these issues. Let alone change something. Isn’t that just frustrating? I have this colleague that really is creating the jQuery soup you are talking about. He just looks at me blankly when I mention the problems and alternatives.

    • Adam Yee

      Yes, it can be frustrating. Unfortunately, not every developer values testable and maintainable code. Couple reasons could be because they simply can’t understand how it makes a developer’s (long-term) work easier (and more enjoyable) or they could be too proud to admit they’ve been writing instant legacy prone code.

  • dmij

    > which of the following would you prefer to digest while skimming through a high-level function

    Not to be contrary, but I would prefer:

    $(‘.userInfo input.name’).val()

    The difference is that I know *exactly* what that expression will return. If I were skimming code that was new to me and saw”getUserForm().name”I would guess that the latter expression refers to the value of the form’s name attribute… which would be wrong.

    There’s always a struggle between “being too reliant on a library” vs. “reinventing the wheel”. I suppose the key question as a developer should be “where is my time best spent”. In this case, I would have leveraged jQuery Validation plugin which implements most of the functionality described in this article. It would have freed me up to concentrate on what makes my app unique.

    • phodgson

      How would you test that you were using the plugin correctly; that it correctly identified valid and invalid input?

      • dmij

        A fair question. I’ve written qunit tests for custom validation rules that I’ve written, and the plugin project itself has good test coverage (not sure if links are allowed here, but it’s on github under “jzaefferer/jquery-validation”). Many of which are pure javascript (no DOM requirement).

        I guess it goes back to where you want to spend your time. Sometimes it’s best to leverage a 3rd party library so you can limit your focus. On the other hand, you may want to remove 3rd-party dependence for features that are a vital to your app.

    • spencercarnage

      Pointing out that a function name is confusing is missing the point here.

      Yes, getUserForm().name could be interpreted as having several different meanings but that’s not what’s important. What is important is that the you have taken a portion of code and made it explicitly responsible for a specific task. This helps with testing in that you can ideologically test against a specific “action” vs testing against a few lines of jQuery code. This not only helps the code to find a more comfortable spot in your brain while working with it but it also allows for you safely and sanely change how it is implemented in the future because you have set up tests that check the intended result.

      And yes, you can and should use the excellent jQuery Validation plugin if it fits your needs. I do all the time but I never need to write tests for it because those tests have already been written long ago. However when the validation succeeds and my form submits an AJAX request which can come back with 1 of 3 different results with very different consequences, I should write a test for each of those scenarios.

  • brianm101

    The strange thing about the obvious is that it’s so often ignored!
    So thanks for a reminder!

  • Rich

    The DOM is meant to be a model already – why is it always conflated with UI?