Javascript: Require and Import Found Harmful

For the moment, let’s go ahead and make an assumption: automated tests (unit tests, integration tests, etc.) make code safer to write, update and change. Even if tests break, it says something about how the code was written and pulled together while running. I will address this concern in another post at a later date. Nevertheless, I am going to rely on this assumption throughout this post, so if you disagree with the initial assumption, you might be better served to drop out now.

Now that the grumpy anti-testers are gone, let’s talk, just you and I.

I don’t actually believe that require or import — from the freshly minted ES module system — are inherently bad; somewhere, someone needs to be in charge of loading stuff from the filesystem, after all. Nevertheless require and import tie us directly to the filesystem which makes our code brittle and tightly coupled. This coupling makes all of the following things harder to accomplish:

  • Module Isolation
  • Extracting Dependencies
  • Moving Files
  • Testing
  • Creating Test Doubles
  • General Project Refactoring

The Setup

Let’s take a look at an example which will probably make things clearer:

To get a sense of what we have to do to isolate this code, let’s talk about a very popular library for introducing test doubles into Node tests: Mockery. This package manipulates the node cache, inserting a module into the runtime to break dependencies for a module. Particularly worrisome is the fact that you must copy the path for your module dependencies into your test, tightening the noose and deeply seating this dependence on the actual filesystem.

When we try to test this, we either have to use Mockery to jam fakes into the node module cache or we actually have to interact directly with the external systems: the filesystem, and the external logging system. I would lean — and have leaned — toward using Mockery, but it leads us down another dangerous road: what happens if the dependencies change location? Now we are interacting with the live system whether we want to or not.

This actually happened on a project I was on. At one point all of our tests were real unit tests: i.e. they tested only the local unit we cared about, but something moved, a module changed and all of a sudden we were interacting with real databases and cloud services. Our tests slowed to a crawl and we noticed unexpected spikes on systems which should have been relatively low-load.

Mind you, this is not an indictment of test tooling. Mockery is great at what it does. Instead, the tool highlights the pitfalls built into the system. I offer an alternative question: is there a better tool we could build which breaks the localized dependence on the filesystem altogether?

It’s worthwhile to consider a couple design patterns which could lead us away from the filesystem and toward something which could fully decouple our code: Inversion of Control (of SOLID fame) and the Factory pattern.

Breaking it Down

To get a sense of how the factory pattern helps us, let’s isolate our modules and see what it looks like when we break all the pieces up.

With this refactoring, some really nice things happen: our abstractions are cleaner, our code becomes more declarative, and all of the explicit module references simply disappear. When modules no longer need to be concerned with the filesystem, everything becomes much freer regarding moving files around and decoupling concerns. Of course, it’s unclear who is actually in charge of loading the files into memory…

Whether it be in your tests or in your production code, the ideal solution would be some sort of filesystem aware module which knows what name is associated with which module. The classic name for something like this is either a Dependency Injection (DI) system or an Inversion of Control (IoC) container.

My team has been using the Dject library to manage our dependencies. Dject abstracts away all of the filesystem concerns which allows us to write code exactly how we have it above. Here’s what the configuration would look like:

Module Loading With Our Container

Now our main application file can load dependencies with a container and everything can be loosely referenced by name alone. If we only use our main module for loading core application modules, it allows us to isolate our entire application module structure!

Containers, Tests and A Better Life

Let’s have a look at what a test might look like using the same application modules. A few things will jump out. First, faking system modules becomes a trivial affair. Since everything is declared by name, we don’t have to worry about the module cache. In the same vein, any of our application internals are also easy to fake. Since we don’t have to worry about file paths and file-relative references, simply reorganizing our files doesn’t impact our tests which continue to be valid and useful. Lastly, our module entry point location is also managed externally, so we don’t have to run around updating tests if the module under test moves. Who really wants to test whether the node file module loading system works in their application tests?

Wrapping it All Up

With all of the overhead around filesystem management removed, it becomes much easier to think about what our code is doing in isolation. This means our application is far easier to manage and our tests are easier to write and maintain. Now, isn’t that really what we all want in the end?

For examples of full applications written using DI/IoC and Dject in specific, I encourage you to check out the code for JS Refactor (the application that birthed Dject in the first place) and Stubcontractor (a test helper for automatically generating fakes).

Comments Off on Javascript: Require and Import Found Harmful

Why Should I Write Tests?

There has been an ongoing discussion for quite some time about whether automated tests are or are not a good idea. This is especially common when talking about testing web applications, where the argument is often made that it is faster to simply hack in a solution and immediately refreshed the browser.

Rather than trying to make the argument that tests are worthwhile and they save time in the long run I would rather take a look at what it looks like to start with no tests and build from there. Beyond the case for writing tests at all, I thought it would be useful to take a look at the progression of testing when we start with nothing.

Without further ado, let’s take a look at a small, simple application which computes a couple of statistical values from a set of sample numbers. Following are the source files for our stats app.

Now, this application is simple enough we could easily write it and refresh the browser to test it. We can just add a few numbers to the input field and then get results back. The hard part, from there, is to perform the computation by hand and then verify the values we get back.

Example of manual test output

Manual test example

Doing several of these tests should be sufficient to prove we are seeing the results we expect. This gives us a reasonable first pass at whether our application is working as it should or not. Beyond testing successful cases, we can try things like putting in numbers and letters, leaving the field blank or adding unicode or other strange input. In each of these cases we will get NaN as a result. Let’s simply accept the NaN result as an expected value and codify that as part of our test suite. My test script contains the following values:

Input Mean Standard Deviation
Success Cases
1, 2, 1, 2 1.5 0.5
1, 2, 3, 1, 2, 3 2 0.816496580927726
Failure Cases
a, b NaN NaN

Obviously, this is not a complete test of the system, but I think it’s enough to show the kinds of tests we might throw at our application. Now that we have explored our application, we have a simple test script we can follow each time we modify our code.

This is, of course, a set up. Each time that we modify our code, we will have to copy and paste each input and manually check the output to ensure all previous behavior is preserved, while adding to our script. This is the way manual QA is typically done and it takes a long time.

Because we are programmers, we can do better. Let’s, instead, automate some of this and see if we can speed up the testing process a little. Below is the source code for a simple test behavior for our single-screen application.

With our new test runner, we can simply call the function and pass in our test values to verify the behavior of the application. This small step helps us to speed the process of checking the behaviors we already have tests for and we only need to explore the app for any new functionality we have added.

Once the app is loaded into our browser, we can open the console and start running test cases against the UI with a small amount of effort. The output would look something like the image below.

Single-run tests scripted against the UI

Single-run tests scripted against the UI

We can see this adds a certain amount of value to our development effort by automating away the “copy, paste, click, check” tests we would be doing again and again to ensure our app continued to work the way we wanted it. Of course, there is still a fair amount of manual work we are doing to type or paste our values into the console. Fortunately we have a testing API ready to use for more scripting. Let’s extend our API a little bit and add some test cases.

Now, this is the kind of automated power we can get from being programmers. We have combined our test value table and our UI tests into a single script which will allow us to simply run the test suite over and over. We are still close enough to the original edit/refresh cycle that our development feels fast, but we also have test suites we can run without having to constantly refer back to our test value table.

As we write new code, we can guarantee in milliseconds whether our app is still working as designed or not. Moreover, we are able to perform exploratory tests on the app and add the new-found results to our test suite, ensuring our app is quick to test from one run to the next. Let’s have a look at running the test suite from the console.

Test suite run from the browser console

Test suite run from the browser console

Being able to rerun our tests from the console helps to speed the manual write/refresh/check loop significantly. Of course, this is also no longer just a manual process. We have started relying on automated tests to speed our job.

This is exactly where I expected we would end up. Although this is a far cry from using a full framework to test our code, we can see how this walks us much closer to the act of writing automated tests to remove manual hurdles from our development process.

With this simple framework, it would even be possible to anticipate the results we would want and code them into the tests before writing our code so we can simply modify the code, refresh and run our tests to see if the new results appear as we expected. This kind of behavior would allow us to prove we are inserting required functionality into our programs as we work.

Though I don’t expect this single post to convince anyone they should completely change the way they develop, hopefully this gives anyone who reads it something to think about when they start working on their next project, or when they go back to revisit existing code. In the next post, we’ll take a look at separating the UI and business logic so we can test the logic in greater depth. Until then, go build software that makes the world awesome!

Comments Off on Why Should I Write Tests?

Intent: Why Types Are Important

A common complaint from both the Javascript religious and the newcomers alike is Javascript is tremendously difficult to debug when things go sideways. When a null or undefined reference is passed, the stack trace can be illuminating or it can be completely obscure. Couple this with the growing popularity to use anonymous functions assigned to variables in lieu of named functions and you have a recipe for tremendous difficulty.

In modern compiled languages like C# and Java or F# and Scala, there is an enforced, static type system in place which ensures values which would cause a malfunction are disallowed from function calls. This does not guarantee program correctness, but it does help eliminate strange errors.

Of course, the really important thing which comes from static typing is less about the compile-time checking as it is the integration of typed thinking into the entire development process. While developers working in statically typed languages are thinking about the logic for their programs, they are also considering the data types they are using to arrive at their results.

Revealing Intent

When a programming language provides type annotations, it means the programmer can declare what they intend the program to do up front. Most statically typed languages typically have an editor (or two) which provide insight into what the annotations say elsewhere in the codebase, possibly quite far from where the programmer is currently looking.

What these editors are really providing is a look into the intent of the work which was done before. I refer to this kind of behavior as revealing intent. By revealing intent to the programmer, they can make choices which simplify the work of understanding other objects, functions and behaviors.

Javascript, for better or worse, does not allow for revealing intent other than variable names. This means that either each variable name must contain information about its type so people can opt to conform to the expected types or, alternatively, misbehave and intentionally break the function being called. I am a noted fan of dynamic languages and like my functions free-flowing, but sometimes I long for a good, strong contract.

Let’s pick a simple function written in Javascript and see what our baseline is.

Our function could add two numbers, but it could also be called upon to do other things the name does not specifically call out. AddJS could concatenate strings, coerce numbers to strings or act upon functions, objects and many other data types. Clearly this is not what we intended.

Microsoft designed a language called TypeScript which transpiles to Javascript and includes features from ES Next as well as a quasi-static type system. The type system in TypeScript is a step in the right direction when types are used, but there are still some problems. Let’s have a look at our function rewritten in TypeScript.

Our new add function declares it takes two numbers and returns a number. This is really handy when we are exposing a function as an API to the rest of the world because other programmers can then capture this type information as they program and use it to make their calls conform to the expectation declared in the type contract…

Unless they aren’t using Typescript in the rest of their application.

Typescript really only solves the problem if someone has bought into the entire ecosystem and uses modules which exclusively have TypeScript annotations attached or a TypeScript Definition file. Atop all of this anyone who wants to access these annotations will need to use an editor which supports that kind of behavior.

A Type By Any Other Name…

As functional programming continues to gain traction, patterns like function currying become more common in codebases. This means we now have functions which could return other functions (higher-order functions) which run 2 or more layers deep. This kind of behavior can be demonstrated by a small rewrite of our add function.

With this setup in vanilla Javascript, we have really challenged the next programmer to try and decode our intent if they aren’t familiar with a currying pattern. Due to the lack of types and intent declaration in Javascript, this function, for as simple as it is, tells us almost nothing about intent since even the input variables are separated across different functions and the result is the product of a closure.

If we were working in Scala we would get intent and behavior bundled in, possibly, too terse a form. Nonetheless, the full intent of our behavior is described by the following line of code.

This function definition actually declares not only data types for the function, but how they interact. We could almost say add moves out of the range of function definition and into a type definition on its own. That is, however, a little more esoteric than we need to be.

Of course, moving back to TypeScript, we can see where data types, function definitions and intent start to fall apart. Although data types are declared and displayed, it is difficult to write a curried function in a way that is both clear and declarative of intent. Let’s have a look at our curriedAdd function in TypeScript.

That’s kind of like a punch to the eyeball, isn’t it?

Tying Intent and Implementation

At the end of the day, the challenge we keep coming up against is the fact that our intent is either declared, vaguely, in code and lost before execution, or it is not declared at all. Really, though, data types are not intent. This is one of the biggest problems we have.

Although programming deals in data and behavior, the problem we have introduced is we have become obsessed with the data types and we have forgotten that they are only meaningful within the context of behavior.

A couple of weeks ago, we looked at a way to add a little metadata to our function in order to communicate information to programmers who use our API in the future. I also introduced a small library called Signet, which helps to simplify the process of attaching metadata so we don’t have to litter our code with a bunch of Object.append calls.

By using the language we introduced in Typed Thinking, we can actually get a full declaration around our function and add meaning and context to our API in general. Let’s try applying our full type, declaring intent, to our vanilla JS implementation of curriedAdd.

This definition helps us to fully understand what curriedAdd will do. When we get back our final function, we can query the signature from anywhere in our code, including execution time, and get a full report back on what our function will do. Let’s take a look.

This is a simple riff on the previous post since we already knew this was possible. Where our simple type language becomes useful is when we start working with curriedAdd. Instead of getting back an untyped function, which gives us no more information than we ever had before, we have a fully parsed AST which comes along for the ride with our function. This means we will actually get all of our other types for free all the way through our entire execution path. Let’s have a look at what happens when we call curriedAdd with a single value and check the signature.

This means our initial type declaration actually allows us to understand the return types without any further declaration anywhere else!

Our enforced type declaration has given us a way to communicate our intent to anyone interacting with our code whether we are available to answer questions or not. This type assignment reveals intent and helps to make library APIs and code in other files easier to work with.

On top of the ability to clearly identify intent and keeping our APIs static and safe, we still get to keep the dynamic nature of Javascript and all of the convenience that comes with it anywhere we are using local functions. This blended static/dynamic coding allows us to quickly and easily iterate on implementation details within a module or namespace while keeping the actual interface well-defined for the external user!


Languages and the way they manage types can be a really divisive topic, but with the flexibility built into Javascript, we can actually manage types on a case by case basis and surgically insert requirements without having to tie our hands elsewhere in our code.

Although TypeScript is a popular solution for trying to get a handle on the declaration of intent throughout our codebase, it starts to fray at the edges with advanced techniques, making it unsuitable for the move toward a more functional style of programming.

In the end, enforcing a type on our API endpoints with a strong, lightweight library like Signet gives us the right blend of enforced static typing where we need it and a fully dynamic environment when we want it. This seems like the only sane direction to go when working in a language as rich and flexible as Javascript. Go declare your intent and make people awesome.

Comments Off on Intent: Why Types Are Important

Unit Testing Express Routing

If you are unit testing your code, and you should be, then it is likely you have encountered certain patterns which make testing more difficult. One of the patterns which pops up often is centered around Express.js routes. Although the router has a nice, simple API to code against, the actual testing of the route action code can be a little challenging if you aren’t used to using tools such as mocks, fakes and stubs.

Today we are going to explore using the Mockery library and an express router fake module to simplify the process of reaching into our modules and getting ahold of the route actions we provide to express for the sake of testing and ensuring program correctness.

The Libraries

The post today will make use of Mocha, Mockery and Express Route Fake. The first two you may have heard of, the last you likely have not. Let’s walk the list and get a handle on what each of these tools does for us at a high level.


If you have done any TDD or unit testing in Javascript and especially in Node, you have likely already encountered Mocha. Mocha is a popular BDD tool for testing Javascript. We will have examples of what test code in Mocha looks like later.


Mockery is a tool for manipulating packages and modules which are required within script files for Node. Although Mockery is quite useful for breaking tight coupling, it is not as well known as Mocha. Unit testing scripts which are tightly coupled through require statements is challenging since there is no clean way to inject a dependency into a dependency chain without a third party tool. Mockery is key to getting good tests around Express route actions as we will see.

Express Route Fake

Express Route Fake is a module I wrote to emulate behavior we use at Hunter to gain access to our route actions as we get tests written around our code. The core idea of Express Route Fake is to substitute the fake module in for Express in order to capture references to the actions which are assigned to different routes. We will explore this shortly.

Registering Replacements with Mockery

I am assuming you are already familiar with a testing framework, so I am not going to cover the basics of using Mocha. Instead we are going to start off looking at how to register a faked module with Mockery so we can break a dependency in Node.

Mockery actually manipulates the Node module cache and updates it with code of our choosing. This gives us the ability, at test time, to dig in and create a fake chunk of code which we control so we can ensure our tests actually send and receive the right output without doing dangerous things like interacting with a network connection or a database. We want the scope of our unit tests to be as tight as is reasonable so we can ensure the code under test actually performs the correct action every time.

This kind of guarantee around tests is called determinism. Deterministic tests are tests which, when provided a single input, always return the same output. Ideally any interactions which would break the determinism of our functionality should be stripped away and replaced with code which always does the same thing. This gives us guarantees and makes it easier to identify and test a variety of different cases. Let’s have a look at an example of breaking a dependency with Mockery and replacing the code with something else.

The beforeEach block sets up a fake MySQL module with a query method which immediately calls a passed callback. Mocking MySQL does two things for us. First it removes the asynchronous behavior which comes from interacting with a database, ensuring all of our tests run inside the event loop. The second thing this does for us is it allows us to provide a guarantee that the results passed back from our MySQL fake are always the same. This means we don’t have to set up and tear down an actual database instance. We can simply test the code we care about and assume the database library does what it is supposed to.

Important items to note about the code in the beforeEach and afterEach blocks: Mockery takes a certain amount of work to get working. The enable call in beforeEach starts Mockery working in memory and tells it not to post warning messages every time it does something. It is safe to assume, if you see the results you want, that Mockery is working. The code in afterEach returns the cache back to its original state, leaving everything clean for the following test.

Faking Router For Great Good

Now that we have looked a little bit at how Mockery works, we are ready to start digging into what we really care about. Let’s start testing our Express route actions. The first thing we should look at is a little bit of example Express routing code. Below is a simple route example which just receives a request and then responds with 200 and a little message. It’s not exciting code, but we can actually test it.

We can see a few things here which will be really important to get the tests right. First, Router is a factory function. This means anything that is going to stand in for express will have to implement this correctly. The next thing we know is, the router object which is returned has methods on it like ‘get.’ We will want our fake object to replicate this as well. Now is probably a good time to take a look at the source code for Express Route Fake.

Express Route Fake is a small module which packs a pretty massive punch. Everything is set up for us to quickly capture and test our route actions. The very first thing we have is a cache object which will store key/value pairs so we can request whichever function we want to test easily.

After the cache object we have a simple function which captures the route method, the route path and the route action. This function is the real workhorse of the entire module. Every route call will go through this one function and be stored in our cache. It is critical to understand that all of the methods, paths and actions will be captured and related so we can test them later.

Finally we have the Router factory fake, getRouteAction and reset functions. Reset is exactly what one might expect, it resets the cache to empty so the entire process can be repeated without starting with a dirty object. getRouteAction performs two critical activities. First, it verifies the route exists and throws an error if it doesn’t. Secondly, it passes back the route action so we can test the function outside of the express framework.

A side note on the getRouteAction error behavior; it is important and useful to get clear errors from our fake in this case. Over time my team has run into confusing situations because our implementation was home-grown and does not throw useful errors. This means we get an error stating “undefined is not a function” which does not really tell us what is wrong. By getting an error which informs you the route doesn’t exist, you can immediately verify whether the route is being created correctly and not need to troubleshoot your tests.

Putting the Setup Together

Now that we have the tools and have taken a look at them, let’s start constructing what our tests would look like. We know Mockery is going to substitute in our fake module and we also know that Express Route Fake will capture the actions. All we really need to do is a little setup to get things rolling.

In our setup we have a little bit of extra setup. Since Node and Express interact with the http module through request and response objects (typically called req and res respectively), we will need to create objects we can pass through and use as well. Considering the calls we are making on res, I just included the bare minimum functionality to actually test our route action: status, send and end.

Please note we are actually requiring the module under test AFTER we perform our Mockery setup. It’s really important to do this, otherwise Node will provide the actual Express module and our fake code won’t be used.

Now that we have our code set up and ready to go, let’s take a look at what our tests look like.

We can see that actually testing the actions, now, has become three simple lines of Javascript. What is happening in each of these tests is, our module is registering actions against routes, which are stored by our Express Route Fake module. Once the test starts, we simply ask for the function and execute it as if it were called by Express because of an HTTP request. We pass in our fake objects and capture the result of our action behavior. This allows us to see directly inside of our method and test the interesting parts, throwing away the stuff that would be, otherwise, obscured by frameworks, libraries and Node itself.

A second, important item to note is that we get extra guarantees around our route paths. If someone were to come along and change the path in our module, but not update the tests, or vice versa, we would get immediate feedback since getRouteAction would throw an error telling us the path does not exist. That’s a whole lot of security for a little bit of up-front work!

Summing Up

Today, wee looked at how to use just a couple of modules to insert a fake for Express and get better tests around our code. By using Mockery and Express Route Fake, you can capture route actions and get them under test. Moreover, you can do this while only writing code that is specific to the tests you are writing.

As you write more tests, it might become useful to create a factory for creating custom request and response objects which would simplify the test code even more. Of course, with all of this abstraction it does become more challenging to see what is happening under the covers. Ultimately, this kind of tradeoff can be useful in situations like this where repeated code is more of a liability than a help.

The next time you sit down to create new functionality and wire it into an Express route, consider starting off with Mockery and Express Route Fake. They will simplify the tests you need to write and limit the amount of code you have to change in order to get tests in place. Happy coding!

Comments Off on Unit Testing Express Routing

Anonymous Functions: Extract and Name

It’s really great to see functional patterns become more accepted since they add a lot of really powerful tools to any programmer’s toolbox. Unfortunately, because functional programming was relegated primarily to the academic world for many years, there aren’t as many professional programmers who have developed a strong feel for good patterns and share them with more junior programmers. This is not to say there are none, but it is important to note that most programmers think of functional programming and say “it has map, filter and reduce; it’s functional.”

Though having those three higher-order functions does provide a functional flavor, it is more important that there are higher-order functions at all. With higher-order functions come the use of anonymous functions. Anonymous functions (also known as lambda functions) provide a great facility for expressing singleton behavior inline. This kind of expressiveness is great when the function is small and does something unexciting, like basic arithmetic or testing with a predicate expression. The problem is anonymous functions introduce cognitive load very quickly which makes them a liability when code gets long or complex.

Today I’d like to take a look at a common use of anonymous functions and how they can cause harm when used incorrectly. There are many times that anonymous functions are assigned directly to variables, which actually introduces one of the same issues we are going to deal with today, but I am not going to linger on that topic. Please consider this a more robust example of why even assigning anonymous functions to variables is dangerous.

Jumbled Anonymous Functions – Our First Contestant

In Javascript, people use promises; it’s a fact of life. Chris Kowal’s Q library is a common library to see used in a variety of codebases and it works pretty well. Now, when someone writes an async function, it’s common to return the promise so it can be “then’ed” against with appropriate behavior. The then function takes two arguments, a resolve state function and a reject state function. These basically translate into a success and error state. I’ve created a common promise scenario so we have something to refer to.

Extract Method

The very first thing I see here that is a problem is, we have two functions logging an error. This behavior is not DRY which is a code smell and violates a commonly held best practice. There is a known refactoring for this kind of redundancy called “extract method,” or “extract function.” Technically we already have a function in place, so we can simply lift it and name it. This will reduce our footprint and make this code cleaner already. Let’s see what this would look like with our logging behavior extracted.

With this simple extraction, we now know more about what our function does and our code has become more declarative. Although logError is a one-line function, the fact that it does exactly one thing makes it both easy to reason about and easy to test. We can inject a fake logger and capture the logging side effect, which gives us direct insight into what it does. Another benefit we get is that we can hoist this function further if need be, so we can reuse it across different modules or files.

Debugging Problems

Now we get to the real nitty gritty. We have two anonymous functions which do not explicitly tell us what they do. Instead, they just contain a much of code which performs references into an object. We run up against two different issues because of this. First, the lack of declarative code means the next person who looks at this, which might be you, will have to sit and stare at this to understand what is happening.

Another, bigger issue than immediate comprehension is debugging. Suppose we take this file and concatenate it with all of the other files in our project and then uglify the whole thing and deploy it out for use in someone’s browser. All of our code now lives on a single line and may not even have meaningful variable names anymore. Now, suppose one of the data objects comes back null. Our debugging error will contain something like “error at line 1:89726348976 cannot treat null as an object.”

This is bad, bad news. Now we have an error which we can’t easily identify or triage. One of the calls we are making no longer does what we think it does and it’s causing our code to break… somewhere. Whoops! We can actually use the same pattern we used for our error logging to extract our methods and make sense of the madness. Let’s take a look at what our refactoring would look like.

Now that we have lifted our last two functions out of our promise chain, everything makes a little more sense. Each of our behaviors is easy to reason about, we can test each function independently and all of our functions have a unique identifier in memory which saves us from the insidious debugger issue which can cost time and money.

There are other places we could go from here with our code to make it more fault tolerant, but that’s outside of the scope of this article. Instead, when you look at your code, see if you can easily understand what is going on. Look at it like you’ve never seen it before. How many anonymous functions are you using? How many different steps are crammed into a single function?

When you see this kind of muddy programming, think back on our reduction to simpler functions, avoid complex anonymous functions and think “extract and name.”

Comments Off on Anonymous Functions: Extract and Name

Five Things That Will Improve Your (Javascript) Code

I see lots of discussion around new frameworks, libraries and other odds and ends, which claim to make your code better, cleaner, easier to maintain, etc. Although frameworks are definitely useful for avoiding reinventing the wheel and libraries help clear out boilerplate you would have to litter your code with, the cake they offer of better code is a lie.

As client-side development in the browser becomes more focused on solving application-sized problems, the days of merely dropping in a plugin or a library to accomplish what you need are coming to an end. Javascript living both in the browser and on the server further leads to large, potentially complex code bases. If we simply continue working the way we did 10 years ago, you end up with a spaghetti mess that is completely unmaintainable.

Don’t panic!

There are things you can do to save yourself from impending doom. There are actually five things I recommend every Javascript programmer do to make their code and their lives better. It’s a lot like reinventing yourself, though, it takes work and dedication, but it is worth every ounce of work you put in. I would do a countdown, but I always recommend these items are done in order.

1. Test Driven Development (TDD)

If there is one thing on this list you should start doing immediately, it is test driven development. TDD allows you to define, up front, what the business requirements are that your code should adhere to. You actually write tests before you write code.


This means, first you write a test and run it. That test should fail. If your test doesn’t fail you are either writing tests against existing code (not TDD) or you wrote a test that tests nothing. Make sure your test fails first. Once your test fails, write just enough code to pass that test. Run the test again. Green means passing. When your test goes green, your code is good. Once you have written enough code to get messy, refactor, ensuring your tests continue to pass.

It doesn’t really matter which testing framework you choose. At my work we use Jasmine, but Mocha is fine as is Tape. There is always a flavor of the week testing framework, so pick one that makes sense to you and use it. The choice is completely up to you.

2. Static Analysis

Static analysis is, basically, a program that checks your source code against a set of rules and warns you of potential errors and bugs in your code. The sooner you can get static analysis going on your code, the better. I recommend you set it up before you even write your first unit test.

Current popular static analysis tools are JSLint, JSHint and ESLint. I prefer JSHint only because I have used it and I’m very familiar with the tool. ESLint is the newest of the three and people swear by it. Both support ECMAScript 2015/2016 syntax. JSLint is the oldest and I am unsure as to the progress regarding new syntax. Douglas Crockford is a busy guy and he may or may not have the time to maintain it.

Another way to get static analysis into your project is to use a language designed for transpilation. Transpilation is the process of compiling source code into Javascript. The output could be either human-readable code or ASM, though I tend to prefer human-readable output for no particularly good reason except ASM makes me think of Emscripten which makes me cringe.

Languages which will provide good static analysis include TypeScript and Elm. TypeScript allows you to define the type contracts your functions and methods will adhere to, which means you get some good information up front about the safety of a function call. Elm is a functional view-layer language with very strict code and type rules; because of this Elm provides greater code stability guarantees.

3. Functional Programming

The kind of functional programming I am talking about is not just introducing Underscore or Lodash into your project, but really embracing the ideas that make functional programming great: immutability, no side effects, function composition and function abstraction.

Even though Javascript variables are mutable, by treating them as immutable, you add stability to your code by avoiding changing state under your own feet. When a value is set and remains as it was assigned, you can rest assured that your code will always behave the same way when you refer to that variable.

By eliminating side effects in as much of your code as you can, you create well defined units of code which behave the same way every time and, better yet, are easy to test. This makes the first item on our list even easier to satisfy which makes your program better.

Function composition is the process of creating small abstracted units of code as functions without side effects which are then put together to do more complex work. This kind of modularity and abstraction makes it much easier to test and, when necessary, debug your code.

4. Data Structures and Algorithms

In any computer science program, the data structures and algorithms course is considered a critical course in computer science thinking. By getting familiar with the well known data structures and algorithms, you lay a foundation upon which you can build your knowledge and skills which will help to more quickly analyze and work with business concerns as well as existing code which will start to display recognizable patterns.

Data structures reach beyond the basics of variables, arrays and objects and dive into the concept of lists, stacks, queues and trees. These complex structures provide much cleaner, smarter solutions to common problems and can provide insight into problems which might be hard to identify without this kind of core understanding.

In much the same way that data structures provide data-related solutions to problems, algorithms provide insight into code and how to build in efficiency and good structure. Topics like sorting, searching and working with complex data structures will give a functioning foundation for how to integrate data solutions into your projects.

5. Design Patterns

Finally, to really cap the knowledge you have gained from the rest of the list, design patterns are a set of solutions which have been discovered and well documented. Design patterns tie together all of the practices with testing, abstraction, data solutions and algorithms and provide a well known structure to add to your program as common problems and patterns emerge.

As evidenced by my list, design patterns are not where you begin, but where you enhance. Good use of design patterns will enhance well-architected code and provide clarity when the going gets rough. These patterns are not so much a law as a guideline to help make good programs better and bad programs stable.


There are many other important practices I could have thrown into this list, like polygot programming and theoretical studies, but these things are only useful once a strong foundation is set. In studying these five major topics it is highly likely you will encounter different programming languages and different schools of thought. This is a good thing.

The greater the number of perspectives a programmer is introduced to, the better they are bound to become. This list is not something a good developer can go through once and be done. Instead it is a cycle which should be recognized and embraced. By learning, developers grow and by growing, developers improve their world.

Hopefully the topics I presented here resonate with you and you share this with others who want to continue their journey to be their best. Even the journey of a thousand miles is started with a single step, so take your step and become better every day.

Comments Off on Five Things That Will Improve Your (Javascript) Code

Code Generation and You

A friend of mine and I have discussed code generation on several occasions. We both agree that any enterprise development process should involve code generation, without exception. Though it is possible for development from scratch may not provide enough identifiable boilerplate to justify building templates for your code, the moment a framework is in use, you will, undoubtedly, have a certain amount of boilerplate introduced which must be set up repeatedly for each new file you create.

I work with a team using Angular. We have certain style requirements for code, and Angular has a particular way for handling the setup for every code component you would create. Beyond this, we have certain patterns which have been uncovered which introduces sets of modules which we need to construct, which introduces even more boilerplate.

I have timed the setup for a standard controller and the baseline unit tests which need to be created before any new development and on average it takes about 10 minutes to create all of the files and type all of the boilerplate. For a one-time cost, this is a minimal requirement, but when working in an enterprise environment, this can add up to many lost hours each month.

Here’s the worst part of losing those hours: no problem was solved in that time.

This means for any given person who is creating a new module, they are performing rote behavior over and over without adding any real value to the project. Programmers are not paid to simply pound keys. If key pounding was all that was necessary to create an application, they would pay the smallest wage possible. Data entry is a good example of a key-pounding job. I think we can all agree that programming is more than simply performing data entry.

If this kind of rote file creation and boilerplate typing is the most basic of work, then why do we continue to do it? It seems as though rote behavior is something we often look to automate, which is why the profession exists in the first place.

I say, are we not programmers?

Automation is the name of our game, so it seems reasonable that we would want to take this wasted time and have the computer do it for us. This is what computers are for. Fortunately, this is a solved problem!

My team uses Yeoman to create our boilerplate. We have a defined application file structure, and the modules we create always have a certain amount of boilerplate associated with them. Yeoman is really, really good and taking templates and creating files with them. Here’s an example of what an Angular controller (in ES6) looks like, with no interesting bits:

That is about 20 lines of boilerplate, and we haven’t even created the unit tests for it, yet! Beyond that, most controllers are associated with a view or a directive, which means we have at least two or three more files, full of boilerplate, left to create. If we were using ES5, there would be even more code here that produced nothing of new value. This is not the reason we got into programming.

Let’s take a look at what a Yeoman template would look like instead.

This is so similar to the code we wrote above it’s difficult to see where one leaves off and the other picks up. Here’s the big win, though, We spent 10 or 15 minutes creating this template and now we never have to do it again!

Let’s have a look at what the actual script looks like to populate this controller:

That’s it. This might not be the fanciest Yeoman generator, but it reliably outputs a controller, with the correct parameters filled, every. single. time. With just a few more lines of code and another couple templates for our tests and views, we can type one line into the shell and get all of our working files spun up in seconds.

Let’s do a rough, back of the envelope comparison so we can see the amount of time saved by using generators. Let’s say you have a small to medium-sized SPA you are building and it contains 150 Javascript source files. On top of that, you will have unit test files for each of those source files, so that’s another 150 files. Atop all of that, you have views that associate with, let’s say, 1/3 of those files, so that’s another 50 files. Let’s say, for argument’s sake that it takes 3 minutes per file to generate these files by hand.

350 * 3 / 60 = 1050 / 60 = 17.5 hours

Now, let’s assume you created each of these files with a generator, and let’s assume your computer is slow, so it takes 1.5 seconds per file. Let’s do a little more math so we can see how long this takes in comparison:

350 * 1.5 / 60 = 525 / 60 = 8.75 minutes

Let’s take the value that hired says is average pay for a Javascript developer in Los Angeles, $130,000US/year, and divide it by the number of hours in the average work year, 2087. This means we have an hourly rate of about $62. Now, let’s multiply that $62 by 17.5 and we get $1085. That’s some expensive boilerplate!

With our same numbers, a developer is working for a little more than $1/minute to generate boilerplate. Let’s assume this same developer generated all of their code with templates instead of by hand. Now our code cost around $10 to generate on a slow computer. That is TWO ORDERS OF MAGNITUDE different.

People like to talk about exponential growth and this is exactly what we have here. Using code generation versus writing boilerplate by hand literally increases the cost of each base file exponentially by a factor of 2. Hand-typed boilerplate is 100 times as expensive.

The short version of all of this is, if you aren’t using code generation, you should be. If your manager tells you that it takes too much time to get code generation integrated into your project, just send them to me. Does it take too much time to decrease the cost and time of production by a factor of 100? I think not.

Comments Off on Code Generation and You
Similar posts in Coding, Javascript, Unit Testing

Testing Promises: Promise Fakes

Javascript developers notoriously say unit testing in hard. I think the problem is actually more specific than that. Unit testing pure functions and business logic actually pretty easy.You put a value in, you get a value out. If the value you get back is what you expected, your test passes. Something that is hard to unit test is asynchronous logic.

Asynchronous logic is so hard to test, the angular team actually built a special mocking system for testing calls made through the $http service. In reality, though, if your unit tests are littered with $httpBackend references, you’re doing it wrong.

I won’t go into philosophical discussions about promises and callbacks, so let’s just agree that people use promises. A lot. Promises have become the currency for modern asynchronous requests, whether to another thread or across the internet, if you have to wait for your execution to come back from an asynchronous behavior, you might see a promise in the mix. Your code might look a little like this:

Let’s cut to the chase, promises are easy to throw into the middle of your code, but they are really hard to test… until today.

In unit testing there is a concept known as fakes. Fakes are often, mistakenly called mocks, but mocks actually store something to use for later when you are handing your test expectations. No, fakes are just what they sound like: a fake something.

Fakes contain the minimum amount of code to satisfy your functionality and nothing more. Sometimes fakes are nothing more than an empty object or function. Other times they are a little more involved, doing stuff like calling passed functions and returning values, but at the end of the day, a fake is a placeholder used for guaranteeing your unit under test won’t fail and will be isolated.

So, promises and fakes.

It is a law of unit testing that you do not talk to the outside world and you do not talk about fight club. This means, if you have a function that is going to call a service which will, in turn, interact with the world at large, you have a big problem. Fortunately, you probably wrapped that up in a promise and that is the crack in the armor we can use to break our unit out into its own isolated space.

Underneath it all, promises are nothing more than objects with a bunch of trickery hidden inside. With the knowledge of fakes and a general understanding of the promise syntax, we can build a stand-in object. A stunt promise, if you will. It’s pretty common to use Chris Kowal’s concept of promises, as developed in Q, so let’s build our fake around that.

Here’s what a promise fake might look like:

It’s about 40 lines of code, but now we have something we can work with. This promise fake can be instantiated for each test we write and it won’t muddy the state from test to test. It’s chainable, so if there is code using chained promises, it can be tested. Finally, success and failure states can be set with errors and response values so any function that might be relying on a specific response value will be testable with little extra effort.

Faking can be hard work, but if you do it right, you only ever have to do it once. Hooray! Now let’s look at a test for our method.

That was easy! We didn’t have to do a whole mess of monkey patching to our code and we didn’t have to use some crazy mechanism to intercept HTTP requests way down the stack. Fakes are great, when used for the powers of good. The goodness doesn’t stop there, though. Here’s how we can actually test our promise fake actually works as expected.

That’s pretty much it!

We had to do a little grunt work at the beginning, but after we built our class, we could fake promises all day long and save ourselves headaches forever more. The best part is, now we have eliminated the asynchronous behavior from our code and made everything testable. This makes our tests easier to read, easier to maintain and clearer to write. Who could argue with that?

What this all really leads us to is this, promises are tough to test, but when we absolutely, positively have to have them, we can trim the fat, clean out the code and fake our way to a brighter tomorrow. Isn’t that what we all really want? Go write some tests and make your code a better place to live.

Mainstay Monday: SOLID – Dependency Inversion

This post is part of a series on the SOLID programming principles.

This is it, the final SOLID principle. Dependency inversion is probably one of the most talked about SOLID principles since it drives so much of what we see in programming today. From Inversion of Control (IoC) libraries to the dependency injection in modern Javascript frameworks like Angular and Ember, popular OO programming has embraced this principle more than any other. It’s possible that the success of dependency inversion is related to the fact that it can be enforced with a technical solution.

Let’s start by talking about what dependency inversion is. As Uncle Bob Martin says, dependency inversion is the dependency on abstractions instead of concretions. More specifically, this means that any object or function should not rely on the existence of specific concrete parts of an object, but instead, it should expect and use a contract while letting an outside entity define the concrete object that will be used.

To demonstrate this idea, let’s first take a look at a counter-example to our principle. This is probably one of the most flagrant violations of the dependency inversion principle I have encountered in Javascript and it is right on a popular examples page. The following example is lifted directly from that examples page:

Clearly, this is an example from the Backbone site. Backbone, you’re doing it wrong.

If you look back to my post on dependency injection, you’ll see we could easily create a factory for each of these instance-creation lines. It could be simple, like the following code:

It’s a few extra lines of code, but the separation of concerns you get can make a huge difference when you write unit tests or if the creation of a message or mailbox ever becomes more complicated than simply calling new. By inverting the dependency chain on this small example, we centralize our creation, eliminate the new keyword from our code and provide a facility for easily injecting a substitute factory to help in testing that our code does only what it is supposed to do.

The other thing that happens when we break out the creation logic is, it becomes obvious what is really happening in the code: we are creating a messages object that is, really, an event handler. Now we can isolate this behavior fully and put guarantees around the actions our model will take when we trigger certain events.

Let’s take a look at the other side of the picture and write a little Jasmine code to test our message model.

If we had tried to do that without our factory, it would have been a lot more challenging to wrap everything up in tests. Since we broke out the new object from the mailbox definition testing became as easy as replacing our definition for the factory and we got direct access to everything we needed inside the object setup.

Finally, when we separate our object instantiation out from the core body of our code, it becomes much easier to modularize all of our code and separate pieces as they become unwieldy. This gives us better guarantees around the stability of each part of our code, since each module can be written and tested independently, breaking the code apart at intentional seams.

The takeaway from all of this is, when we invert our dependencies in our code and rely on abstractions to define our program, we improve stability, guarantee contract integrity and improve the testing story. All of these things add up to better programming and more stable output. Isn’t that really what we all want?

Comments Off on Mainstay Monday: SOLID – Dependency Inversion

Contracts for Better Code

With programming languages which have a greater draw for classically trained computer science types, there is a common discussion of contracts and data expectations. Contracts are so inherent in the code that it’s hard to see a declaration without one. Type systems emerge from the idea of contracts and every function, constructor and return path comes with an expectation that is defined and declared in the code.

Javascript, being dynamically typed and a little loose with the morals, attempts to sidestep contracts altogether. Function arguments are little more than a strong suggestion to the programmer, and every function returns something even if that something is merely the undefined type. Contracts are just not something we do.

Here’s an example of exactly how squirrely Javascript can really be.

Obviously each of these functions does essentially the same thing, but in the last example a and b are nothing more than a suggestion of how you could use the function. You could run addSuggestion() and get 0 or you could run addSuggestion(1, 2) and get 3. There’s no requirement that you actually adhere to the contract at all.

You are doing it wrong.

There, I said it. Mucking about with all of these bits and pieces that may or may not exist and allowing programmers to play fast and loose with your carefully constructed function is just plain wrong. I makes me want to take a shower. It’s like the midnight movie on the horror channel: gross.

Rule number one of contract club: You ALWAYS talk about contract club.

If someone created a contract in their function they are setting expectations. If you don’t play by the rules someone set in the function, you should not expect the function to work properly. It’s just that simple. The contract is there to save you from yourself.

At this point, I think I should mention, I understand that Javascript doesn’t support function overloading, so you can’t create optional variations on a function and the loose requirements around the contract are there so you can get something akin to overloaded functions.

To this I say hogwash!

Actually that’s not true. Optional arguments are good, however it is better if we use them in a safe way. Overloaded functions, even in languages that allow them, can get dangerous. It’s preferable to write code that says what it means and does what it says. Let’s take a look.

That may not be the most beautiful code I’ve written, but it illustrates the importance of what I am saying. Here we can see that there is a function, buildUrl, which takes four parameters. Hostname is required, but all of the rest are optional. Once we get to the specific implementations of what we are actually doing, the contract becomes a firm handshake and it’s backed by the interpreter threatening to throw an error if something goes wrong. Mind you, the interpreter is going to just concatenate a whole bunch of undefined values, but that’s beside the point. You won’t get what you want if you don’t meet the contract.

So, there is another side to the contract that is also illustrated here. Regardless of what happens, you can guarantee you will always, ALWAYS get a string back when you run buildUrl. This is the promise made by the person who wrote the code before you came along. So, on the one hand, you must meet the requirements of the contract in order for the function to properly execute. On the other hand, you are allowed righteous indignation when you expect a string and you get a boolean or something else.

Return types are contracts.

When you, as the developer, write a function and claim you are returning a specific type, understand that the next person will hunt you down with hungry dogs if you promise a string, but sometimes return an object. What is returned is really, REALLY important. People rely on it. Imagine if this happened:

I mean, that was downright malicious. Who goes around saying they are returning an array and then they return null. That’s seriously sadistic. What’s worse is, if listify was buried somewhere in a library and all you had was the crummy documentation they wrote, you would never be able to figure out what you are doing wrong to cause listify to return null.

I dunno, it just blows up sometimes.

The short version of this post goes a little like this: When you write a function, you are writing a contract. That is a contract you are required to honor and you can, almost always, expect the programmer who uses your function to adhere to that contract.

The longer sum-up goes more like this: A contract is a guarantee. The guarantee that is given states expectations for what the user will do and it provides assurances about what the user will get in return. Contracts revolve around data, and everything is data. This means that regardless of what you are writing or what you expect, in Javascript, you should always say what you mean and do what you say.

By providing a strong contract for the user to rely on, you are making the world a little better place to live in. You are giving guarantees when so many programmers around you might not. Best of all, when you write a good data contract in your functions you can come back after a year of looking at something completely unrelated and get back up to speed almost instantly on what goes in and what comes out. Doesn’t that sound like better code? It does to me.