Extending Functions with Decoration through Composition

In object oriented design, the decorator pattern is commonly used to extend and enhance classes so they perform some new or more refined functionality. In functional programming it is possible to decorate functions as well. The decoration must follow a few rules, but the result is a very powerful technique to enhance functions statically and at run time.

At the core of functional decoration is function composition. Composition is a straightforward practice that relies on pure functions which take a predictable input and output. A trivial example is something like the following:

This is so foundational to what we know of math and computing there is actually a special notation in mathematics to compose functions this way. It’s common to see this kind of thing annotated as f ∘ g.

I was trying to think of a toy problem that could be used to demonstrate the kind of power we can get from function composition, but then it dawned on me. A great example of real powerful decoration through composition can be demonstrated through a very common problem in statistics: Computing x-bar for a set of data points.

We actually already have almost everything we need to do it. Let’s create a divide function to round out the basic functions we will need to compute x-bar.

That’s it. Let’s do some statistics.

The first thing we are going to need to compute x-bar is a simple mean. This is typically referred to as an average in daily life. We’re all pretty familiar with taking a basic average: take the sum of all values and divide by the number of values. Let’s build the simpleMean function.

SimpleMean is our first big example of function decoration through composition. As it turns out the line gets rather fuzzy when dealing with composition and decoration. The important distinction to make is when a function is decorated, the new function will honor the original function contract. What we can see here is sum takes a list and returns an integer. SimpleMean also takes a list and returns an integer. Most importantly, simpleMean wraps up the behavior of sum with another function to extend the behavior of sum without modifying or duplicating it.

Let’s dig a little deeper into our statistical function and create a new function that normalizes the values in our list using the simpleMean function and map. It’s really important to note that normalize values is a composite function but it is not a decorated function. Although we are using simpleMean to create our new function, the resulting contract does not adhere to the original simpleMean contract.

By creating this new function, we are now ready to start piecing together our final function which will provide x-bar. X-bar is the sum of the squares of normalized values. We have our normalized values, so let’s get to squaring and summing!

The power we are ultimately seeing here is something that comes out of strong composition and an adherence to the contracts of our functions to produce richer and richer behaviors with only one or two lines at a time. The important thing to note while looking at these functions is the extremely limited use of variables. By building functions through composing and decorating functions, state is eliminated and the spaces where bugs and incorrect logic is reduced to the smallest footprint.

As you work on new systems of behaviors, think about what your goal is and break it down into small, easy to understand steps which can be composed to create a powerful, easy-to-understand function which avoids state and provides fast, crisp, high-quality behavior.

Mainstay Monday: Hiatus

I have spent the last several months writing two blog posts per week. I have really enjoyed writing both the Mainstay Monday and the regular Wednesday posts, but time is a limited resource. I have started a new writing project which will consume a considerable amount of time, so I had to make a choice: I could either continue writing two blog posts a week, or I could pursue my new project.

By cutting back to a single post per week, I free up enough time to pursue my new project while maintaining my blog regularly with high-quality content. Though this means I will publish less frequently, it will serve not only me, but the Javascript community at large in the long-run.

There is no current estimate when Mainstay Monday will resume, but it is not gone forever. I am working on sorting out how I will present foundation material, and whether it makes sense to ever resume the two-post pace. Only time will tell, but in the meanwhile, we’ll meet on Wednesday, just like always!

Similar posts in Mainstay Monday

Speeding Up Your App with Request Caching

Recently in my Mainstay Monday posts I’ve talked about creating linked lists and queues in Javascript. Each of those topics is so dense that just writing about the basics of creating and using them took full blog posts. Really all of that was the lead up to today’s post.

I’ve worked in a couple of different, large single page applications and in every one I have, ultimately, encountered a need to cache requested data and respond to multiple functions needing that data before the original request is complete. No promise framework or other specialized library ever fit the need because, really, the call should only ever be made once.

I’ve had team members solve this problem in a naive way and simply make the call over and over again. What happens, though, if the data you’re requesting is large and takes a long time to retrieve? This means you have now introduced multiple seconds of latency into your app. I think we can all agree that waiting for 5-20 seconds for data to come back is about the worst thing you can do to a user. They get frustrated and confused. They think the app has stalled or their browser has crashed.

Problem #1: I need to store the data when I get it.

This is easy. If you just need to store the data and retrieve it upon request, you can create a hashmap with keys and values. Now, when a request comes in, first your data service will look in the hashmap and see if the data already exists. If not, you go fetch it, bring it back and then, upon return, you hand the data back into your app.

This is the most basic kind of cache and it will suffice for calls that are made infrequently. Initial data for your app to bootstrap can be handled this way. Typically a single request will be made and you can just go fetch it, then cache it. If the app wants to rebootstrap for some reason, the data is in memory and you can skip the wire call.

The more challenging issue is this: what if you have data that is requested often?

Problem #2: I need cached data to only be requested once, but the program asks for it everywhere.

This is where it gets really interesting. It’s quite common for a program to need to refer back to permissions and user capabilities often. ACL tables can get quite large and it is preferable to only request these once. The problem is, the program will need access, possibly multiple times, for even a single page. This means your application will request the same data multiple times before the service you’re accessing can return.

I’ve seen a page make 100+ requests at the same time to get data from a service. It’s not pretty.

Fortunately, queues provide the solution for this. We can queue all of the callbacks that our application generates and resolve them at once when we get the data back. This means we could, in theory, request the data on app bootstrap and never request it again. Worst case scenario is we request it just in time and the user has to wait once.

This is where the real meat of the problem is. We need to construct a queue backed request system with a cache layer to manage our data. This all sounds a bit scary but, once we break it down, it’s all just bite-sized pieces we can easily manage. We have even already decided on the data cache structure.

Before we start down this road, I would like to point out, a friend of mine introduced me to a rule I use all the time. It makes testing easier and it makes coding easier.

Never create and use an object in the same place.

The easiest way to answer this rule, for our current problem is with the factory pattern. Our factories are going to be relatively uninteresting, but because it creates good seams to work with in our code, and it does a nice job of separating our concerns so we can reasonable, correct abstractions.

So, since we know our mechanism is going to be queue-backed, we need a linked list. I went ahead and created a linked list item factory as a gist. It creates generic linked list items, so you could really use it for anything. We’re going to use it to construct our queue. Here’s our first factory, linked list items:

Once we have our linked list item factory set to go, we are ready to construct our queue. Once again, we are going to want to work with a factory. Our queue logic comes straight out of the queues post, it’s just wrapped up in a factory so we can easily separate it and work with it alone. Here’s our queue:

Now is where we start breaking new ground. Our cache, and its factory, are going to handle a few things. Consider this a little like learning to juggle. We have to get things in the right order, and it’s all interconnected, so you might want to read the code a couple times. Let’s have a look at the cache factory and then we can talk about it.

The short and sweet version of this is, we receive a local request for data, if it is in the cache, we resolve the request and move on. If we don’t have the data, we queue the callback with any other outstanding callbacks and wait for our service to return the data we need. In order to ensure we don’t have overlapping concerns, we rely on a cache instance per unique data request. Basically, this means if you make a request with { id: 1 }, it should always go through the { id: 1 } cache. This way if your application needs to come back later and request data using a different id, it can without colliding with the original data cache.

To expand this idea, let’s take a look at the steps that happen.

First, we have a cache factory. The factory takes in a request function, which it assumes only needs a callback to be complete. With this function, it news up an instance of the cache object. By using a factory, we can guarantee instantiation correctly, every time. Here’s what it would look like in code:

I’m assuming permissionService is already created and get is a method to perform a standard HTTP GET. Honestly, this could be anything, just as long as all of the correct parameters are bound.

It’s important to note that no request has been made yet. All we have done is create a cache we can make requests against. Our app is still sitting there patiently awaiting instructions for what to do. This entire setup process takes microseconds to complete and you now have everything you need to manage bursts of traffic gracefully.

Once we have our cache ready to go, we can make a whole bunch of calls and it will only make a single call across the wire. This gives us a major performance boost, which allowing our app to carry on blissfully unaware that anything has changed. Here’s what it looks like:

The first request, regardless of which it is, will cause our service to create the cache. Each subsequent call will just end up the queue. Once the request comes back from our remote service, all callbacks will be resolved in the order they were received. Even better, if something else is kicked off in the meanwhile, this will simply be added to the queue and all is set and ready to go.

Adding this kind of data management to an application adds some complexity, so it may not be worthwhile to manage all data requests this way, however, when a particular behavior starts making lots of requests across the wire, this is a great way to throttle the requests back and improve efficiency in your app. The best part, as long as you are working modularly, is that you will only need to introduce changes in one place. This means the bulk of your application will remain precisely the way it is today while repairing a bottleneck that can slow your app down and frustrate users. So, get to profiling your apps and cache some data.

Mainstay Monday: Sorting

If you’re reading this you’re likely already a programmer. This means you have likely used [].sort() in your code many times. I honestly can’t remember how many times I’ve sorted data one way or another. Just because you’ve called sort on an array doesn’t mean you’re necessarily doing the best way you can. Does the following code look familiar to you?

If that code looks familiar, today is the day you stop. Javascript’s Array.prototype.sort function is actually a rich function that will allow you to specify sorting far beyond just ascending search. Although reverse has its uses, the example above is just plain abuse. Stop abusing your code!

Let’s take a look at how we can make use of sort in a smarter way. Sort will take a sorting function which it uses to compare two values to decide their correct order. You can define your sorting any way, but let’s start with just sorting numbers. Here’s our same array and a standard, ascending sort, but we’re actually defining the direction of the sort by hand. Let’s take a look what this expansion would look like:

Yep, that looks like extra work to accomplish the same thing. SortNumbers just does the standard comparison and returns the array, sorted as expected. The win we get here is we can now specify the sort directly. Let’s have a look at reversing the sort:

If we are lucky enough to only ever have to sort numbers, this knowledge isn’t particularly helpful. We eliminated a linear algorithm for the sort behavior which might have shaved a couple milliseconds off the total processing time. No big woo, right?

Actually it is. Have a look:

Our little abstraction made specifying the sort really, really easy. This means you could actually change the sorting behavior at runtime based on user input! Now, that’s pretty useful.

Sorting is definitely not limited to numbers. Strings are another commonly sorted array. Much like numbers, strings have a predefined comparison inside Javascript. That said, we can’t simply subtract them if we want to reverse the order. Strings have an ordinal number (numeric) value. This means you can’t subtract strings, because it’s meaningless, but one string can be greater or less than another. Here’s how we would perform a reverse sort on a string array:

Now we can see, a little more clearly, what sort is really looking for. Negative numbers move values to the left, positive numbers move values to the right and zero means the values are equal. This is very helpful information we can capitalize on to do more interesting sorting. Suppose, instead of sorting trivial arrays, we wanted to sort arrays of objects.

A common sorting happens when objects have an ID and we want the objects in ID order. By understanding how to sort strings, which have an inequality comparison operator, we can create a function that gives us the meaning objectA is greater, less than or equal to objectB, i.e. objectA < objectB. Supposing we were comparing two objects, objectA = { id: 1 } and objectB = { id: 2 }. We (might or might not) know that objectA < objectB === objectA > objectB because both inequality operators evaluate to false for objects. With that in mind, we know that objectA.id < objectB.id === true. This is what we are going to use to write our object sorting function.

This is the first time we really couldn't just use Array.prototype.sort on our array. This problem can only be solved with a specialized sorting function. This is where the power provided by the built-in sort really shines. We can actually define data comparisons on the fly, which means we can actually create a much richer experience for our users with a lot less code. Let's actually take our object sorting one step further. Suppose we wanted to sort an array of people. The most common way this kind of list is sorted is by last name, then by first if the last names are the same. This leads us into uncharted territory. If you were simply relying on the basic sort, you would end up sorting, partitioning and sorting again. Let's have a look at how we might solve this problem.

Now, that's some good sorting! We have all of the names in the right order with just a little bit of work. All of a sudden what looked like unnecessary abstraction becomes a big win. We sort on last name for every record, but we only sort on first name if the last name is the same. This allows us to not only sort complex arrays, but do it in a smart way to guarantee the best performance we can get. This algorithm is great if you only need to sort first name and last name. What if you actually want to sort on a set of different keys? More importantly, what if you want to sort on a set of keys that are specified at runtime? This is a new, interesting problem which relies on what we learned with our people array. Let's generalize.

This is a good start, but we still have a problem. We are rebuilding our sort function array on every step of the sort. For small arrays, this is probably fine, but when our arrays get big, this is dangerous. We will start seeing a bottleneck and it will be difficult to identify. Let's use the factory pattern to retool our function and get some really great code.

With all of this complexity, we are actually missing one last piece of the puzzle, a reverse sort. Even with our most complex sort managed by our reducedComplexKeyedSort, we might still want to reverse the entire sort. We agreed at the beginning that calling .reverse() on a sorted array is kind of a gross, hacky way to do things. Fortunately, reversing the order is really, really easy. All that has to be done is multiply the sort outcome by -1. Here's some evidence:

I'm not going to walk through a formal proof, but the evidence is pretty compelling. This means we could actually write a function which will return a reverse sort function. Now we only have to know how to sort one direction and we can actually switch between directional sorts easily. Here's a sort reverse function:

Let's use our reverseSort to reverse the order of our most complex sort constructing our sort function from the ground up.

By abstracting out our object sorting behavior, we have taken an object we know nothing about and produced a sorting algorithm which will sort our objects in reverse by keys in a ordered, refined way. That's some pretty powerful stuff. To sum up, sometimes we can get away with simply using the built-in sort, and we can even hack in a reverse to give us ascending and descending sort behavior, but the world of array sorting is large and full of twists and turns. We've introduced a simple way to address sorting which requires more than just relying on the language supported comparison, which will likely carry you through most common sorting tasks. We have also introduced a higher-level abstraction for defining complex sorting. Finally we developed a higher-order function which allows us to easily reverse any sort function. This kind of development provides a strong way to reduce the amount of code you write and enhance the functionality of your program while you do it. You can look back at the code you've written and refactor it to be more maintainable and simpler to build upon. Now go forth and do some great sorting!

Blog Notes

I created a final factory for handling complex sorting with ascending and descending order handled in a SQL style. This will allow for sorts like the following:

keyedSortFactory.build([‘column1 asc’, ‘column2 desc’, ‘column3 desc’]);

Please see the gist here:

Keyed Sort Factory

Five Things That Will Improve Your (Javascript) Code

I see lots of discussion around new frameworks, libraries and other odds and ends, which claim to make your code better, cleaner, easier to maintain, etc. Although frameworks are definitely useful for avoiding reinventing the wheel and libraries help clear out boilerplate you would have to litter your code with, the cake they offer of better code is a lie.

As client-side development in the browser becomes more focused on solving application-sized problems, the days of merely dropping in a plugin or a library to accomplish what you need are coming to an end. Javascript living both in the browser and on the server further leads to large, potentially complex code bases. If we simply continue working the way we did 10 years ago, you end up with a spaghetti mess that is completely unmaintainable.

Don’t panic!

There are things you can do to save yourself from impending doom. There are actually five things I recommend every Javascript programmer do to make their code and their lives better. It’s a lot like reinventing yourself, though, it takes work and dedication, but it is worth every ounce of work you put in. I would do a countdown, but I always recommend these items are done in order.

1. Test Driven Development (TDD)

If there is one thing on this list you should start doing immediately, it is test driven development. TDD allows you to define, up front, what the business requirements are that your code should adhere to. You actually write tests before you write code.


This means, first you write a test and run it. That test should fail. If your test doesn’t fail you are either writing tests against existing code (not TDD) or you wrote a test that tests nothing. Make sure your test fails first. Once your test fails, write just enough code to pass that test. Run the test again. Green means passing. When your test goes green, your code is good. Once you have written enough code to get messy, refactor, ensuring your tests continue to pass.

It doesn’t really matter which testing framework you choose. At my work we use Jasmine, but Mocha is fine as is Tape. There is always a flavor of the week testing framework, so pick one that makes sense to you and use it. The choice is completely up to you.

2. Static Analysis

Static analysis is, basically, a program that checks your source code against a set of rules and warns you of potential errors and bugs in your code. The sooner you can get static analysis going on your code, the better. I recommend you set it up before you even write your first unit test.

Current popular static analysis tools are JSLint, JSHint and ESLint. I prefer JSHint only because I have used it and I’m very familiar with the tool. ESLint is the newest of the three and people swear by it. Both support ECMAScript 2015/2016 syntax. JSLint is the oldest and I am unsure as to the progress regarding new syntax. Douglas Crockford is a busy guy and he may or may not have the time to maintain it.

Another way to get static analysis into your project is to use a language designed for transpilation. Transpilation is the process of compiling source code into Javascript. The output could be either human-readable code or ASM, though I tend to prefer human-readable output for no particularly good reason except ASM makes me think of Emscripten which makes me cringe.

Languages which will provide good static analysis include TypeScript and Elm. TypeScript allows you to define the type contracts your functions and methods will adhere to, which means you get some good information up front about the safety of a function call. Elm is a functional view-layer language with very strict code and type rules; because of this Elm provides greater code stability guarantees.

3. Functional Programming

The kind of functional programming I am talking about is not just introducing Underscore or Lodash into your project, but really embracing the ideas that make functional programming great: immutability, no side effects, function composition and function abstraction.

Even though Javascript variables are mutable, by treating them as immutable, you add stability to your code by avoiding changing state under your own feet. When a value is set and remains as it was assigned, you can rest assured that your code will always behave the same way when you refer to that variable.

By eliminating side effects in as much of your code as you can, you create well defined units of code which behave the same way every time and, better yet, are easy to test. This makes the first item on our list even easier to satisfy which makes your program better.

Function composition is the process of creating small abstracted units of code as functions without side effects which are then put together to do more complex work. This kind of modularity and abstraction makes it much easier to test and, when necessary, debug your code.

4. Data Structures and Algorithms

In any computer science program, the data structures and algorithms course is considered a critical course in computer science thinking. By getting familiar with the well known data structures and algorithms, you lay a foundation upon which you can build your knowledge and skills which will help to more quickly analyze and work with business concerns as well as existing code which will start to display recognizable patterns.

Data structures reach beyond the basics of variables, arrays and objects and dive into the concept of lists, stacks, queues and trees. These complex structures provide much cleaner, smarter solutions to common problems and can provide insight into problems which might be hard to identify without this kind of core understanding.

In much the same way that data structures provide data-related solutions to problems, algorithms provide insight into code and how to build in efficiency and good structure. Topics like sorting, searching and working with complex data structures will give a functioning foundation for how to integrate data solutions into your projects.

5. Design Patterns

Finally, to really cap the knowledge you have gained from the rest of the list, design patterns are a set of solutions which have been discovered and well documented. Design patterns tie together all of the practices with testing, abstraction, data solutions and algorithms and provide a well known structure to add to your program as common problems and patterns emerge.

As evidenced by my list, design patterns are not where you begin, but where you enhance. Good use of design patterns will enhance well-architected code and provide clarity when the going gets rough. These patterns are not so much a law as a guideline to help make good programs better and bad programs stable.


There are many other important practices I could have thrown into this list, like polygot programming and theoretical studies, but these things are only useful once a strong foundation is set. In studying these five major topics it is highly likely you will encounter different programming languages and different schools of thought. This is a good thing.

The greater the number of perspectives a programmer is introduced to, the better they are bound to become. This list is not something a good developer can go through once and be done. Instead it is a cycle which should be recognized and embraced. By learning, developers grow and by growing, developers improve their world.

Hopefully the topics I presented here resonate with you and you share this with others who want to continue their journey to be their best. Even the journey of a thousand miles is started with a single step, so take your step and become better every day.

Mainstay Monday: Queues

A couple weeks ago, we looked into linked lists. Sadly linked lists are kind of a standalone topic that don’t use much more than basic objects in order to function as designed. Queues, on the other hand can easily spring forward from linked lists, and they are a way of working with data as you might with generators!

Queues are a great resource for dealing with anything from data stores which are being updated in one place and read from another to dealing with data requests against an endpoint and a cache. Queues give you a reliable way to store data and interact with it later in the same order. Since functions are first class in Javascript, this means functions are actually data, which can be stored in a queue as well. This kind of power opens a world of possibilities.

Let’s take a look at a queue in action.

That’s a lot of code for a simple example, but I think you’ll get the idea. Essentially we want to call a service and get a value, resolve the value and store it in the queue. One that is complete, we want to log everything that was queued up. The problem we would have run into is, the queue may not be filled completely before we start dequeueing. This allows us start with the first value and let the rest filter in over time.

Queues are a common data structure which are used throughout programs in every language to solve the same kinds of problem: we want to perform one action and allow other actions or data to wait until we are ready. It’s a lot like people waiting to get into a movie on opening night.

Since I already talked about linked lists, you probably have an idea where I am going with all of this. Let’s use the list object we created in the last post to build our queue. It turns out that once we have a linked list item definition, we can create a queue with just a bit of effort.

There is a little bit of logic here to ensure we don’t leave dangling pointers and we don’t have null pointer exceptions, but other than this, we’re basically moving through a list that has the capability to grow on one side and shrink on the other. This is the perfect structure for dealing with data which isn’t infinite, but could grow to an arbitrary size at an arbitrary rate.

Why not just use an array for this?

It turns out we can do this in a quick and dirty way with an array. I’ve done it and you can do it too. I’ll even give an example of how to do it, but I would argue that you shouldn’t. Arrays are really, really good at handling random reads from a location. This means, if you have data you want to bounce around in, arrays are a good way to manage that. While you get this random access behavior, you have to pay for it somewhere. That cost is built in to the allocation and management of array memory.

Let’s take a look at a queue built on an array and then we can talk about the problems.

As you can see, this code is really short and easy to write. For a small queue written in a naive way, this will suffice, but there is something dangerous lurking just beneath the surface. When we create our array, it allocates space for the array to live in. This space will grow with our array at a linear rate, which is okay, though non-optimal. The real problem comes in when we perform a shift.

Shifting an array involves retrieving the value from the head of the array, and then moving each of the elements into a new position in the array to fill the head space which was shifted out of the array. This kind of element movement and array space reallocation is really, really slow.

This slowness comes from the fact that an array in Javascript has to abide by particular rules to be predictable. If we were to do the following and the elements weren’t moved as described here’s what would happen:

This kind of reallocation is exactly what we avoid by using a linked list. Linked lists grow and shrink in constant time and position 0 is always the head of the list. Since queues only ever interact with the first and last elements of a set of values, lists give us the improved performance we need to ensure, even with large queues, we don’t encounter the kinds of difficult to diagnose bottlenecks in our code that can cause slowness.

Queues are a great example of a use for linked lists in the wild and they provide a useful mechanism for lining up data and handling it in a predictable order. With a little creativity, queues can provide a means to manage a cache, handle processes in an orderly way and even provide the backbone for generator-like behavior. Take our queue code and play with it. You might find a solution to a problem that has been challenging you.

Code Generation and You

A friend of mine and I have discussed code generation on several occasions. We both agree that any enterprise development process should involve code generation, without exception. Though it is possible for development from scratch may not provide enough identifiable boilerplate to justify building templates for your code, the moment a framework is in use, you will, undoubtedly, have a certain amount of boilerplate introduced which must be set up repeatedly for each new file you create.

I work with a team using Angular. We have certain style requirements for code, and Angular has a particular way for handling the setup for every code component you would create. Beyond this, we have certain patterns which have been uncovered which introduces sets of modules which we need to construct, which introduces even more boilerplate.

I have timed the setup for a standard controller and the baseline unit tests which need to be created before any new development and on average it takes about 10 minutes to create all of the files and type all of the boilerplate. For a one-time cost, this is a minimal requirement, but when working in an enterprise environment, this can add up to many lost hours each month.

Here’s the worst part of losing those hours: no problem was solved in that time.

This means for any given person who is creating a new module, they are performing rote behavior over and over without adding any real value to the project. Programmers are not paid to simply pound keys. If key pounding was all that was necessary to create an application, they would pay the smallest wage possible. Data entry is a good example of a key-pounding job. I think we can all agree that programming is more than simply performing data entry.

If this kind of rote file creation and boilerplate typing is the most basic of work, then why do we continue to do it? It seems as though rote behavior is something we often look to automate, which is why the profession exists in the first place.

I say, are we not programmers?

Automation is the name of our game, so it seems reasonable that we would want to take this wasted time and have the computer do it for us. This is what computers are for. Fortunately, this is a solved problem!

My team uses Yeoman to create our boilerplate. We have a defined application file structure, and the modules we create always have a certain amount of boilerplate associated with them. Yeoman is really, really good and taking templates and creating files with them. Here’s an example of what an Angular controller (in ES6) looks like, with no interesting bits:

That is about 20 lines of boilerplate, and we haven’t even created the unit tests for it, yet! Beyond that, most controllers are associated with a view or a directive, which means we have at least two or three more files, full of boilerplate, left to create. If we were using ES5, there would be even more code here that produced nothing of new value. This is not the reason we got into programming.

Let’s take a look at what a Yeoman template would look like instead.

This is so similar to the code we wrote above it’s difficult to see where one leaves off and the other picks up. Here’s the big win, though, We spent 10 or 15 minutes creating this template and now we never have to do it again!

Let’s have a look at what the actual script looks like to populate this controller:

That’s it. This might not be the fanciest Yeoman generator, but it reliably outputs a controller, with the correct parameters filled, every. single. time. With just a few more lines of code and another couple templates for our tests and views, we can type one line into the shell and get all of our working files spun up in seconds.

Let’s do a rough, back of the envelope comparison so we can see the amount of time saved by using generators. Let’s say you have a small to medium-sized SPA you are building and it contains 150 Javascript source files. On top of that, you will have unit test files for each of those source files, so that’s another 150 files. Atop all of that, you have views that associate with, let’s say, 1/3 of those files, so that’s another 50 files. Let’s say, for argument’s sake that it takes 3 minutes per file to generate these files by hand.

350 * 3 / 60 = 1050 / 60 = 17.5 hours

Now, let’s assume you created each of these files with a generator, and let’s assume your computer is slow, so it takes 1.5 seconds per file. Let’s do a little more math so we can see how long this takes in comparison:

350 * 1.5 / 60 = 525 / 60 = 8.75 minutes

Let’s take the value that hired says is average pay for a Javascript developer in Los Angeles, $130,000US/year, and divide it by the number of hours in the average work year, 2087. This means we have an hourly rate of about $62. Now, let’s multiply that $62 by 17.5 and we get $1085. That’s some expensive boilerplate!

With our same numbers, a developer is working for a little more than $1/minute to generate boilerplate. Let’s assume this same developer generated all of their code with templates instead of by hand. Now our code cost around $10 to generate on a slow computer. That is TWO ORDERS OF MAGNITUDE different.

People like to talk about exponential growth and this is exactly what we have here. Using code generation versus writing boilerplate by hand literally increases the cost of each base file exponentially by a factor of 2. Hand-typed boilerplate is 100 times as expensive.

The short version of all of this is, if you aren’t using code generation, you should be. If your manager tells you that it takes too much time to get code generation integrated into your project, just send them to me. Does it take too much time to decrease the cost and time of production by a factor of 100? I think not.

Similar posts in Coding, Javascript, Unit Testing

Mathematicode a Love Letter to Math

I’ll open up by saying I am a mathematician by education. I’m not one of the mathematical great minds and I am unlikely to ever win a Fields medal, but I have a degree in applied math and it’s what I know. The reason I share this is because it helps define the way I think about problems I work on. Regardless of whether I am working on a problem at work or working on something in my spare time, my math background helps to define the approach I take to solving many of the problems I encounter.

I have worked in the software world for a while now and I have encountered a good number of developers who come from a variety of backgrounds. I won’t say all the best programmers I have met are math people, but I won’t say they aren’t. Some people come to programming through an organic sort of process and others go through a formal education process. Either way, there is something that comes forward really fast, Boolean logic.

Boolean logic is very closely tied to the mathematical work that George Boole did in the 19th century. This means, regardless of anything else you do with programming, you are doing math. Always.

I believe that even the people who started off as designers or biologists or cognitive scientists, if they become good programmers, they end up being good at logic and, yes, math.

There is a common debate around what a programmer is. Are they scientists? Plumbers? Engineers? Developers? What are they?

I believe that every software engineer is a developer. I do not believe that every developer is a software engineer. By this, I can say, I believe that every programmer is a developer. An author of code. Programmers are writers.

As is true with journalists and novelists, there are good code authors and bad ones. Some people take pride in writing their code in a crafted, carefully maintained sort of way. I think this is a good thing and more people should do it.

The difference between a good and a bad author of code is the math they do. If we look to other engineers in fields like mechanics or structure, you will see math and science. Computer science incorporates these same elements in different ways. This means, in order to truly author great code, you must embrace the applications of math and make it part of your programming experience.

I believe that in order to be elevated to the state of being an engineer, you must look beyond the simple trappings of the language you work in. I speak the English language, but this does not mean that I am a linguist. I am merely a user of the language. Many developers live in much the same state in their code. Become more than you are and explore the things that make your language work.

I am not prepared to call myself an engineer, by the way. I believe I would make a terrible engineer. I am prone to flights of fancy and I am much more interested in the reasons why than the application of the math I understand. I bury myself in books about math and the underpinnings of programming rather than looking for sturdy materials to construct things with. I, with all of my foibles, am more aligned with philosophers than I am with the engineers who apply their knowledge to make the world great. I am okay with this realization.

In the end, however, I would encourage anyone who calls themselves a developer to dig and pry and tear whole chunks from computing. Find the math that lives underneath it all and apply it to what you know. As you grow and become a greater developer, embrace the math that will make you an engineer and bring sanity and soundness to the code that you and others like you will have to support and maintain for years to come.

Comments Off on Mathematicode a Love Letter to Math

Testing Promises: Promise Fakes

Javascript developers notoriously say unit testing in hard. I think the problem is actually more specific than that. Unit testing pure functions and business logic actually pretty easy.You put a value in, you get a value out. If the value you get back is what you expected, your test passes. Something that is hard to unit test is asynchronous logic.

Asynchronous logic is so hard to test, the angular team actually built a special mocking system for testing calls made through the $http service. In reality, though, if your unit tests are littered with $httpBackend references, you’re doing it wrong.

I won’t go into philosophical discussions about promises and callbacks, so let’s just agree that people use promises. A lot. Promises have become the currency for modern asynchronous requests, whether to another thread or across the internet, if you have to wait for your execution to come back from an asynchronous behavior, you might see a promise in the mix. Your code might look a little like this:

Let’s cut to the chase, promises are easy to throw into the middle of your code, but they are really hard to test… until today.

In unit testing there is a concept known as fakes. Fakes are often, mistakenly called mocks, but mocks actually store something to use for later when you are handing your test expectations. No, fakes are just what they sound like: a fake something.

Fakes contain the minimum amount of code to satisfy your functionality and nothing more. Sometimes fakes are nothing more than an empty object or function. Other times they are a little more involved, doing stuff like calling passed functions and returning values, but at the end of the day, a fake is a placeholder used for guaranteeing your unit under test won’t fail and will be isolated.

So, promises and fakes.

It is a law of unit testing that you do not talk to the outside world and you do not talk about fight club. This means, if you have a function that is going to call a service which will, in turn, interact with the world at large, you have a big problem. Fortunately, you probably wrapped that up in a promise and that is the crack in the armor we can use to break our unit out into its own isolated space.

Underneath it all, promises are nothing more than objects with a bunch of trickery hidden inside. With the knowledge of fakes and a general understanding of the promise syntax, we can build a stand-in object. A stunt promise, if you will. It’s pretty common to use Chris Kowal’s concept of promises, as developed in Q, so let’s build our fake around that.

Here’s what a promise fake might look like:

It’s about 40 lines of code, but now we have something we can work with. This promise fake can be instantiated for each test we write and it won’t muddy the state from test to test. It’s chainable, so if there is code using chained promises, it can be tested. Finally, success and failure states can be set with errors and response values so any function that might be relying on a specific response value will be testable with little extra effort.

Faking can be hard work, but if you do it right, you only ever have to do it once. Hooray! Now let’s look at a test for our method.

That was easy! We didn’t have to do a whole mess of monkey patching to our code and we didn’t have to use some crazy mechanism to intercept HTTP requests way down the stack. Fakes are great, when used for the powers of good. The goodness doesn’t stop there, though. Here’s how we can actually test our promise fake actually works as expected.

That’s pretty much it!

We had to do a little grunt work at the beginning, but after we built our class, we could fake promises all day long and save ourselves headaches forever more. The best part is, now we have eliminated the asynchronous behavior from our code and made everything testable. This makes our tests easier to read, easier to maintain and clearer to write. Who could argue with that?

What this all really leads us to is this, promises are tough to test, but when we absolutely, positively have to have them, we can trim the fat, clean out the code and fake our way to a brighter tomorrow. Isn’t that what we all really want? Go write some tests and make your code a better place to live.

Mainstay Monday: Linked Lists

Dear data structures, which of you is most useful? I don’t know, but linked lists are pretty awesome. Linked lists are great for any number of things, a list of data that can be grown without bound, a data structure that can be incrementally increased, a queue or a stack. There are even more things that can be done with linked lists, but before we can dig into what can be done with a linked list, we need to understand what it even is.

The most basic linked list is a series of objects that point from one to another. This is what we are going to dig into today. To get a basic understanding of how a linked list works, let’s have a look at a basic diagram of a linked list.

|Object A| -> |Object B| -> |Object C| -> null

By this diagram, it makes sense to say that an empty list is null. It’s not like null or falsey, but it’s actually null. Null contains nothing and points to nothing. As soon as you put your first element into the list, you get this:

=> |Object A| -> null

The last object in our linked list, always, is null. This actually makes it really easy to identify when we have hit the end of the list. This kind of list is called a singly-linked list. This means we can only traverse the list in one direction. Typically when a linked list is created, the head of the list is held as a pointer. Each object, including the head is an object and they are all, essentially, the same. Let’s have a look at an implementation of a basic linked list item.

There’s really not much to this, but the important items here are the constructor which sets the internal value, which I am treating as read-only for our case. Let’s just say it’s better that way. After that, we need an accessor, so let’s use val. Finally we need to be able to set the next value in the list and retrieve the next value in the list; append and next will do just fine for this. Now if we want to create and use our list it would look like the following:

This is a pretty manual process, but it gets the job done and we can see the basic use of our linked list object. Now we can see how each of the objects links to the next and the last object always refers to null. This gives us a nice, predictable structure and clear, obvious algorithm for accessing each element in the list.

Let’s add some convenience functionality around our list so we can dig into some of the interesting characteristics of a linked list. We can create a list object and an iteration object. These will give us a nice API to work with.

Here’s what our original usage looks like once we wrap everything up:

This is much cleaner and it gives us a little insight into the first aspect of linked lists, access performance. If we want to access the first or last element of our list, it happens in constant time, which we can express in big-o notation as O(1). This is really fast. It’s just about as fast as you can get for value access.

On the other hand, we also lose something for this gain at the front and back of the list. Accessing any of the elements in the middle can only be accessed in linear, or O(n), time. This means, to reach the nth element, you have to cycle through each element before it. For small lists this is not a problem, but for large lists, this can be a problem.

These performance characteristics make linked lists great for small data sets and things like stacks and queues. Linked lists, however, are not suited for random access or repetitive search. Sorting’s not great either, but that’s another discussion for another day. Let’s look at accessing elements.

Modification characteristics, on the other hand, are fantastic. If you need to add elements to a list, it’s fast. The addition action is nearly as fast as reading the head or tail of the list. If you have the list element you want to insert after, you get an O(n) insertion speed. The most costly part is the instantiation of a ListItem object. Each time you call append, it just adds an element and you’re done. Speedy!

At the end of the day, there is another kind of list: the doubly-linked list. As it turns out the performance characteristics aren’t terribly better. You get the benefit of moving up and down through the list, but access is about the same speed as is appending.

Linked lists, by themselves, are useful for the purpose of fast writing and small memory footprint for storage expansion. It also doesn’t require any pre-allocation, and can grow incrementally. When we look at other data structures, linked lists can make a good foundation structure because of the fast write behavior. There is also a removal characteristic that is equally fast. We’ll take a look at those structures in another post. Until then, think about your data and how you need to use it. Perhaps a list is what you’ve needed all along.