Jesse Warden
15 Jun 2021
β’
13 min read
Writing Functional Programming in Node is one challenge, but unit testing it is another. Mainly because many middlewares in Node use the connect middleware approach, and libraries in Node are not written in a pure function way.
This six part series will go over how to make the unit testing part of easier, some strategies to tackle common impurity problems, and hopefully enable to make 100% test coverge a common part of your job vs. the "not worth the client investment" people commonly associate with it.
Introduction
Contents
Before We Begin
Refactor Existing Route
Export Something for Basic Test Setup
Server Control
Input Validation
Asynchronous Functions
Factory Errors
Mutating Arrays & Point Free
Functional Code Calling Non-Functional Code
Class Wrangling
Compose in Dem Rows
Define Your Dependencies
Currying Options
Start The Monad Train... Not Yet
Compose Again
Coverage Report
The Final Battle?
Class Composition is Hard, Get You Some Integration Tests
Next is Next
Mopping Up
Should You Unit Test Your Logger!?
Conclusions
Code & Help
Feel free to skip these. I'll cover each in the refactoring and unit testing of our Node example and will cite which rule I'm covering so you have context.
The pureness level associated with Functional Programming is maleable, ecspecially considering JavaScript is not a primarely functional language, and many libraries are created & contributed to by a wide variety of developers with varying opinions on how pure is pure enough and coveniant for them. So let's define what we consider "pure enough".
Pure functions are same input, same output, no side effects. Not all Node code is like this, nor are the libraries. Sadly, this means the onus is on you to do the work and judge when it's pure enough. If you don't know if it's pure enough, ask yourself, "Do I need mocks?" If you can't use stubs only, it's not pure enough.
Don't use var
or let
. Only use const
. If this is too hard, use let
, and ensure the let is only used inside the function.
The function
keyword retains scope. Scope is not pure and causes all kinds of side effects. Instead, use arrow functions. While they technically adopt whatever scope they are defined in, we are NOT creating, nor using scope. Avoid using the this
keyword at all costs.
While newer versions of Node now natively support the class
keyword, as stated above, avoid scope at all costs. Do not activately create classes.
While there are tricks, we'll assume even your Node logger has to be as pure as Haskell is about including logging as as a side effect. Many believe this is taking things too far.
This article won't focus on types. They are useful in solving a ton of errors, but not at runtime. For now, see suggestions below in "Proper Function Naming of Impurity". This article assumes you're not creating total functions; pure functions that can handle any type. const add(a, b) => a + b
is a pure function, but is not a total function because although you can call add({}, new Error('wat')
, the result of '[object Object]Error: wat' isn't really what we'd expect from an addition function. Types help solve that even beyond using total functions.
Creating unsafe functions and noops (no operation) functions that have no return values is fine as long as you label them as such. If you have a function that calls an Express/Restify/Hapi next
function and that's it, either return a meaningful value, else leable the function as a noop suffix or prefix (i.e. sendResponseAndCallNext
or sendResponseCallNextNoop
).
If you using a library like Lodash or Ramda, and not using a transpiled language like TypeScript/PureScript/Flow/Reason, then you probably don't care about types. While I don't like current transpiler speeds for using types, I DO like runtime enforcement. My current tactic has been to use Folktale validators on public functions (functions exposed through modules) to ensure the parameters are of the proper type. Sanctuary adds that for you over top a Ramda like interface. The issue I have with it is that it throws exceptions vs. returning validation errors.
Either way, for functions that may fail from types, just label it with an unsafe suffix.
const config = require('config')
// config.get will throw if key doesn't exist
const getSecretKeyUnsafe = config => config.get('keys.gateway.secretKey')
For functions that may throw an exception, either use Result.try, simply wrap them with a try/catch and return a Folktale Result
, a normal JavaScript Promise
, or even just an Object like Go and Elixir do. Conversely, if it's a 3rd party library/function you are wrapping, change it to a suffix of safe to help differentiate.
// return an Object
const getSecretKeySafe = config => {
try {
const value = config.get('keys.gateway.secretKey')
return {ok: true, value}
} catch (error) {
return {ok: false, error}
}
}
const {ok, value, error} = getSecretKeySafe(config)
if(ok) {
// use value
} else {
// log/react to error
}
// return a Promise
const getSecretKeySafe = config => {
try {
const value = config.get('keys.gateway.secretKey')
return Promise.resolve(value)
} catch (error) {
return Promise.reject(error)
}
}
// return a Folktale Result
const getSecretKeySafe = config => {
try {
const value = config.get('keys.gateway.secretKey')
return Result.Ok({value})
} catch (error) {
return Result.Error(error)
}
}
Exceptions are impure, and violate function purity. Instead of same input, same output, you have no output because it exploded. Worse, if you compose it with other pure functions, it can affect their purity by making them all impure because you put a grenade in it. Very unpredictable and worse in server scenarios.
Don't throw Error
s. Endeavor to either return Maybe
's for possibly missing values, Result
's or Either
's for errors, or even a Promise
if you're just starting out. If you can, endeavor to have promises not have a catch as this implies you know about an error. If you know about it and what can go wrong, instead return a Promise.resolve
in the .catch
with a Maybe
, Result
, or Validation
to indicate what actually went wrong. Avoid creating null pointers intentionally.
If you're using Sanctuary, don't use try/catch, and assume those errors be will sussed out in property and integration/functional tests (Sanctuary will throw on invalid types unless you turn it off).
Given we have no types, Node frameworks ecspecially adds things onto request objects dynamically, and compounded with the fact we're dealing with a lot of dynamic data. Accessing a non-existent property is ok, but accessing a property on something that doesn't exist results in a null pointer error.
const cow = { name: 'Dat Cow' }
console.log(cow.chicken) // undefined, but ok to do
console.log(undefined.chicken) // throws an Error
While languages like Swift and Dart have null aware access, those are operators, not pure functions. Those are fine and encouraged to use. Unless your compiler or transpiler has support for infix operators, you should stick with pure functions, unless those operators are used within pure functions. Lodash has support for get
and getOr
. That said, in certain predicates where you know it's of a specific type, it's ok to use dot access. For example, if I know something is an Array, I'll access it directly like theArray.length
vs. get('length', theArray)
. Just be aware of the risk.
No mocks allowed in your unit tests. Stubs are fine and encouraged. Martin Fowler convers their differences which are way more pronounced in Java examples. If you can't because it's a third party library that has an API that's too hard to unravel, or you're on a time crunch and the existing API is too challenging to refactor, then this is exactly the nitch that sinon fills. As Eric Elliot says, Mocks are a Code Smell. Endeavor to make your functions pure so you only need stubs, and don't have to mock anything.
If your unit tests work, then you turn your wireless off, and they fail, those are not unit tests, those are integration tests, or bad unit tests, or both. We'll mostly write unit tests, and in Part 5 show you how to use Mountebank for better integration tests. Supertest is fine too.
Once you go always-curry, there's no going back. There are basically 3 strategies for currying functions in JavaScript, some intermingle.
curry
keyword in Lodash/Ramda/Sanctuary.curryN
.If you're using# 3, or Sanctuary, then all functions only take 1 argument, so you can't call a curied function like doSomething(a, b, c)
whereas in examples# 1 and# 2 that would work fine. If you're using# 3 or Sanctuary, it must be written as doSomething(a)(b)(c)
.
This also means you shouldn't be using default values for function parameters as that doesn't really jive with curried functions.
Danger: Please note that Express and other functions will check arity at runtime. Ramda retains arity via function.length
, while Lodash reports 0, and Sanctuary reports 1. Only Ramda curried functions will work in Express's error handling for example.
Whatever you use, ensure all functions that take more than 1 argument are curried by default.
Favor destructuring. Instead of creating Object copies manually which you may accidentally mutate something, favor Object.assign
for Objects out of your control (so it calls getter/setters if need be) and Object destructuring for the ones you do. For Arrays, favor destructuring and using immutable Array methods vs. mutatble ones like .push
.
The use of curly braces {}
in functions implies you're doing imperative code by defining a function block. This is usually a sign your function can be refactored to something more composable/chainable. Using them in Object definitions, functions that return only an Object, or matchWith
syntax that defines function callbacks is fine.
Understand if you get higher than 100% test coverage, you'll still have bugs. That's ok.
This article will keep it for the sake of showing you how to pragmatically incorporate good practices into existing code bases that may be too big to refactor, or may have dependencies that are out of your control. That said, it's built around the noop next
function, which is a noop (more about this in Part 5) and full of side effects. Better to use Promise chains at a minimum. This is part of the Express/Resitify/Hapi ecosystem so they're okay to use, just don't create your own middlewares.
We're going to refactor an existing route that is used for uploading files that are virus scanned and then emailed. We'll write it in the typical Node imperative way, and slowly refactor each part to pure functions, and test each one to get 100% unit test coverage or more.
The function is an Express middleware (a function that takes 1 or 2 arguments, req
, res
, and/or next
) that takes files uploaded by the user and emails them. It does a good job of sending validating the files, and sending back errors with context of what went wrong. There is actually nothing wrong with this code and it works. This is not an exercise to say imperative or OOP code is bad, rather to see how to refactor from one to the other.
function sendEmail(req, res, next) {
const files = req.files
if (!Array.isArray(files) || !files.length) {
return next()
}
userModule.getUserEmail(req.cookie.sessionID).then(value => {
fs.readFile('./templates/email.html', 'utf-8', (err, template) => {
if (err) {
console.log(err)
err.message = 'Cannot read email template'
err.httpStatusCode = 500
return next(err)
}
let attachments = []
files.map(file => {
if (file.scan === 'clean') {
attachments.push({ filename: file.originalname, path: file.path })
}
})
value.attachments = attachments
req.attachments = attachments
let emailBody = Mustache.render(template, value)
let emailService = config.get('emailService')
const transporter = nodemailer.createTransport({
host: emailService.host,
port: emailService.port,
secure: false,
})
const mailOptions = {
from: emailService.from,
to: emailService.to,
subject: emailService.subject,
html: emailBody,
attachments: attachments,
}
transporter.sendMail(mailOptions, (err, info) => {
if (err) {
err.message = 'Email service unavailable'
err.httpStatusCode = 500
return next(err)
} else {
return next()
}
})
})
}, reason => {
return next(reason)
})
}
You can't unit test a module functionally unless it exports something for you to test. Typical Hello World examples of Express/Restify/Hapi only show the server importing things and using those libraries, not actually testing the server.js / app.js itself. Let's start that now as this'll be a pattern we'll continue to build upon.
Open up your server.js, and let's add some code, doesn't matter where.
const howFly = () => 'sooooo fly'
Now let's export that function at the very bottom:
module.exports = {
howFly
}
We'll be using Mocha as our test runner and Chai as our assertion library. Let's create our first unit test file in a test
folder, using expect
keyword. I like should
but I appear to be in the minority:
const { expect } = require('chai')
const { howFly } = require('../src/server')
describe('src/server.js', ()=> {
describe('howFly when called', ()=> {
it('should return how fly', ()=> {
expect(howFly()).to.equal('sooooo fly')
})
})
})
If you don't have a package.json
, run npm init -y
. If you haven't installed test stuff, run npm i mocha chai istanbul --save-dev
.
Open up package.json
, and let's add 3 scripts to help you out.
...
"scripts": {
"test": "mocha './test/**/*.test.js'",
"coverage": "istanbul cover _mocha './test/**/*.test.js'",
...
Now you can run npm test
and it'll show your new unit test.
Great, 1 passing test.
However, you may have noticed that the unit tests do not complete and you have to use Control + C to stop it. That's because as soon as you import anything from server.js
, it starts a server and keeps it running. Let's encapsulate our server into a function, yet still retain the ability to run it in the file via commandline.
All the examples to solve this problem, allowing the file to act differently if it's used like node server.js
vs. if you import a function from it like const { someFunction } = require('./server.js')
look something like this:
if (require.main === module) {
console.log('called directly');
} else {
console.log('required as a module');
}
Basically, if require.main
, then you used node server.js
, else you require
'd the module. If else are fine in functional programming, but imperative code floating in a file is not. Let's wrap with 2 functions.
First, are we being called commandline or not?
const mainIsModule = (module, main) => main === module
Note module
and main
are required as inputs; no globals or magical closure variables allowed here. If we didn't include module and main as arguments, the function would require us to mock those values before hand in the unit tests. Given they're run before the unit tests since they're part of how Node works, that's super hard and hurts your brain. If you just make 'em arguments, suddenly things get really easy to unit test, and the function gets more flexible.
Next up, start the server or not:
const startServerIfCommandline = (main, module, app, port) =>
mainIsModule(main, module)
? app.listen(3000, () => console.log('Example app listening on port 3000!'))
: ... uh, what do we return?
Great, but... what do we return? It turns out, app.listen
is not an noop (a function that returns undefined
), it actually returns a net.Server class instance.
Maybe we'll get a server instance back... maybe we won't. Let's return a Maybe
then. Install Folktale via npm install folktale
then import it up top:
const Maybe = require('folktale/maybe')
const { Just, Nothing } = Maybe
Normally we could do that in 1 line, but let's keep Maybe
around for now. We'll refactor our function to use Just
or Nothing
.
const startServerIfCommandline = (main, module, app, port) =>
mainIsModule(main, module)
? Just(app.listen(3000, () => console.log('Example app listening on port 3000!')))
: Nothing()
Finally, call it below module.exports
:
startServerIfCommandline(require.main, module, app, 3000)
Test it out via npm start
(this should map to "start": "node src/server.js"
in your package.json. You should see your server start.
Now, re-run your unit tests, and they should immediately stop after the test(s) are successful/failed.
Let's unit test those 2 functions. Ensure you export out the main function, and we'll just end up testing the other one through the public interface:
module.exports = {
howFly,
startServerIfCommandline
}
Import into your test file and let's test that it'll give us the net.Server instance if we tell the function we're running via commandline:
...
const { howFly, startServerIfCommandline } = require('../src/server')
describe('src/server.js', ()=> {
...
describe('startServerIfCommandline when called', ()=> {
const mainStub = {}
const appStub = { listen: () => 'net.Server' }
it('should return a net.Server if commandline', ()=> {
const result = startServerIfCommandline(mainStub, mainStub, appStub, 3000).getOrElse('# x1f42e;')
expect(result).to.equal('net.Server')
})
})
})
Great, now let's ensure we get nothing back if we're importing the module:
it('should return nothing if requiring', ()=> {
const result = startServerIfCommandline(mainStub, {}, appStub, 3000).getOrElse('# x1f42e;')
expect(result).to.not.equal('net.Server')
})
Ballin'. Our server in much better shape to test, yet still continues to run if started normally. In the next part, we'll focus on validation, error handling, and calling non-FP code from FP code.
If you liked this blog, click the ππ over on the right to let me know!
Ground Floor, Verse Building, 18 Brunswick Place, London, N1 6DZ
108 E 16th Street, New York, NY 10003
Join over 111,000 others and get access to exclusive content, job opportunities and more!