FP Unit Testing in Node - Part 2: Predicates, Async, and Unsafe

Predicates, Async, and Unsafe

In part 2, we'll validate our inputs using predicates, show how to make asynchronous pure functions, and show various techniques on how you call unsafe code from Functional Code.

Contents

This is a 6 part series on refactoring imperative code in Node to a functional programming style with unit tests. You are currently on Part 2.

• Part 1 - Ground Rules, Export, and Server Control

Quick History About Middleware

A middleware is the name for "a function that takes 2 or 3 arguments, namely req, res, or req, res, and next. In Express, Restify, and Hapi, req is a Request, and represents what the client sent to the server in a request (GET, POST, etc). That is where you inspect whatever form data or JSON or XML they sent to your API. The res is the Response, and typically what you use to respond back to the client via res.send. The next function is optional, and it's how the whole connect middleware concept works.

Before Promise chains were commonplace, there was no defined way to connect up a bunch of functions and have an escape hatch for errors. Promises do that now using 50 billion .then functions, and 1 .catch for errors that happen anywhere in the chain. Instead of calling .then or .catch like you do in Promises, instead, your function agrees to call next() when you're done, or next(error) when you have an error, and connect will handle error propagation.

File Validation

The first thing we have to refactor in sendEmail is the validation of the files array being on Request.

function sendEmail(req, res, next) {
	const files = req.files
	if (!Array.isArray(files) || !files.length) {
		return next()
	}
	...

First, let's add some predicates that are easier to compose (i.e. use together) and easier to unit test indepdently.

const legitFiles = files => Array.isArray(files) && files.length > 0

A predicate is a function that returns true or false. This as opposed to one that returns true, false, undefined, null, NaN, or ''... or even throws an Error. Making true predicates in JavaScript usually requires it to be a total function; a pure function that doesn't care what types you throw at it. Note we check if it's an Array first, and if it is, we can confidently access the .length property. Except, you can't. Remember, libraries will still override the Object.prototype of various built-in classes, so using get('length', files) would be a safer option here.

describe('legitFiles when called', ()=> {
    it('should work with an Array of 1', ()=> {
        expect(legitFiles(['cow'])).to.be.true
    })
    it('should fail with an Array of 0', ()=> {
        expect(legitFiles([])).to.be.false
    })
    it('should fail with popcorn', ()=> {
        expect(legitFiles('# x1f37f;')).to.be.false
    })
})

Note this function is a prime candidate for property testing using jsverify for example. Property tests throw 100 random values at your function vs. you creating those yourself. John Hughes who created the inspiration, Quickcheck, has a good YouTube video explaining the rationale.

Now that we can verify what a legit files Array looks like, let's ensure the request has them:

const validFilesOnRequest = req => legitFiles(get('files', req))

And to test, we just give either an Object with an files Array property, or anything else to make it fail:

describe('validFilesOnRequest when called', ()=> {
    it('should work with valid files on request', ()=> {
        expect(validFilesOnRequest({files: ['cow']})).to.be.true
    })
    it('should fail with empty files', ()=> {
        expect(validFilesOnRequest({files: []})).to.be.false
    })
    it('should fail with no files', ()=> {
        expect(validFilesOnRequest({})).to.be.false
    })
    it('should fail with piggy', ()=> {
        expect(validFilesOnRequest('# x1f437;')).to.be.false
    })
})

Ok, we're well on our way now to building up some quality functions to refactor the beast route.

Asynchronous Functions

With file validation behind us, let's tackle the part in the middle that assembles the email. A lot of imperative code in here requiring various mocks and stubs to ensure that part of the code is covered. Instead, we'll create pure functions for each part, test independently, then wire together later.

We'll hit the fs.readFile first. Callbacks are not pure functions; they are noops, functions that return undefined, but typically intetionally have side effects. Whether you use Node's built in promisify or wrap it yourself is up to you. We'll do it manually to show you how.

const readEmailTemplate = fs =>
  new Promise((success, failure) =>
    fs.readFile('./templates/email.html', 'utf-8', (err, template) =>
      err
      ? failure(err)
      : success(template)))

The only true impurity was fs being a global closure. Now, it's a required function parameter. Given this an asynchronous function, let's install chai-as-promised to give us some nice functions to test promises with via npm i chai-as-promised --save-dev.

Let's refactor the top of our unit test a bit to import the new test library:

const chai = require('chai')
const { expect } = chai
const chaiAsPromised = require('chai-as-promised')
chai.use(chaiAsPromised)

Now chai will have new assertion functions we can use to test async functions.

describe('readEmailTemplate when called', ()=> {
    const fsStub = {
        readFile: (path, encoding, callback) => callback(undefined, 'email')
    }
    const fsStubBad = {
        readFile: (path, encoding, callback) => callback(new Error('b00mz'))
    }
    it('should read an email template file with good stubs', ()=> {
        return readEmailTemplate(fsStub)
    })
    it('should read an email template called email', ()=> {
        return expect(readEmailTemplate(fsStub)).to.become('email')
    })
    it('should fail if fails to read', ()=> {
        return expect(readEmailTemplate(fsStubBad)).to.be.rejected
    })
})

How bangin'? Sttraiiiggghhttt bangin'. Note 2 simple stubs are required; one for an fs that successfully reads a file, and fs that fails. Note they aren't mocks because we don't care how they were used, what parameters were sent to them, how many times they were called, etc. We just do the bare minimum to get a test to pass.

Factory Errors

With the exception of Maybe, we'll avoid using union types for now, and instead stick with Promises to know if a function worked or not, regardless if it's async or sync. I encourage you to read Folktale's union type's documentation on your own time and perhaps watch my video on Folktale and skip to 22:55.

If the server fails to read the email template, we have a specific error for that so the client knows what happened. Let's create a factory function for that vs. class constructors and imperative code property setting.

describe('getCannotReadEmailTemplateError when called', ()=> {
    it('should give you an error message', ()=> {
        expect(getCannotReadEmailTemplateError().message).to.equal('Cannot read email template')
    })
})

Mutating Arrays & Point Free

The attachments code has a lot of mutation. It also makes the assumption at this point that the virus scan has already run and the files have a scan property. Mutation === bad. Assumption around order === imperative thinking === bad. Let's fix both. You're welcome to use Array's native map and filter methods, I'm just using Lodash's fp because they're curried by default.

First, we need to filter only the files that have been scanned by the virus scanner. It'll have a property on it called scan, and if the value does not equal lowercase 'clean', then we'll assume it's unsafe.

const filterCleanFiles = filter(
  file => get('scan', file) === 'clean'
)

You'll notice we didn't define a function here, we actually made one from calling filter. Lodash, Ramda, and other FP libraries are curried by default. They put the most commonly known ahead of time parameters to the left, and the dynamic ones to the right. If you don't provide all arguments, it'll return a partial application (not to be confused with partial function which I do all the time). It's also known as a "partially applied function". The filter function takes 2 arguments, I've only applied 1, so it'll return a function that has my arguments saved inside, and is simply waiting for the last parameter: the list to filter on.

You could write it as:

const filterCleanFiles = files => filter(
  file => get('scan', file) === 'clean',
  files
)

... but like Jesse Warden's mouth, it's too many, unneeded words. And to test:

describe('filterCleanFiles when called', ()=> {
    it('should filter only clean files', ()=> {
        const result = filterCleanFiles([
            {scan: 'clean'},
            {scan: 'unknown'},
            {scan: 'clean'}
        ])
        expect(result.length).to.equal(2)
    })
    it('should be empty if only whack files', () => {
        const result = filterCleanFiles([
            {},
            {},
            {}
        ])
        expect(result.length).to.equal(0)
    })
    it('should be empty no files', () => {
        const result = filterCleanFiles([])
        expect(result.length).to.equal(0)
    })
})

Note that the File object is quite large in terms of number of properties. However, we're just doing the bare minimum stubs to make the tests pass.

For map, however, we have a decision to make:

const mapFilesToAttachments = map(
  file => ({filename: get('originalname', file), path: get('path', file)})
)

If the files are either broken, or we mispelled something, we won't really know. We'll get undefined. Instead, we should provide some reasonable defaults to indicate what exactly failed. It isn't perfect, but is throwing our future selves or fellow developers a bone to help clue them in on where to look. So, we'll change to getOr instead of get to provide defaults:

const {
  get,
  getOr,
  filter,
  map
} = require('lodash/fp')

And the map:

const mapFilesToAttachments = map(
  file => ({
    filename: getOr('unknown originalname', 'originalname', file),
    path: get('unknown path', 'path', file)
  })
)

If point free functions (functions that don't mention their arguments, also called "pointless" lol) aren't comfortable for you, feel free to use instead:

const mapFilesToAttachments = files => map(
  file => ({
    filename: getOr('unknown originalname', 'originalname', file),
    path: get('unknown path', 'path', file)
  }),
	files
)

And the tests for both known, good values, and missing values:

describe('mapFilesToAttachments when called', ()=> {
    it('should work with good stubs for filename', ()=> {
        const result = mapFilesToAttachments([
            {originalname: 'cow'}
        ])
        expect(result[0].filename).to.equal('cow')
    })
    it('should work with good stubs for path', ()=> {
        const result = mapFilesToAttachments([
            {path: 'of the righteous man'}
        ])
        expect(result[0].path).to.equal('of the righteous man')
    })
    it('should have reasonable default filename', ()=> {
        const result = mapFilesToAttachments([{}])
        expect(result[0].filename).to.equal('unknown originalname')
    })
    it('should have reasonable default filename', ()=> {
        const result = mapFilesToAttachments([{}])
        expect(result[0].path).to.equal('unknown path')
    })
})

We're on a roll.

Functional Code Calling Non-Functional Code

As soon as you bring something impure into pure code, it's impure. Nowhere is that more common than in JavaScript where most of our code calls 3rd party libraries we install via npm, the package manager for JavaScript. JavaScript is not a functional language, and although a lot is written in FP style, most is not. Trust no one... including yourself.

Our email rendering uses an extremely popular templating engine called Mustache. You markup HTML with {{yourVariableGoesHere}}, and then call a function with the HTML template string, and your Object that has your variables, and poof, HTML with your data injected pops out. This was the basis for Backbone, and is similiar to how Angular and React work.

However, it can throw. This can negatively affect the rest of our functions, even if sequester it in a Promise chain to contain the blast radius. Or maybe it doesn't, it doesn't really matter. If you don't know the code, or you open up the source code in node_modules and do not see good error handling practices, just wrap with a try/catch, or Promise, and call it a day. We'll take more about Promises built in error handling below in "Extra Credit".

So, good ole' try/catch to the rescue.

const render = curry((renderFunction, template, value) => {
  try {
    const result = renderFunction(template, value)
    return Promise.resolve(result)
  } catch(error) {
    return Promise.reject(error)
  }
})

Clearly Defining Your Dependencies & Higher Order Functions

A few things going on here, so let's discuss each. Notice to make the function pure, we have to say where the render function is coming from. You can't just import Mustache up top and use it; that's a side effect or "outside thing that could effect" the function. Since JavaScript supports higher order functions (functions can be values, storied in variables, passed as function parameters, and returned from functions), we declare that first. Since everyone and their mom reading this code base knows at runtime in production code, that will be Mustache.render. For creating curried functions, you put the "most known/early thing first, dynamic/unknown things to the right".

For unit tests, though, we'll simply provide a stub, a function that just returns a string. We're not in the business of testing 3rd party libraries, and we don't want to have to mock it using Sinon which requires mutating 3rd party code before and after the tests, of which we didn't want to test anyway.

Creating Curried Functions

Note it's curried using the Lodash curry function. This means I can pass in Mustache.render as the first parameter, call it with the second parameter once the fs reads the email template string, and finally the 3rd parameter once we know the user's information to email from the async getUserEmail call. For unit tests, we supply stubs for all 3 without any requirement for 3rd party libraries/dependencies. It's implied you have to "figure out how they work" so you can properly stub them. This is where the lack of types forces you to go on the hunt into the source code.

Error Handling

Note the error handling via try catch and Promises. if we get a result, we can return it, else we return the Error. We're using a Promise to clearly indicate there are only 2 ways this function can go: it worked and here's your email template, or it didn't and here's why. Since it's a Promise, it has the side benefit of being easy to chain with other Promises. This ensures no matter what happens in the render function, whether our fault or it, the function will remain pure from 3rd party libraries causing explosions.

Note: This is not foolproof. Nor is using uncaughtException for global synchronous error handling, nor using unhandledrejection for global asynchronous error handling. Various stream API's and others in Node can cause runtime exceptions that are uncaught and can exit the Node process. Just try your best.

Extra Credit

You could also utilize the built-in exception handling that promises (both native and most libraries like Bluebird) have:

const render = curry((renderFunction, template, value) =>
  new Promise( success => success(renderFunction(template, value))))

But the intent isn't very clear form an imperative perspective. Meaning, "if it explodes in the middle of calling success, it'll call failure". So you could rewrite:

const render = curry((renderFunction, template, value) =>
  new Promise( success => {
    const result = renderFunction(template, value)
    success(result)
  })
)

:: frownie face :: "Your call, rookie."

has and get vs. get or boom

The config module in Node is an special case. If the config.get method fails to find the key in the various places it could be (config.json, environment variables, etc), then it'll throw. They recommend you use config.has first. Instead of using 2 functions, in a specific order, to compensate for 1 potentially failing, let's just instead return a Maybe because maybe our configs will be there, or they won't, and if they aren't, we'll just use default values.

const getEmailService = config =>
  config.has('emailService')
  ? Just(config.get('emailService'))
  : Nothing()

And the tests:

describe('getEmailService when called', ()=> {
    const configStub = { has: stubTrue, get: () => 'yup' }
    const configStubBad = { has: stubFalse }
    it('should work if config has defined value found', ()=> {
        expect(getEmailService(configStub).getOrElse('nope')).to.equal('yup')
    })
    it('should work if config has defined value found', ()=> {
        expect(getEmailService(configStubBad).getOrElse('nope')).to.equal('nope')
    })
})

Note that for our has stubs, we use stubTrue and stubFalse. Instead of writing () => true, you write stubTrue. Instead of writing () => false, you write stubFalse.