Expressively
Custom
Expectations
in Chai.js

November 2017

Titus Stone

Ever had This Happen?

  • You implement some new code
  • Run the tests
  • And some unrelated test fails with a meaningless error
1) UserService #get returns a user:
     AssertionError: expected false to be true

How About This?

  • You add 1 new property to a model
  • Run the tests
  • 32 tests fail because it's missing your new property
  1 passing (1m 21s)
  32 failing

What Both of Scenarios Have in Common Is Less-Than-Great Expectations

Expressively Custom
Expectations in Chai.js

  1. Language included with Chai.js
  2. Matching expectations to behavior
  3. Defining custom expectations
  4. When to and not to write chai plugins

Language Included with Chai.js

Chai includes three syntaxes:


value.should.XXX
expect(value).XXX
assert.XXX(value)
					

The 'expect style' is primarily used at Ibotta

Expect

Expect is a function that takes a single argument,
the subject under test.


expect(value)
					

Expect

What is expected about the subject is a property OR (chainable) method on the return of the expect function.


expect(value).<property or method>
					

Expect

If you're coming from an RSpec or JUnit influecned background, properties and chainable methods have additional use cases compared to matchers and assertions


"property" and "chainable method" are Chai vocabulary

Chai Properties

A property can be an/the assertion


expect('bar').undefined // fails
					

"assertion property" (my term)

Assertion Property

An assertion property works because of getters in javascript, which run a function when a
property is accessed.

This can be imagined as*...


get undefined() {
  return typeof this._obj === 'undefined'
}
					
* = the actual implementation is much different. This is just for illustrative purposes.

Built-in Assertion Properties

ok, true, false, null, undefined, NaN, exist, empty, arguments, itself, sealed, frozen, finite

Chai Properties

A property can be purely cosmetic

to and be have no effect on the assertion but make it clearer for human readers of the code


expect('foo').to.be.undefined // fails
					

"cosmetic property" (my term)

Built-in Cosmetic Properties

to, be, been, is, that, which, and, has,
have, with, at, of, same, but, does

Chai Properties

Be aware that cosmetic properties are no-ops, thus used by themselves will always pass because they have no assertions (no reason to fail)


expect('foo').have.to.with.same.be; // passes
					

Chai Properties

Properties can set a flag that alters the behavior of assertions coming after it. not is a flagging property.


expect('foo').to.not.be.undefined // passes
					

"flagging property" (my term)

Built-in Flagging Properties

not, deep, nested, own, ordered, any, all

Chai Methods

A method is like an assertion property that uses one or more additional values when expected


expect('foo').equal('foo'); // passes
					

Built-in Methods

a, include, equal, eql, above, below, least, most, within, instanceof, property, ownPropertyDescriptor, lengthOf, match, string, keys, throw, respondTo, satisfy, closeTo, members, oneOf, change, increase, decrease, by, fail

Chai Chainability

Properties and methods can go after any
other property or method


				expect('foo').to.equal('foo').and.not.be.undefined; // passes
					

Chai Chainability

Flags apply to all subsequent properties and method,
so order may matter


expect('foo').to.not.be.undefined.and.equal('foo') // fails
					

fails because the not is applyed to equal

Chai Chainable Methods

Use a method like a method -OR- an assertion/flagging property. No examples as used in core language.


user acts as both a property and method:


expect(usr).to.be.a.user({ name: 'whatever' });
expect(usr).to.be.a.user.with.property('name', 'whatever');
					

Questions on the core language of properties and chainable methods Chai includes?

Matching expectations to behavior

Q: What should I expect on?

A: How do you know it's working?

Rules of Thumb

1. Always expect on the most specific thing you can

Why: By testing on the most specific thing, the error or problem identified by a failing test has a much higher likelyhood to be obvious to the human what is wrong

Rules of Thumb

2. Only expect on what is being tested and no more

Why: By not testing on things not under test, it reduces the likelyhood of many tests failing because 1 thing changed

An ideal -- When 1 thing changes, 1 test fails

How do you know the most specific thing to expect on?

All code can be divided
into two buckets:

Side effecting and non-side effecting

Side Effecting: Something about the world changes

Example: A property gets mutated on an object

Example: A metric is recorded

Non-Side Effecting: Nothing about the world changes; I get the result of a calcuation back

Example: Adding two numbers together

Example: Getting a record from a database

Side Effecting Expectations

  1. Do I know this is working because of the state of an object given to another object?
  2. Do I know this is working because of a value given to another object?
  3. Do I know this is working because of a method invoked on another object (ie. another side effect)?

Non-Side Effecting

  1. I know this is working because of the value that I get back


Caveat: For operations that depend on other services it's usually necessary to either a. abandon unit testing for integration testing, or b. to treat a non-side effecting function like a side effecting function in order to test one particlar path through the computation.

Example Behavior:
When the "active" argument is given on the read function, only return records that are flagged active


We could a. read from a test version of the database
(expect on the result)

-OR-

We could b. test for the existence of "WHERE ACTIVE = true" on the query, treating the function like a side effecting function but keeping it a unit test
(expect on the state of some object given to another object)

Expect on a Return Value


const result = subject.behavior();
expect(result).to.equal(whatever);
					

Expect on the State of an Object Given to Another Object


const dependency = { get: sandbox.spy() };
const subject = new MyService(dependency);
subject.behavior();

const query = dependency.get.getCall(0).args[0];
expect(query.property).to.equal(whatever);
					

sandbox.spy comes from Sinon

Expect on a Value Given to Another Object
(Introspetion Approach)


const dependency = { get: sandbox.spy() };
const subject = new MyService(dependency);
subject.behavior();

const query = dependency.get.getCall(0).args[0];
expect(query).to.equal(whatever);
					

Expect on a Value Given to Another Object
(withArgs Approach)


const dependency = { get: function(){} };
sandbox.mock(dependency)
  .expects('get')
  .withArgs(whatever);

subject.behavior();
					

this does not use Chai at all.
sandbox.mock comes from Sinon.

Questions on matching
expectations to behavior?

Sometimes the built in properties and methods aren't enough

  • Extracting a piece of state from a complex object to assert on it is long, multi-line effort
  • Certain behavior rules in the domain are commonly asserted on in many tests

In these cases a Chai plugin can be written to extend the included properties and methods

  • Do you need a property or a method?
  • Do you just need to check for the truthiness of something? (Yes: use an assertion property)
  • Do you need an additional value to know if something is correct? (Yes: method)
  • Do you need to toggle checking for certain things? (Yes: use a flagging property and method or assertion property)

Chai Plugin Skeleton


export default function(chai, utils) {
  const Assertion = chai.Assertion;

  // your plugin here
}
					

Using Your Plugin


// test-setup.js
import plugin from './myPlugin';
chai.use(plugin);
					

Custom Assertion Property

Definition


utils.addProperty(Assertion.prototype, 'example', function(){
  // make assertions here
});
					

In use


expect(value).to.be.a.example;
					

Custom Assertion Property


The value under test is available via this._obj


Assertions against that value can be performed with one of two approaches:

  1. this.assert -- Allows any boolean test with completely custom failure messages
  2. new Assertion(givenValue, customMessage) -- Allows composition with existing expectations

Custom Assertion Property Example (this.assert)


utils.addProperty(Assertion.prototype, 'example', function(){
  this.assert(
    this._obj === 'expected',
    'Expected #{this} to be "expected" but was not',
    'Expected #{this} to not be "expected" but was',
  );
});
					

On failure:


AssertionError: Expected value to be "expected" but was not
					

Custom Assertion Property Example (Assertion)


utils.addProperty(Assertion.prototype, 'example', function(){
  const msg = 'value was not "example"';
  new Assertion(this._obj, msg).to.eql('example');
});
					

On failure:


AssertionError: value was not "example": Expected value to equal "expected"
					

Custom Method

Definition


Assertion.addMethod('method', function(arg1) {
  // make assertions here.
  // arg1 is the value given during the expectation
});
					

In use


expect(value).to.method('arg1');
					

Custom Flagging Property

Set


utils.addProperty(Assertion.prototype, 'mcomm', function(){
  utils.flag(this, 'mcomm', true);  // set a flag
});
					

Get


utils.addProperty(Assertion.prototype, 'offer', function() {
  new Assertion(this._obj).is.an.instanceof(Offer);
  if (utils.flag(this, 'mcomm')) {
    this.assert(this._obj.mcommerce == true, /* positive msg */, /* negative msg */);
  }
});
					

Custom Flagging Property

In use


expect(offer).is.a.offer;
// vs.
expect(offer).is.a.mcomm.offer;
					

Questions writing chai plugins?

When to and not to
write custom plugins

aka Points of consideration before diving
head long into chai plugins

Custom Plugins Aren't Free

  • Team on-boarding/training
  • Ambiguity / Sizing
  • Time to implement
  • Future maintenance / backwards compatibilty

Team on-boarding/training

  • How will people know and remember they exist?
  • Are the plugins discoverable?
  • How do I know how to use them? (e.g. documentation)
  • Overall project design vs. just-in-time creation

Ambiguity / Sizing

  • Is the proprety or method the smallest it can be?
  • How do you know you got all the cases?

Time to implement

  • Do we write unit tests for chai plugins?
  • Do all chai plugins need to be (and incur the overhead of) an NPM module?

Future maintenance

  • How often will this need to change?
  • Will the chai plugins need to be re-written every time a refactor happens?
  • If it becomes an NPM module how are breaking changes handled?

Any final questions?

just wanted to point out this presentation was 100% meme free,
which is kind of an accomplishment