1st Sep 2019

Brittle selectors make brittle tests

A unit or integration test should never fail because of a brittle element selector.

Selectors that rely on an elements type, class, ID or position in the DOM compromise the quality of tests because the conditions of their success are highly likey to change.

In the following component we want to make an asserstion using the "Pay" button. The test should pass when the button text is "Pay" and should fail when the text is anything but "Pay":


const ShoppingBasket = () => (
  <form>
    <button type="submit">Pay</button>
  </form>
);

The button can be targeted using an element type selector:


it("Element based selector", () => {
  const wrapper = shallow(<ShoppingBasket />);
  const payButton = wrapper.find("button"); // ← element selector

  expect(payButton.text()).toBe("Pay");
});

// Produces:

 PASS  src/ShoppingBasket.spec.js
  ✅ Element based selector

A new feature requirement comes in from product. They want to test whether encouraging users to continue shopping will drive up sales.


const ShoppingBasket = () => (
  <form>
    <button type="button">Continue shopping</button> {/* ← new button */}
    <button type="submit">Pay</button>
  </form>
);

// Produces:

 FAIL  src/ShoppingBasket.spec.js
  ❌ Element based selector

  ● Element based selector

    Method “text” is meant to be run on 1 node. 2 found instead.

Our test fails because the element selector wrapper.find("button") yields both buttons, whereas we only want to interact with the second button.

We may be inclined to use a postional selector to target the "Pay" button based on it's position in the DOM:


it("Position based selector", () => {
  const wrapper = shallow(<ShoppingBasket />);
  const payButton = wrapper.find("button").at(1); // ← positional selector

  expect(payButton.text()).toBe("Pay");
});

// Produces:

 PASS  src/ShoppingBasket.spec.js
  ✅ Position based selector

The "Continue shopping" button has been live for a week. It hasn't yielded the returns product had hoped so they ask to move it from the left to the right hand side of the page. Perhaps a more prominent position will tempt users to continue shopping:


const ShoppingBasket = () => (
  <form>
    <button type="submit">Pay</button>
    <button type="button">Continue shopping</button> {/* ← new position */}
  </form>
);

// Produces:

 FAIL  src/ShoppingBasket.spec.js
  ❌ Position based selector

  ● Position based selector

    expect(received).toBe(expected) // Object.is equality

    Expected: "Pay"
    Received: "Continue shopping"

Once again our test fails because the positional selector .at(1) now points to the "Continue shopping" button.

A temptation is to add a class to the "Pay" button and use its class in the test:


const ShoppingBasket = () => (
  <form>
    <button type="submit" className="btn"> {/* ← added class */}
      Pay
    </button>
    <button type="button">Continue shopping</button>
  </form>
);

Now we can change the selector to use the btn class and our test passes again:


it("Class based selector", () => {
  const wrapper = shallow(<ShoppingBasket />);
  const payButton = wrapper.find(".btn"); // ← class selector

  expect(payButton.text()).toBe("Pay");
});

// Produces:

 PASS  src/ShoppingBasket.spec.js
  ✅ Class based selector

A few months have passed and a new technical requirement comes in from the platform team. They want us to start using the BEM naming convension.

The class btn doesn't provide much context as to where the button is located so we change it to ShoppingBasket__btn:


const ShoppingBasket = () => (
  <form>
    <button type="submit" className="ShoppingBasket__btn"> {/* ← edited class */}
      Pay
    </button>
    <button type="button">Continue shopping</button>
  </form>
);

// Produces:

 FAIL  src/ShoppingBasket.spec.js
  ❌ Class based selector (53ms)

  ● Class based selector

    Method “text” is meant to be run on 1 node. 0 found instead.

Yet again our test fails. This time it's because the selector expects the "Pay" button to have the class btn.

We may be tempted to update the selector in the test accordingly:


it("Class based selector", () => {
  const wrapper = shallow(<ShoppingBasket />);
  const payButton = wrapper.find(".ShoppingBasket__btn"); // ← update class?

  expect(payButton.text()).toBe("Pay");
});

However this is the third time we've had to change the selector and it's becoming cumbersome to maintain.

A more resiliant approach is to use a custom attribute reserved soley for finding elements in tests. The name of the attribute is up to you. Most commonly I have seen data-test and data-qa:


const ShoppingBasket = () => (
  <form>
    <button
      type="submit"
      className="ShoppingBasket__btn"
      data-test="pay-button" // ← custom attribute
    >
      Pay
    </button>
    <button type="button">Continue shopping</button>
  </form>
);

For the final time we can update our selector to use our new custom attribute:


it("Custom attribute based selector", () => {
  const wrapper = shallow(<ShoppingBasket />);
  const payButton = wrapper.find("[data-test='pay-button']"); // ← custom attribute selector

  expect(payButton.text()).toBe("Pay");
});

// Produces:

 PASS  src/ShoppingBasket.spec.js
  ✅ Custom attribute based selector

Now should the product team decide to add a thrid button and subsequently AB test its position, or the platform team decide we should migrate from BEM to a CSS-in-JS library the integrity of our tests remain intact.


import styled from "styled-components";

const Button = styled.button`
  border: none;
  font-size: 1rem;
  background: cadetblue;
`;

const ShoppingBasket = () => (
  <form>
    <button type="button">Pay later</button> {/* ← third button */}
    <Button type="submit" data-test="pay-button">
      {/* ↑ change CSS strategy from BEM to styled-components */}
      Pay
    </Button>
    <button type="button">Continue shopping</button>
  </form>
);

// Produces:

 PASS  src/ShoppingBasket.spec.js
  ✅ Custom attribute based selector

This greatly hardens the durability of our test. The data-test selector is protected from changes to the buttons styling, behaviour and position in the DOM.

It also conveys meaning by indicating to future us and our teamates that tests depend on the data-test attribute.

Finally it seperates concerns between application code and quality assurance code. And since the data-test attribute isn't needed in production builds we can remove it using the babel-plugin-react-remove-properties plugin:


// .babelrc

{
  "env": {
    "production": {
      "plugins": [["react-remove-properties", { "properties": ["data-test"] }]]
    }
  }
}

There's nothing worse than fixing brittle tests!

Never Miss a Post!

If you have enjoyed this blog post I invite you to join my newsletter:

Emails are kept safe with MailChimp And I never ever send spammy emails (Or stay updated with RSS)