SWPD #011: The Four Levels of Automated Testing

This week, I’m going to walk you through the four levels of automated testing.

Each level of testing covers a different aspect of the overall quality of your plugin code.

It provides confidence that the code does what you expect it to do, in all cases.

Unfortunately, lots of developers aren’t aware of each level and what they should be testing in each.

The four levels of testing cover each aspect of plugin code

Without testing each level:

  • Refactoring is difficult
  • Each new release is stressful
  • Debugging is near impossible
  • You’ll keep fixing the same features

Once you understand how all tests work together, you’ll no longer need to worry about breaking your plugin when adding new features or adjusting existing ones.

Let’s dig in:

Acceptance Tests

The first tests that you’ll want to write when first starting to add tests are acceptance tests.

This initial level verifies that your plugin does, from the user’s perspective, exactly what you expect.

When running acceptance tests, you’re ensuring that:

  • When a user deletes a post, the post should no longer be visible in the posts list
  • When a plugin is activated, no errors are thrown on the plugins page

Acceptance tests are the best initial tests to start writing because they cover the whole plugin.

Functional Tests

Like acceptance tests, functional tests are used to make assertions as a developer would when interacting with your plugin.

Imagine that you have a form in your plugin that lets the user save some text in their profile.

A functional test would send a POST request to the right endpoint to save those fields and you would assert that the expected values are saved to the database how you expect.

These are a little more granular that acceptance tests, but are really beneficial to coax out hard to debug issues.

Integration Tests

The next level down is testing whole features (or modules).

That means loading up each dependency, and making assertions about the state of the plugin after the full module runs.

For example, imagine that you are registering a custom post type: Book

When writing your tests, you would be testing that all the related WordPress functions run properly so that you can prove that the new post type was registered.

You use integration tests to cover large modules of code to test that whole features are working as they should.

Unit Tests

At the very bottom of your test suite are unit tests. These tests verify each method and function in isolation.

As an example, I needed to write a regex function that extracted text from the title of a post.

I was able to provide the test with a list of titles with various options of the extracted text.

Then I could work through each variance to nail down the right regex that picked them all out correctly.

I was confident all edge cases were covered because my test passed with all the different combinations.

In Summary

The four levels of automated testing all work together to provide a wholistic view of how consistent your code is when handling various unique situations.

Acceptance Tests: Verify that interacting with your code from a user’s perspective works as intended

Functional Tests: Verify that interactions with your code from a developer’s perspective do what you expect

Integration Tests: Validate whole features or modules to add confidence they handle all scenarios consistently

Unit Tests: Test functions and methods in as much isolation as possible. Inputs should have consistent and repeatable outputs

If you’re interested in learning more, I’ve gathered a couple resources that should help: