Tips to Speed up Your Phpunit Tests

Having a fast test suite can be just as important as having a fast application. As a developer, getting feedback quickly about the state of your code allows for a much quicker development turnaround. Here we are going to run through some tips you can implement today to make your tests run faster.

The example test suites have been made intentionally slow to simulate a broader set of tests and also to emphasize the improvements possible. Your real-world mileage may vary.

#ParaTest

This package is a PHPUnit extension that runs your test suite, but instead of running each test case in series (one after the other) like PHPUnit does, it can utilize your machine’s CPU cores to run them in parallel.

To get started with ParaTest you will want to install it as a dev dependency via composer.

composer require --dev brianium/paratest

Now, all we need to do is call ParaTest, just like we would call PHPUnit. It will automatically determine how many processes to utilize based on the number of cores available on your machine.

You can see in the console output above that it has determined that five parallel processes will be used to run the test suite. Comparatively, below is the same test suite run in series with PHPUnit.

1.49 seconds versus 6.15 seconds!

Although ParaTest does determine the number of processes to spin up by itself, you may want to try playing around with this number to find the optimal setup for your machine. To specify the number of processes you can use the —processes option. You should try adding and removing processes, as more does not always result in faster tests.

./vendor/bin/paratest --processes 6

Caveat: Before using ParaTest with a test suite that is hitting a database, you need to consider how you are preparing the database. If you are using Laravel’s RefreshDatabase trait, you will run into issues as a test may be rolling back or migrating the database as another is trying to write to it. Instead, skip persisting data by utilizing the DatabaseTransactions trait, which also does not attempt to change the database structure during the test suite run.

#Re-running failed tests

PHPUnit has a handy feature that allows you to re-run only the tests that failed in the previous run. If you are doing red green TDD style development, this is going to speed up your development cycle. Let’s take a look at this feature by starting with a test suite that is passing all the existing tests.

Next, you add a new test that, going by the red-green-refactor model, fails as expected:

After making the changes to your codebase that you believe will make this new test pass, you want to re-run the suite to verify it is functioning as expected. The problem is that this test suite already takes 1.3 seconds to run, so as we continue to add more tests the time spent waiting to verify your code increases.

Wouldn’t it be great if we could run only the failed test we are trying to address? Luckily for us PHPUnit v7.3 added the ability to do this.

To get this working add cacheResult="true" to your phpunit.xml configuration. This tells PHPUnit always to remember which tests previously failed.

<?xml version="1.0" encoding="UTF-8"?>
<phpunit cacheResult="true"
         backupGlobals="false"
         ...>

Now when we run our test suite, PHPUnit will remember which tests are failing and using the following options we can re-run only those that failed.

./vendor/bin/phpunit --order-by=defects --stop-on-defect

We no longer need to wait around for the entire suite to run to see if the one test we are attempting to address is passing.

It is also a good idea to add the cache file .phpunit.result.cache to your .gitignore so that it does not end up being committed to your repository.

#Group slow tests

PHPUnit allows you to add tests to different “groups” with the @group annotation. If you have a bunch of tests that are particularly slow, it might be good to add them all to the same group.

class MyTest extends TestCase
{
    public function test_that_is_fast()
    {
        $this->assertTrue(true);
    }
 
    /**
     * @group slow
     */
    public function test_that_is_slow()
    {
        sleep(10);
 
        $this->assertTrue(true);
    }
 
    /**
     * @group slow
     */
    public function test_that_is_slow_2_adrians_revenge()
    {
        sleep(10);
 
        $this->assertFalse(false);
    }
}

In this example, we have two tests that are going to take 10 seconds to run. The last thing we want is to be running these tests during our development cycle, especially if you are doing test driven development you need your test suit to be snappy.

As the two slower tests are both in the same group, you can now exclude them from a test run by using PHPUnit’s --exclude-group option.

./vendor/bin/phpunit --exclude-group slow

This command will run all your tests except for those in the slow group which will make your tests run much faster. Another benefit of grouping your tests like this is that you are documenting the slow tests so hopefully you can come back and improve them.

It is important however to have some checks in place to ensure that all your tests, including the slow tests, are run before deploying to production. A good way of doing this is having a CI pipeline setup that runs all your tests.

#Filtering tests

PHPUnit has a --filter option which accepts a pattern that determines which tests are run. If, for example, you have all your tests namespaced, you can run a specific subset of tests by specifying a namespace. The following command will only run tests in the Tests\Unit\Models namespace and exclude all others.

./vendor/bin/phpunit --filter 'Tests\\Unit\\Models'

The --filter option is flexible and allows filtering by methodName, Class::methodName, and even by file path with /path/to/my/test.php. You should review the PHPUnit docs for this option and check out what is possible.

#Password hash rounds

Laravel uses the bcrypt password hashing algorithm by default, which is by design slow and expensive on system resources. If your tests verify user passwords, you could potentially trim more time off your test run by setting the number of rounds the algorithm uses, as the more rounds it performs, the longer it takes.

If you keep your app in sync with the latest changes in the laravel/laravel project you will find that the number of hashing rounds is customizable with an environment variable and is already set to 4, the minimum bcrypt allows, in the phpunit.xml file.

However, if you have not kept up with the latest changes, you can set it in the CreatesApplication trait with the Hash facade.

public function createApplication()
{
    $app = require __DIR__.'/../bootstrap/app.php';
 
    $app->make(Kernel::class)->bootstrap();
 
    // set the bcrypt hashing rounds...
    Hash::rounds(4);
 
    return $app;
}

You can see some pretty neat results of this change in the comments of this tweet from Taylor.

Update your CreatesApplication trait to include this Hash::setRounds call and then compare the speed difference in running your tests. pic.twitter.com/WKGUOgyTCI

— Taylor Otwell (@taylorotwell) December 19, 2017

#In-memory database

Utilizing an in-memory SQLite database is another way to increase the speed of your tests that hit the database. You can get started with this quickly by adding these two environment keys to your phpunit.xml configuration.

<php>
    ...
    <env name="DB_CONNECTION" value="sqlite"/>
    <env name="DB_DATABASE" value=":memory:"/>
</php>

Caveat: Although this might seem like an easy win, you should consider having database parity with your production environment. If you are using something like MySQL in production, then you should be aware of the potential issues that could be introduced by testing with a different database, such as SQLite. I go into more detail on the differences presented in my feature test suite setup. tl;dr; I believe having parity with your production environment during testing is more important than gaining a small speed increase.

#Disable Xdebug

If you are not using Xdebug regularly, you might consider disabling it until you need it. It slows down PHP execution, and as a result, your test suite. If you are using it for debugging on the daily, disabling it for test runs probably isn’t a great option – but it is something to keep in mind when it comes to the speed of your test suite.

You can see in this test suite a substantial speed increase once we disable Xdebug. This is the suite running with Xdebug enabled:

and the same test suite with Xdebug disabled:

#Fix your slow tests

The best tip in all of this is of course: fix your slow tests! If you are struggling to pinpoint which tests are causing your test suite to be slow, you might want to look at PHPUnit Report. It is an open-source tool that allows you to visualize your test suite’s performance by generating a cloud, shown below, with the bigger bubbles representing slow tests. This will enable you to find the slowest tests in your suite and incrementally improve their performance.