Find Feature Tests Creating Database Records without Refreshing the Database in Laravel

When feature testing a Laravel application, Laravel provides useful tools for database testing, like refreshing the database, factories, seeders, and database assertions. In a modern Laravel application, a Pest test using database refreshing might look like the following:

use Illuminate\Foundation\Testing\RefreshDatabase;
 
pest()->use(RefreshDatabase::class);
 
test('basic example', function () {
    $user = User::factory()->create();
    $response = $this->actingAs($user)->get('/');
 
    // ...
});

Unfortunately, developers may sometimes forget to include the RefreshDatabase trait, which can cause random database assertion failures in other tests. There are a few ways to mitigate this, such as including this trait across all feature tests, so it's always included. In the tests/Pest.php file, you can uncomment this line:

pest()->extend(Tests\TestCase::class)
    ->use(Illuminate\Foundation\Testing\RefreshDatabase::class)
    ->in('Feature');

The Laravel docs have this to say about the RefreshDatabase trait:

The Illuminate\Foundation\Testing\RefreshDatabase trait does not migrate your database if your schema is up to date. Instead, it will only execute the test within a database transaction. Therefore, any records added to the database by test cases that do not use this trait may still exist in the database.

So, any tests that don't use this trait won't run in a rolled-back transaction and can cause weird, unexpected test assertion failures in your application. One way I've found to hunt down offending tests more quickly is using grep to find files that don't use RefreshDatabase, but also include factory()/seed()/etc. calls:

grep -rL 'RefreshDatabase' tests/Feature | xargs grep -l '::factory('

This command lists out files that do not contain the RefreshDatabase trait, yet use ::factory() of some kind. Let's say we remove the trait in our example test above and run this command:

$ grep -rL 'RefreshDatabase' tests/Feature | xargs grep -l '::factory('
 
tests/Feature/ExampleTest.php

One caveat here is that a test without ::factory() calls might still create records via seeders or HTTP requests. You can just run the first part of the command and investigate more carefully, ask AI, or further filter. Using the first part of the original command still lists our example file. Usually, this is probably good enough to investigate further files missing this trait:

$ grep -rL 'RefreshDatabase' tests/Feature
 
tests/Feature/ExampleTest.php

You could get a little crazy, but this will find files that don't use the RefreshDatabase trait, and contain one of the following calls:

grep -rL 'RefreshDatabase' tests/Feature \
  | xargs grep -El '::factory\(|->seed\(|->putJson\(|->postJson\(|->post\(|->put\('

There are some caveats here: Laravel has other database migration traits such as DatabaseMigrations and DatabaseTruncation. Also, if you have the RefreshDatabase trait as an unused import, that file will not be reported. The above search command isn't perfect, but it can be modified to your needs.

You could even turn this into a GitHub workflow that fails if any files are returned using the above search. Your mileage might vary here, but here's a hypothetical (untested) version of a GitHub workflow that checks for any files fitting this condition:

name: Check for Missing RefreshDatabase
 
on:
  pull_request:
    paths:
      - 'tests/Feature/**'
 
jobs:
  check-refresh-database:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
 
      - name: Find files missing RefreshDatabase that use database features
        run: |
          set -e
          files=$(grep -rL 'RefreshDatabase' tests/Feature | xargs grep -El '::factory\(|->seed\(|->putJson\(|->postJson\(|->post\(|->put\(' || true)
          if [ -n "$files" ]; then
            echo "The following files are missing 'RefreshDatabase':"
            echo "$files"
            exit 1
          fi

Another caveat: database records could be inserted via models! As I mentioned, this grep search isn't perfect; it's a tool to help you find an offending file that inserts database records without a migration more quickly.

I recommend using the following conventions when possible:

• Stick to the RefreshDatabase trait in your tests whenever possible. It's faster than database truncation and the conventional (non-transactional) database migration trait.
• Prefer using factories and seeders in tests over manually creating model data via models.
• Make sure HTTP tests creating records use the RefreshDatabase trait.

There are probably other conventions I am not thinking of—please let us know on your favorite social network what tools you use to debug tests that create database records without migrations.