Laravel Packages

Find Unexpected Test Inputs with Fuzz for Pest

Published
Find Unexpected Test Inputs with Fuzz for Pest image

A test for a parser might cover a valid string and an empty one. Fuzz testing searches for failures beyond those hand-written cases by generating inputs and passing them to your code.

With Fuzz, a package by Jon Purvis, you can do this inside a Pest 5 test. The package uses nikic's PHP-Fuzzer to change strings you provide and report any failures through Pest.

What Coverage-Guided Means

A fuzzer changes an input by adding, removing, or replacing parts of the string. It then runs your code with the changed input.

A coverage-guided fuzzer also watches which routes those inputs take through your code. When an input reaches a previously unexplored route, the fuzzer saves it and uses it to make more variations. The saved collection is called a corpus.

Suppose a parser checks for an opening bracket before reading the rest of a string. Many random inputs will fail that first check. Once an input gets past it, the fuzzer can keep changing that string to exercise the code that follows.

PHP-Fuzzer collects this feedback by tracking transitions between blocks of PHP code and roughly how often they run. Fuzz handles this for you; you do not need Xdebug or Pest's --coverage option.

You choose the code to call. Fuzz tries generated strings until it finds a failure or reaches its run budget. A passing run means it found no failure in those attempts; bugs may still remain.

Trying It in Pest

Fuzz requires PHP 8.4+ and Pest 5. Install it with Composer:

composer require jonpurvis/fuzz --dev

Suppose your app reads a rate limit such as 100/60s, meaning 100 requests per 60 seconds. This helper converts it to requests per second, but does not validate the input:

namespace App;
 
final class RateLimit
{
public static function perSecond(string $spec): float
{
$parts = explode('/', $spec);
 
$count = (int) $parts[0];
$window = (int) rtrim($parts[1] ?? '1s', 's');
 
return $count / $window;
}
}

Save the helper in app/RateLimit.php. In tests/Unit/RateLimitTest.php, define a function outside the test to call the helper:

use App\RateLimit;
use function Fuzz\fuzz;
 
$target = static function (string $input): void {
RateLimit::perSecond($input);
};
 
test('rate limit spec parser never fatals', function () use ($target): void {
fuzz($target)
->seed(['100/60s', '5/1s', '1000/3600s'])
->withDictionary(['/', 's', '0', '1'])
->runs(2000)
->maxLen(16)
->run('rate-limit-parser');
});

The strings passed to seed() are starting examples. withDictionary() supplies fragments the fuzzer can insert, without restricting it to those characters. runs(2000) sets the search budget, and maxLen(16) limits generated strings to 16 bytes.

Give each fuzz test its own name in run(). Fuzz uses that name to separate its saved inputs and crash files.

Keep the $target function outside test() as shown. In our check of Fuzz v1.0.1, this wrapper recorded coverage, while passing Closure::fromCallable() directly recorded none. Fuzz runs the function in a separate PHP process, where Pest's generated test class is unavailable. Defining it outside test() avoids depending on that class.

Run the test as usual:

./vendor/bin/pest tests/Unit/RateLimitTest.php

In our run, Fuzz found 5/. The missing window becomes an empty string, which PHP casts to 0. Dividing by it throws DivisionByZeroError, and Pest reports the test as failed. Your run may find a different input or take a different number of attempts.

Fuzz saves the failing input under .pest/fuzz-crashes/ by default. You can read that file to reproduce the failure. After fixing the parser, add the input to a named dataset and assert the expected behavior so the same bug cannot pass unnoticed.

What the Test Checks

This example fails when the helper crashes, but a wrong return value could pass. To catch that, add a Pest expectation inside the target function so it checks each generated input. For example, a test for an encoder and decoder could check that encoding then decoding a string returns the original value.

Fuzz reports errors such as TypeError, as well as unsuppressed PHP warnings and notices. Ordinary exceptions are ignored by default, including Laravel validation exceptions. The allow() method lets you narrow that list when unexpected exceptions should fail the test. You can also set a per-input timeout with timeout(), which requires PHP's pcntl extension.

Using Fuzz Alongside Your Tests

Keep your regular tests and datasets for known cases and expected results. Fuzz is useful when code accepts more possible inputs than you can reasonably list, such as a parser reading user-supplied text.

The rate limit helper has little branching. Coverage guidance is more useful in parsers with successive validation steps, where passing one check lets the fuzzer reach the next.

Start with a small run budget in your normal suite. If you need a longer search, run it in a scheduled CI job. The Fuzz README covers the remaining options, including dictionaries and custom storage directories.

Yannick Lyn Fatt photo

Staff Writer at Laravel News and Full stack web developer.

Sponsored

laravelcloud logo
Laravel Cloud

Easily create and manage your servers and deploy your Laravel applications in seconds.

Visit Laravel Cloud

The latest

View all →
Unlearn.dev goes free for a weekend, October 10 and 11 image

Unlearn.dev goes free for a weekend, October 10 and 11

Read article
Tashil: Laravel Subscription Plans and Feature Usage Limits image

Tashil: Laravel Subscription Plans and Feature Usage Limits

Read article
Decide with Jev: Laravel AI That Answers with a Probability image

Decide with Jev: Laravel AI That Answers with a Probability

Read article
JetBrains Air: Run AI Coding Agents in JetBrains IDEs image

JetBrains Air: Run AI Coding Agents in JetBrains IDEs

Read article
Memoize Tagged Cache Reads in Laravel image

Memoize Tagged Cache Reads in Laravel

Read article
Eloquent Refreshes: Load Generated Columns After Save image

Eloquent Refreshes: Load Generated Columns After Save

Read article