Database Job Chains for Laravel

Packages

September 8th, 2022

Database Job Chains for Laravel

Laravel Haystack is a package for beautifully simple but powerful database-driven job chains. Here's an overview of what is available, taken from the package's readme:

1$haystack = Haystack::build()
2 ->withName('Podcast Chain')
3 ->addJob(new RecordPodcast)
4 ->addJob(new ProcessPodcast)
5 ->addJob(new PublishPodcast)
6 ->then(function () {
7 // Haystack completed
8 })
9 ->catch(function () {
10 // Haystack failed
11 })
12 ->finally(function () {
13 // Always run either on success or fail.
14 })
15 ->paused(function () {
16 // Run if the haystack is paused
17 })
18 ->withMiddleware([
19 // Middleware to apply on every job
20 ])
21 ->withDelay(60) // Add a delay to every job
22 ->dispatch();

With this package, you store job chains in the database, which helps make memory consumption low and supports all of Laravel's queue types out of the box. Some of the other main features include:

  • Low memory consumption as one job is processed at a time and the chain is stored in the database
  • You can delay/release jobs for as long as you want since it will use the scheduler to restart a chain. Even if your queue driver is SQS!
  • It provides callback methods like then, catch and finally
  • Global middleware that can be applied to every single job in the chain
  • Delay that can be added to every job in the chain
  • You can store and retrieve data/state that is accessible to every job in the chain.
  • You can store the model for later processing.

You can do pretty cool stuff, such as pausing haystacks if your job chain calls an API and hits rate limits. While Laravel has job chains, I think you should consider this package for batched jobs. Check out the readme for the full details of what differentiates this package from the built-in chain abilities.

You can learn more about this package, get full installation instructions, and view the source code on GitHub.

Filed in:

Paul Redmond

Full stack web developer. Author of Lumen Programming Guide and Docker for PHP Developers.