Fair Queue Distribution with Laravel Balanced Queue

Laravel Balanced Queue, created by YanGusik, manages queue distribution across user groups with per-user concurrency control and multiple partition strategies to prevent queue monopolization.

#How It Works

Default Laravel queues use FIFO processing. If User A dispatches 10,000 jobs while User B dispatches 5, User B waits for all of User A's jobs to complete first. Laravel Balanced Queue divides your queue into partitions—typically one per user or tenant—and rotates through them, giving each user fair access to workers.

#Getting Started

Add the BalancedDispatchable trait to your job classes:

use Illuminate\Contracts\Queue\ShouldQueue;
use YanGusik\BalancedQueue\Jobs\BalancedDispatchable;
 
class GenerateAIImage implements ShouldQueue
{
    use BalancedDispatchable;
 
    public function __construct(
        public int $userId,
        public string $prompt
    ) {}
 
    public function handle(): void
    {
        // AI generation logic
    }
}

Dispatch jobs using the balanced connection:

GenerateAIImage::dispatch($userId, $prompt)->onConnection('balanced');

The package automatically detects partition keys from properties like $userId, $user_id, or $tenantId, grouping each user's jobs into their own partition.

#Partition Strategies

The package offers three rotation strategies in config/balanced-queue.php: round-robin (default) cycles through partitions in strict order for equal distribution, random selects probabilistically for high-throughput scenarios, and smart prioritizes smaller queues to prevent starvation when job volumes vary between users.

#Concurrency Limiting

This package lets you set the max_concurrent in config/balanced-queue.php to limit how many jobs from a single partition can run simultaneously:

'limiters' => [
    'simple' => [
        'max_concurrent' => 2,
    ],
],

This prevents any user from consuming all workers and helps manage API rate limits when jobs make external requests.

#Monitoring

Monitor partition status with php artisan balanced-queue:table --watch for live updates showing pending jobs and active workers per partition. The package also includes commands to clear queues and a programmatic metrics API for custom dashboards.

#Horizon Integration

The package works with Laravel Horizon. Configure a supervisor in config/horizon.php using the balanced connection, and use php artisan balanced-queue:table for accurate metrics since Horizon's pending counts don't reflect partitioned queue structures.

#Learn More

To learn more about Laravel Balanced Queue and view the source code, visit the GitHub repository. The package requires PHP 8.0+ and Laravel 9.0+, with Redis as the queue driver.