Laravel Tutorials

Ship AI with Laravel: Failover, Queues, and Middleware for AI Agents

Published

▶️ Watch the video tutorial (10 minutes)

Our support platform works great in development. Then you ship it. OpenAI has an outage at 2am and every customer gets an error. A hundred people hit the chat at once and requests pile up. A response comes back wrong and three weeks later you have no way to figure out why. This episode is about the infrastructure that handles all of that.

We start with provider failover, the scariest one. Right now the agent is locked to a single provider. I show how to pass an array instead, so if OpenAI fails the SDK automatically retries with Anthropic, then Gemini. Same agent, same tools, same instructions, just a different provider behind the scenes. The customer never knows. I set it up at the call site first, then move the whole failover chain into config so you can change it without touching code. I also add a listener for the AgentFailedOver event so you get alerted the moment a provider drops.

Next, queues. Not every AI call needs to block a request. I take the TicketClassifier from Episode 2 and swap prompt for queue, so classification runs in the background through Laravel's queue system. The customer gets an instant confirmation and the AI work happens behind the scenes, with failover still in place on the queued job.

Then middleware. The SDK lets you intercept every prompt and response flowing through an agent, exactly like HTTP middleware. I build three. One logs each prompt along with response metadata like token counts, provider, and duration. One handles per-user rate limiting at ten prompts a minute. One tracks cost using token counts so you can calculate spend per user, per agent, per day. Then I attach all three to the agent in a deliberate order, because rate limiting should run before you waste time logging or tracking a request you're about to reject.

By the end, the agent survives provider outages, handles background work, and logs everything you need to debug and budget in production.

Next episode we test the entire system with Pest, faking agents, asserting prompts, and testing tools so nothing breaks even when you ship on a Friday afternoon.

⭐ GitHub: https://github.com/harris21/ship-ai-with-laravel

▶️ Watch Episode 9 now →

Harris Raftopoulos photo

Senior Software Engineer • Staff & Educator @ Laravel News • Co-organizer @ Laravel Greece Meetup

Sponsored

laravelcloud logo
Laravel Cloud

Easily create and manage your servers and deploy your Laravel applications in seconds.

Visit Laravel Cloud

The latest

View all →
Durable Workflow: Long-Running Workflows for Laravel image

Durable Workflow: Long-Running Workflows for Laravel

Read article
Session Replay for Laravel: Self-Hosted Browser Recordings image

Session Replay for Laravel: Self-Hosted Browser Recordings

Read article
Laravel AI SDK 1.4 Lets Agents Recover From Tool Failures image

Laravel AI SDK 1.4 Lets Agents Recover From Tool Failures

Read article
Laravel Read Replicas: Check if a Replica Has Caught Up image

Laravel Read Replicas: Check if a Replica Has Caught Up

Read article
UUID Request Input and Migration Failure Events in Laravel 13.42 image

UUID Request Input and Migration Failure Events in Laravel 13.42

Read article
Replay Missed Laravel Scheduled Tasks With Schedule Catchup image

Replay Missed Laravel Scheduled Tasks With Schedule Catchup

Read article