Laravel Jobs and Queue 101: How to run your workers on your production server

Published on by

Laravel Jobs and Queue 101: How to run your workers on your production server image

Now that you’ve built your first queue/jobs based Laravel application from parts one and two, it is time to deploy it to a production server.

The only difference comparing to “regular” Laravel applications (I.e the ones without queues and jobs), is that we need to tell the server to run the queue:work command and keep it running even after we close our ssh connection to the server and even when we restart the server.

We will explore how we are going to do this in two different ways, the first one and it is the simplest one, is by using Forge, the de facto service to deploy Laravel applications, and then we will see how we can deploy without forge, in other terms, we will learn about what Forge is doing to make queues and workers function properly on our production server.

Deploying with forge

I’ll suppose that you have already deployed your application to forge, and I’ll focus mainly and what to do next in order to make the queue:work command work in the background.

So head to your application on Forge, and then click on the “Queue” tab on the left side bar (Site Details).

all we need to do is fill in the form in order to create our background worker(s) as follows:

PS: we do not need all the fields, these are the most important ones

  • Connection: rabbitmq (the name of our queue connection)
  • Queue: jobs. In our last article we had two queues, one called jobs and the other called emails. We are going to create a new worker for each queue.
  • Processes : 4. here we are specifying how many background workers we want to consume the queues in parallel. In the examples we had in the previous articles we were processing one job at a time, if we have enough resources in our server and if we want to consume the queue more quickly, we can create as many processed for the same queue as we wish.
  • Maximum Tries : 3. here we are telling our application to retry the same job 3 times before we mark it as a failed job (I.e sent to the failed_jobs table).

This also depends on the nature of your applications. In some applications you might want to retry the same job multiple times before you mark it as failed, in other application it would be more appropriate to mark a job as failed right away.

Now after you click on the “Start worker” button, it will appear in the “Active workers” table:

As you can see, you can either restart or delete the worker completely from this table.

The only issue you might notice here is that you’d need to recreate the worker from scratch in case you want to update it, even to just update the number of workers.

What’s next?

After you create the workers via Forge, you won’t need to do anything else. Even after you redeploy your application or restart your server, Forge will take care of everything, including making sure that the workers are running and restarting them after each code change.

Deploying without Laravel Forge

Deploying without Forge will show us exactly how much forge is doing behind the seen.

let’s see what are all the steps needed in order to make our workers run on our production server.

Installing Supervisor

To ensure that our queue:work command keep running all the time, we need to install a process monitor on our server like Supervisor.

If you are on Ubuntu, you can install it via this command:

sudo apt-get install supervisor

Configuring Supervisor

We need to tell Supervisor which queues we want to consume and how many workers we want to create.

In order to do that we need to create one configuration file for each queue, and store them in the /etc/supervisor/conf.d directory

Configuration files look like this (example taken from the Laravel documentation):

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /home/forge/app.com/artisan queue:work sqs --sleep=3 --tries=3
autostart=true
autorestart=true
user=forge
numprocs=8
redirect_stderr=true
stdout_logfile=/home/forge/app.com/worker.log
stopwaitsecs=3600

Here is the configuration file that will recreate the same configuration we created in the previous section using forge

[program:jobs-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /home/djug/basic-analytics-v01/artisan queue:work rabbitmq --queue=jobs --tries=3
autostart=true
autorestart=true
user=djug
numprocs=4
redirect_stderr=true
stdout_logfile=/home/djug/basic-analytics-v01/worker.log

After you add all the configuration files you need, you’d need to execute the following commands in order to take the new changes into consideration:

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start all

You’d need to do the same each time you update your configuration files

Conclusion

In this series, we explored all the subjects that will allow you to create a jobs-based Laravel application. We saw how we would create Laravel jobs and how to use different queue connections to handle them. We also saw what we need to do in order to make our workers function properly in our production server.

Youghourta Benali photo

Back-end developer http://youghourta.com I built:

  • botmarker.com
  • bookmarkingBot.com
  • todocol.com

I'm also the author of "Laravel Testing 101" http://laraveltesting101.com

Filed in:
Cube

Laravel Newsletter

Join 40k+ other developers and never miss out on new tips, tutorials, and more.

Laravel Forge logo

Laravel Forge

Easily create and manage your servers and deploy your Laravel applications in seconds.

Laravel Forge
Tinkerwell logo

Tinkerwell

The must-have code runner for Laravel developers. Tinker with AI, autocompletion and instant feedback on local and production environments.

Tinkerwell
No Compromises logo

No Compromises

Joel and Aaron, the two seasoned devs from the No Compromises podcast, are now available to hire for your Laravel project. ⬧ Flat rate of $7500/mo. ⬧ No lengthy sales process. ⬧ No contracts. ⬧ 100% money back guarantee.

No Compromises
Kirschbaum logo

Kirschbaum

Providing innovation and stability to ensure your web application succeeds.

Kirschbaum
Shift logo

Shift

Running an old Laravel version? Instant, automated Laravel upgrades and code modernization to keep your applications fresh.

Shift
Bacancy logo

Bacancy

Supercharge your project with a seasoned Laravel developer with 4-6 years of experience for just $2500/month. Get 160 hours of dedicated expertise & a risk-free 15-day trial. Schedule a call now!

Bacancy
Lucky Media logo

Lucky Media

Get Lucky Now - the ideal choice for Laravel Development, with over a decade of experience!

Lucky Media
Lunar: Laravel E-Commerce logo

Lunar: Laravel E-Commerce

E-Commerce for Laravel. An open-source package that brings the power of modern headless e-commerce functionality to Laravel.

Lunar: Laravel E-Commerce
LaraJobs logo

LaraJobs

The official Laravel job board

LaraJobs
SaaSykit: Laravel SaaS Starter Kit logo

SaaSykit: Laravel SaaS Starter Kit

SaaSykit is a Laravel SaaS Starter Kit that comes with all features required to run a modern SaaS. Payments, Beautiful Checkout, Admin Panel, User dashboard, Auth, Ready Components, Stats, Blog, Docs and more.

SaaSykit: Laravel SaaS Starter Kit
Rector logo

Rector

Your partner for seamless Laravel upgrades, cutting costs, and accelerating innovation for successful companies

Rector
MongoDB logo

MongoDB

Enhance your PHP applications with the powerful integration of MongoDB and Laravel, empowering developers to build applications with ease and efficiency. Support transactional, search, analytics and mobile use cases while using the familiar Eloquent APIs. Discover how MongoDB's flexible, modern database can transform your Laravel applications.

MongoDB

The latest

View all →
A Resize Plugin for Alpine.js image

A Resize Plugin for Alpine.js

Read article
How to Migrate MySQL from DBngin to Laravel Herd image

How to Migrate MySQL from DBngin to Laravel Herd

Read article
Learn to master Query Scopes in Laravel image

Learn to master Query Scopes in Laravel

Read article
How to Redirect Uppercase URLs to Lowercase with Laravel Middleware image

How to Redirect Uppercase URLs to Lowercase with Laravel Middleware

Read article
PHP 8.4 Alpha 1 is now out! image

PHP 8.4 Alpha 1 is now out!

Read article
Generics Added to Eloquent Builder in Laravel 11.15 image

Generics Added to Eloquent Builder in Laravel 11.15

Read article