Running the Laravel Scheduler and Queue with Docker

Published on by

Running the Laravel Scheduler and Queue with Docker image

In Laravel, one of the tricky changes when switching from a virtual server to Docker is figuring out how to run a scheduler and a queue worker. I see this question come up quite a bit when PHP developers are trying to figure out how to use Laravel with Docker.

Should you run them on the host server? Should you run via cron in a Docker container?

There are a couple of ways I recommend running the scheduler command and Laravel queues in Docker, and we’re going to cover the basics of running both with a complete (albeit simple) Docker setup you can use to experiment.

Multi-Purpose Docker Image

In the context of Docker, you can split your workloads into separate containers and scale them independently. You could have multiple containers running queue workers, one running a scheduler and a handful of containers running your web application. Laravel’s design is monolithic, meaning that your queue jobs, scheduled commands, and HTTP endpoints share a single codebase.

When you start splitting up HTTP traffic, queues, and scheduled commands with Docker, you have to make some decisions about how to build an image for each purpose.

For example, do you define a separate Dockerfile for each context you run your Laravel code? One for your web application, one for your queue, and one for your scheduler?

I would propose that with a bit of clever scripting, we can build a single flexible Docker image that can support all three roles. That means one image to build, that can run as a web server, a schedule runner, or a queue worker.

With Docker you can split up your workloads in new and exciting ways that aren’t possible on a traditional server. While on an Ubuntu server, you’d probably run your web server, queues, and the scheduler command on the same machine(s). With Docker however, it doesn’t make sense to run all of these processes in the same container.

Let’s look at how we can accomplish this with a bash script to run our Docker CMD.

Project Setup

Before we get into how we can run our application in different roles within Docker, let’s set up a simple Laravel project with Docker using an Apache web server.

We’ll use the official php Docker image as our base image, and Docker Compose to run MySQL and Redis. First, let’s set up the files we need to set up a Docker environment:

laravel new docker-laravel
cd ./docker-laravel
mkdir docker/
touch docker-compose.yml
touch docker/Dockerfile
touch docker/start.sh
touch docker/vhost.conf

Docker Compose

This post isn’t about using Docker Compose, so if you’re not familiar with that I’d recommend going through the Docker Compose documentation, and my Docker PHP book (there’s also a book-only version) goes through plenty of Docker Compose examples throughout the text.

Here’s the Docker Compose file with the services we’ll need to run our application services:

version: "3"
services:
app:
image: laravel-www
container_name: laravel-www
build:
context: .
dockerfile: docker/Dockerfile
depends_on:
- redis
- mysql
ports:
- 8080:80
volumes:
- .:/var/www/html
environment:
APP_ENV: local
CONTAINER_ROLE: app
CACHE_DRIVER: redis
SESSION_DRIVER: redis
QUEUE_DRIVER: redis
REDIS_HOST: redis
 
redis:
container_name: laravel-redis
image: redis:4-alpine
ports:
- 16379:6379
volumes:
- redis:/data
 
mysql:
container_name: laravel-mysql
image: mysql:5.7
ports:
- 13306:3306
volumes:
- mysql:/var/lib/mysql
environment:
MYSQL_DATABASE: homestead
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: homestead
MYSQL_PASSWORD: secret
 
volumes:
redis:
driver: "local"
mysql:
driver: "local"

Take note of the CONTAINER_ROLE environment variable, which will come into play when we start fleshing out the custom Docker script. We also define some development database settings for MySQL and set up volumes to persist our MySQL and Redis data.

In the app service, we mount a volume so that our PHP and frontend code changes reflect immediately during development.

The Dockerfile

We set up a MySQL, Redis, and Laravel application service which will be built from the docker/Dockerfile we need to update:

FROM php:7.2-apache-stretch
 
COPY . /var/www/html
COPY docker/vhost.conf /etc/apache2/sites-available/000-default.conf
 
RUN chown -R www-data:www-data /var/www/html \
&& a2enmod rewrite

The Dockerfile gets most of its functionality from the official PHP Apache image. We are copying the Laravel code and an Apache Vhost file. The Vhost file override is needed so that we can point the DocumentRoot to /var/www/html/public, which is what Laravel expects.

Then we’re changing the ownership of files to the www-data user for correct permissions in production. Last, we enable mod_rewrite, so that URL rewrites will work.

The Apache Vhost

Next, we need to fill in the docker/vhost.conf file to point to our public folder. Add the following to docker/vhost.conf, which is copied into the image, overriding the 000-default.conf vhost file:

<VirtualHost *:80>
DocumentRoot /var/www/html/public
 
<Directory "/var/www/html/public">
AllowOverride all
Require all granted
</Directory>
 
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

At this point you can build the image and you should be able to get Laravel’s welcome page on localhost:8080:

docker-compose up --build

Custom Docker CMD Instruction

Our docker/Dockerfile is extending the official PHP Apache image using Debian Stretch. If you look closely at the base Apache image we are extending, you will see a CMD instruction:

CMD ["apache2-foreground"]

We are going to override the CMD instruction in our own Dockerfile, so let’s start out by building a structure for our bash script and using apache2-foreground in the script. We will then extend the script to include roles for a scheduler and a queue.

Update the docker/Dockerfile to look like the following:

FROM php:7.2-apache-stretch
 
COPY . /var/www/html
COPY docker/vhost.conf /etc/apache2/sites-available/000-default.conf
COPY docker/start.sh /usr/local/bin/start
 
RUN chown -R www-data:www-data /var/www/html \
&& chmod u+x /usr/local/bin/start \
&& a2enmod rewrite
 
CMD ["/usr/local/bin/start"]

We’re now copying in a bash script from docker/start.sh into /usr/local/bin/start and making it executable. Last, we override the CMD instruction to run our bash script.

Before we fill in a queue worker, let’s get the apache server working again. Add the following to docker/start.sh:

#!/usr/bin/env bash
 
set -e
 
role=${CONTAINER_ROLE:-app}
env=${APP_ENV:-production}
 
if [ "$env" != "local" ]; then
echo "Caching configuration..."
(cd /var/www/html && php artisan config:cache && php artisan route:cache && php artisan view:cache)
fi
 
if [ "$role" = "app" ]; then
 
exec apache2-foreground
 
elif [ "$role" = "queue" ]; then
 
echo "Queue role"
exit 1
 
elif [ "$role" = "scheduler" ]; then
 
echo "Scheduler role"
exit 1
 
else
echo "Could not match the container role \"$role\""
exit 1
fi

There’s quite a bit in this script. First, we are checking for environments other than local, and running production-like caching. The fact that we can cache config and routes is worth using a custom bash script, even if you don’t plan on running a queue or a scheduler. Be aware that if you accidentally run your container without the APP_ENV=local environment that you’ll need to clear your config cache php artisan config:clear to avoid cached values during development.

In bash, you can set default values for variables, so we make the app role the default if the CONTAINER_ROLE env variable is not set. Note, that we are setting this in the docker-compose.yml file.

Last, the heart of our script, is the logic based on the container role. Right now we just echo the role type for the scheduler and the queue, and we run exec apache2-foreground to start Apache in the app role.

If you rebuild the Docker image and run Docker compose again, the start script should run Apache and serve the Laravel application:

docker-compose down
docker-compose build
docker-compose up

Running a Scheduler

We can modify our docker/start.sh script to run a scheduler. The neat thing about this is that we can accomplish this with a single Docker image and modify how it works at runtime based on the role.

If you were setting up the Scheduler to run on a traditional server, you’d set up a Cron entry for it like so:

* * * * * php /path-to-your-project/artisan schedule:run >> /dev/null 2>&1

The Cron entry is defined so that the script will run every 60 seconds. We can emulate this behavior without installing Cron in our Docker image, using bash. The tricky part is that our container must keep a process running in the foreground, otherwise the container will exit.

We can accomplish the same design as the cron using an infinite bash loop:

if [ "$role" = "app" ]; then
 
exec apache2-foreground
 
elif [ "$role" = "queue" ]; then
 
echo "Queue role"
exit 1
 
elif [ "$role" = "scheduler" ]; then
 
while [ true ]
do
php /var/www/html/artisan schedule:run --verbose --no-interaction &
sleep 60
done
 
else
echo "Could not match the container role \"$role\""
exit 1
fi

If we run a container from our image with CONTAINER_ROLE=scheduler an infinite bash while loop will run in the foreground, and every 60 seconds a new schedule:run command will run in the background (using & at the end).

Running this command in the background is vital for this to work like a Cron. If we run the schedule:run command in the foreground, the script will stop execution and the sleep 60 will not fire until the schedule:run command finishes.

In order to try this out, uncomment the inspire command (or add your own) in the app/Console/Kernel.php file:

/**
* Define the application's command schedule.
*
* @param \Illuminate\Console\Scheduling\Schedule $schedule
* @return void
*/
protected function schedule(Schedule $schedule)
{
$schedule->command('inspire')
->everyMinute();
}

Next, we need to define a scheduler service in the docker-compose.yml file that depends on our www-laravel image:

scheduler:
image: laravel-www
container_name: laravel-scheduler
depends_on:
- app
volumes:
- .:/var/www/html
environment:
APP_ENV: local
CONTAINER_ROLE: scheduler
CACHE_DRIVER: redis
SESSION_DRIVER: redis
QUEUE_DRIVER: redis
REDIS_HOST: redis

Take note that we are duplicating some environment variables values between the app and scheduler services. You could optimize this with a dedicated Docker compose env_file property, or using Laravel’s .env file. I prefer to use an external Docker environment file for development.

Here’s the entire docker-compose.yml file for reference. We will also add the queue service now, so you don’t have to do that later on:

version: "3"
services:
app:
image: laravel-www
container_name: laravel-www
build:
context: .
dockerfile: docker/Dockerfile
depends_on:
- redis
- mysql
ports:
- 8080:80
volumes:
- .:/var/www/html
environment:
APP_ENV: local
CONTAINER_ROLE: app
CACHE_DRIVER: redis
SESSION_DRIVER: redis
QUEUE_DRIVER: redis
REDIS_HOST: redis
 
scheduler:
image: laravel-www
container_name: laravel-scheduler
depends_on:
- app
volumes:
- .:/var/www/html
environment:
APP_ENV: local
CONTAINER_ROLE: scheduler
CACHE_DRIVER: redis
SESSION_DRIVER: redis
QUEUE_DRIVER: redis
REDIS_HOST: redis
 
queue:
image: laravel-www
container_name: laravel-queue
depends_on:
- app
volumes:
- .:/var/www/html
environment:
APP_ENV: local
CONTAINER_ROLE: queue
CACHE_DRIVER: redis
SESSION_DRIVER: redis
QUEUE_DRIVER: redis
REDIS_HOST: redis
 
redis:
container_name: laravel-redis
image: redis:4-alpine
ports:
- 16379:6379
volumes:
- redis:/data
 
mysql:
container_name: laravel-mysql
image: mysql:5.7
ports:
- 13306:3306
volumes:
- mysql:/var/lib/mysql
environment:
MYSQL_DATABASE: homestead
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: homestead
MYSQL_PASSWORD: secret
 
volumes:
redis:
driver: "local"
mysql:
driver: "local"

We also need to build our image so that the updates to the docker/start.sh file are part of the image build:

docker-compose down
docker-compose build
docker-compose up

If everything worked, you should see the following output for the scheduler container:

laravel-scheduler | No scheduled commands are ready to run.
laravel-scheduler | Running scheduled command: '/usr/local/bin/php' 'artisan' inspire > '/dev/null' 2>&1
laravel-scheduler | Running scheduled command: '/usr/local/bin/php' 'artisan' inspire > '/dev/null' 2>&1
laravel-scheduler | Running scheduled command: '/usr/local/bin/php' 'artisan' inspire > '/dev/null' 2>&1

Another thing to note about running a scheduler in Docker: as of Laravel 5.6 you can run the onOneServer() command which indicates that the command only runs on one server. You need to use the Memcached or Redis cache driver to support this feature.

Queue Worker

The last thing we’ll update our Image to support is running a queue worker. We’re going to use Redis, so the predis/predis composer dependency is needed. Run this command locally to install predis:

composer require predis/predis

Next, update the docker/start.sh script to run a queue worker if the $role=queue:

if [ "$role" = "app" ]; then
 
exec apache2-foreground
 
elif [ "$role" = "queue" ]; then
 
echo "Running the queue..."
php /var/www/html/artisan queue:work --verbose --tries=3 --timeout=90
 
elif [ "$role" = "scheduler" ]; then
# ...

Here’s the final start script file in full:

#!/usr/bin/env bash
 
set -e
 
role=${CONTAINER_ROLE:-app}
env=${APP_ENV:-production}
 
if [ "$env" != "local" ]; then
echo "Caching configuration..."
(cd /var/www/html && php artisan config:cache && php artisan route:cache && php artisan view:cache)
fi
 
if [ "$role" = "app" ]; then
 
exec apache2-foreground
 
elif [ "$role" = "queue" ]; then
 
echo "Running the queue..."
php /var/www/html/artisan queue:work --verbose --tries=3 --timeout=90
 
elif [ "$role" = "scheduler" ]; then
 
while [ true ]
do
php /var/www/html/artisan schedule:run --verbose --no-interaction &
sleep 60
done
 
else
echo "Could not match the container role \"$role\""
exit 1
fi

If you create a test queue job and dispatch it (you could dispatch it from within the welcome route for example), you will see something like this output in your docker-compose logs during queue processing:

laravel-queue | [2018-04-25 06:45:59][qv5tfCgvUsKxVTTvq0SsF6gx9VLwnd6H] Processing: App\Jobs\ExampleJob
laravel-queue | [2018-04-25 06:45:59][qv5tfCgvUsKxVTTvq0SsF6gx9VLwnd6H] Processed: App\Jobs\ExampleJob

Learn More

If you want to learn more about developing with Docker and PHP, including Laravel, check out my book Docker for PHP Developers. If you don’t want the starter Laravel code, you can also get the book only version. Laravel News readers get a discount on both versions of the book! You can learn more about the book here.

To learn more about running custom start scripts, I would suggest that you read more about the CMD Dockerfile instruction from the official Docker reference on how it works.

The offical Docker PHP image has some excellent documentation, and I suggest you read through the documentation to learn how quickly you can use the PHP image with your Docker projects. We used Apache, but you can use PHP-FPM in a variety of different image types (i.e., Alpine, Debian Stretch, Debian Jesse).


The links included are affiliate links which means if you decide to buy Laravel News gets a little kickback to help run this site.

Paul Redmond photo

Staff writer at Laravel News. Full stack web developer and author.

Cube

Laravel Newsletter

Join 40k+ other developers and never miss out on new tips, tutorials, and more.

image
No Compromises

Join the Mastering Laravel community to level up your skills and get trusted advice

Visit No Compromises
Laravel Forge logo

Laravel Forge

Easily create and manage your servers and deploy your Laravel applications in seconds.

Laravel Forge
Tinkerwell logo

Tinkerwell

The must-have code runner for Laravel developers. Tinker with AI, autocompletion and instant feedback on local and production environments.

Tinkerwell
No Compromises logo

No Compromises

Joel and Aaron, the two seasoned devs from the No Compromises podcast, are now available to hire for your Laravel project. ⬧ Flat rate of $7500/mo. ⬧ No lengthy sales process. ⬧ No contracts. ⬧ 100% money back guarantee.

No Compromises
Kirschbaum logo

Kirschbaum

Providing innovation and stability to ensure your web application succeeds.

Kirschbaum
Shift logo

Shift

Running an old Laravel version? Instant, automated Laravel upgrades and code modernization to keep your applications fresh.

Shift
Bacancy logo

Bacancy

Supercharge your project with a seasoned Laravel developer with 4-6 years of experience for just $2500/month. Get 160 hours of dedicated expertise & a risk-free 15-day trial. Schedule a call now!

Bacancy
Lucky Media logo

Lucky Media

Bespoke software solutions built for your business. We ♥ Laravel

Lucky Media
Lunar: Laravel E-Commerce logo

Lunar: Laravel E-Commerce

E-Commerce for Laravel. An open-source package that brings the power of modern headless e-commerce functionality to Laravel.

Lunar: Laravel E-Commerce
LaraJobs logo

LaraJobs

The official Laravel job board

LaraJobs
Larafast: Laravel SaaS Starter Kit logo

Larafast: Laravel SaaS Starter Kit

Larafast is a Laravel SaaS Starter Kit with ready-to-go features for Payments, Auth, Admin, Blog, SEO, and beautiful themes. Available with Vue and Livewire stacks.

Larafast: Laravel SaaS Starter Kit
SaaSykit: Laravel SaaS Starter Kit logo

SaaSykit: Laravel SaaS Starter Kit

SaaSykit is a Laravel SaaS Starter Kit that comes with all features required to run a modern SaaS. Payments, Beautiful Checkout, Admin Panel, User dashboard, Auth, Ready Components, Stats, Blog, Docs and more.

SaaSykit: Laravel SaaS Starter Kit
Rector logo

Rector

Your partner for seamless Laravel upgrades, cutting costs, and accelerating innovation for successful companies

Rector

The latest

View all →
Asserting a JSON Response Structure in Laravel image

Asserting a JSON Response Structure in Laravel

Read article
Backpack turns 8 years old, celebrates with 40% discount image

Backpack turns 8 years old, celebrates with 40% discount

Read article
Create a DateTime from a Timestamp With this New Method Coming to PHP 8.4 image

Create a DateTime from a Timestamp With this New Method Coming to PHP 8.4

Read article
Neovim Plugin to for Navigating Laravel and Livewire Components image

Neovim Plugin to for Navigating Laravel and Livewire Components

Read article
Laravel Herd v1.7 is out with updates to the dump UI image

Laravel Herd v1.7 is out with updates to the dump UI

Read article
Share Error Package for Laravel's New Exception Page image

Share Error Package for Laravel's New Exception Page

Read article