Custom Route Files

One morning I woke up to Slack notifications. That's never a good sign.

Overnight, my Redis instance had filled completely up. We use Redis for two things:

1. Session storage
2. Caching a few bits of data, nothing substantial

I used TablePlus to see what I could in Redis. It's a bit hard to tell what's going on, since Laravel uses random hashes as part of the cache keys, and the payloads are encoded/encrypted.

However I could see that there were 2 Redis databases (db0 and db1). Checking the config/databases.php file, I found that 2 corresponding databases were indeed defined for Redis.

# File config/databases.php
return [
    // Things ommitted here...
 
 
    /*
    |--------------------------------------------------------------------------
    | Redis Databases
    |--------------------------------------------------------------------------
    |
    | Redis is an open source, fast, and advanced key-value store that also
    | provides a richer body of commands than a typical key-value system
    | such as APC or Memcached. Laravel makes it easy to dig right in.
    |
    */
 
    'redis' => [
 
        'client' => env('REDIS_CLIENT', 'phpredis'),
 
        'options' => [
            'cluster' => env('REDIS_CLUSTER', 'redis'),
            'prefix' => env('REDIS_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_database_'),
        ],
 
        'default' => [
            'url' => env('REDIS_URL'),
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_PORT', '6379'),
            'database' => env('REDIS_DB', '0'),
        ],
 
        'cache' => [
            'url' => env('REDIS_URL'),
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_PORT', '6379'),
            'database' => env('REDIS_CACHE_DB', '1'),
        ],
 
    ],
];

The default connection uses db0 while the cache connection uses db1. It turns out that sessions are stored in db0, while thing we cache in code uses db1. The default database, db0, had MANY more keys than the cache database.

The application was creating too many sessions.

#What Creates a Session?

Every web request (for routes defined in routes/web.php) creates a session (or uses an existing one). Web applications return a cookie when a session is created. Web browsers store these cookies, and send that cookie back when making additional web requests. This allows our web apps to know which session is valid for a given user.

If the browser didn't return a cookie on each request, then the user would not be able to stay logged in.

API-based sessions don't work like this. Each session is created and then destroyed within every web request - there are no cookies involved. Instead, the client needs to send it's authentication information on each web request (usually a token of some sort).

#What Blew Up Redis?

So, what then caused our Redis instance to blow up with sessions?

Dynamically generated assets that others embed on their websites. We had two cases of this:

1. Our application generated a .js file that others embedded on their web sites
2. Our application also generated .svg images for the same purpose

These routes were defined in our routes/web.php file:

Route::get('/embed.js');
Route::get('/{project}/share.js');

Do you see the issue? Customers were putting these into their own websites. Everytime someone visited their website, an HTTP request was made to our application for the embed or SVG, and this created a session.

That means that our customers web traffic was also creating sessions in our web application!

#How to Reduce Session Creation

The fix is that make sure that we don't create sessions for certain routes. Simple enough to say, but how to we accomplish that?

It turns out the creation of cookies and sessions are done in Laravel's middleware. This is good, as we control which middleware are applied to each routes.

To ensure some routes don't create sessions/return cookies, I like to create a separate routes file that has a different middleware stack.

To do that, we need to do a few things:

1. Create a new routes/static.php file (the name is arbitrary)
2. Add a middleware stack to app/Http/Kernel.php
3. Update app/Providers/RouteServiceProvider.php to load our new route file, and apply our new middleware stack

The new routes file is simple - we make a new file and move our route definitions to them:

# File routes/static.php`
 
# Move these from routes/web.php
Route::get('/embed.js');
Route::get('/{project}/share.js');

Then we can update the Kernel.php file to create a new middleware stack. We can copy the web middleware stack and remove the middleware that handle cookies and sessions:

 # File app/Http/Kernel.php
  
 # Items omitted here
  
     /**
      * The application's route middleware groups.
      *
      * @var array
      */
     protected $middlewareGroups = [
         'web' => [
             \App\Http\Middleware\EncryptCookies::class,
             \Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse::class,
             \Illuminate\Session\Middleware\StartSession::class,
             // \Illuminate\Session\Middleware\AuthenticateSession::class,
             \Illuminate\View\Middleware\ShareErrorsFromSession::class,
             \App\Http\Middleware\VerifyCsrfToken::class,
             \Illuminate\Routing\Middleware\SubstituteBindings::class,
         ],
  
         'api' => [
             // \Laravel\Sanctum\Http\Middleware\EnsureFrontendRequestsAreStateful::class,
             'throttle:api',
             \Illuminate\Routing\Middleware\SubstituteBindings::class,
         ],
+ 
+        'static' => [
+            \Illuminate\Routing\Middleware\SubstituteBindings::class,
+        ], 
     ];
  
 # Items omitted here

We created a new middleware group named static. It's similar to the API middleware, but we have no throttling.

Lastly, we need to register the new routes file, and apply our new static middleware group. We'll do that by updating the RouteServiceProvider.php:

 # File app/Providers/RouteServiceProvider.php
  
 # Items omitted here
     /**
      * Define your route model bindings, pattern filters, etc.
      *
      * @return void
      */
     public function boot()
     {
         $this->configureRateLimiting();
  
         $this->routes(function () {
             Route::prefix('api')
                 ->middleware('api')
                 ->namespace($this->namespace)
                 ->group(base_path('routes/api.php'));
  
             Route::middleware('web')
                 ->namespace($this->namespace)
                 ->group(base_path('routes/web.php'));
+ 
+            Route::middleware('static')
+                ->namespace($this->namespace)
+                ->group(base_path('routes/static.php'));
+        }); 
     }
  
 # Items omitted here

The RouteServiveProvider registers each route file, and determines their middleware. This is how everyting in routes/web.php gets the web middleware group assigned to it.

That's also why we create our own route file - we wanted to avoid the web middleware group, and be able to add routes to the new routes file whenever we needed to.

#The Result

The result is that our two "static" routes (ones returning dynamically generated assets - a JS file, and an SVG) no longer create a session, nor return cookies.

This allowed our Redis instance to recover. As the sessions expired, they were deleted from Redis. Since our customers traffic no longer created sessions in our session store, the Redis instance never filled up again!