Amazon releases a summary of the recent S3 outage

Published on by

Amazon releases a summary of the recent S3 outage image

Amazon released a summary of the S3 Service Disruption on US-EAST-1 yesterday. The outage affected many sites, apps, and utilities that rely on the service.

An S3 team member running an established playbook to remove a small number of servers accidently entered a command wrong and a larger set of servers was removed.

The servers that were inadvertently removed supported two other S3 subsystems. One of these subsystems, the index subsystem, manages the metadata and location information of all S3 objects in the region. This subsystem is necessary to serve all GET, LIST, PUT, and DELETE requests. The second subsystem, the placement subsystem, manages allocation of new storage and requires the index subsystem to be functioning properly to correctly operate. The placement subsystem is used during PUT requests to allocate storage for new objects. Removing a significant portion of the capacity caused each of these systems to require a full restart. While these were being restarted, S3 was unable to service requests. Other AWS services in the US-EAST-1 Region that rely on S3 for storage, including the S3 console, Amazon Elastic Compute Cloud (EC2) new instance launches, Amazon Elastic Block Store (EBS) volumes (when data was needed from a S3 snapshot), and AWS Lambda were also impacted while the S3 APIs were unavailable.

They go on to say that because of the massive growth over the last several years the process of restarting these services and running the necessary safety checks to validate the integrity of the metadata took longer than expected.

Finally, because the status board relied on S3 they were unable to update the status icons and had to rely on Twitter and the note at the top.

Eric L. Barnes photo

Eric is the creator of Laravel News and has been covering Laravel since 2012.

Filed in:
Cube

Laravel Newsletter

Join 40k+ other developers and never miss out on new tips, tutorials, and more.

Laravel Forge logo

Laravel Forge

Easily create and manage your servers and deploy your Laravel applications in seconds.

Laravel Forge
Tinkerwell logo

Tinkerwell

The must-have code runner for Laravel developers. Tinker with AI, autocompletion and instant feedback on local and production environments.

Tinkerwell
Cut PHP Code Review Time & Bugs into Half with CodeRabbit logo

Cut PHP Code Review Time & Bugs into Half with CodeRabbit

CodeRabbit is an AI-powered code review tool that specializes in PHP and Laravel, running PHPStan and offering automated PR analysis, security checks, and custom review features while remaining free for open-source projects.

Cut PHP Code Review Time & Bugs into Half with CodeRabbit
Join the Mastering Laravel community logo

Join the Mastering Laravel community

Connect with experienced developers in a friendly, noise-free environment. Get insights, share ideas, and find support for your coding challenges. Join us today and elevate your Laravel skills!

Join the Mastering Laravel community
Laravel Idea for PhpStorm logo

Laravel Idea for PhpStorm

Ultimate PhpStorm plugin for Laravel developers, delivering lightning-fast code completion, intelligent navigation, and powerful generation tools to supercharge productivity.

Laravel Idea for PhpStorm
Kirschbaum logo

Kirschbaum

Providing innovation and stability to ensure your web application succeeds.

Kirschbaum
Shift logo

Shift

Running an old Laravel version? Instant, automated Laravel upgrades and code modernization to keep your applications fresh.

Shift
Bacancy logo

Bacancy

Supercharge your project with a seasoned Laravel developer with 4-6 years of experience for just $2500/month. Get 160 hours of dedicated expertise & a risk-free 15-day trial. Schedule a call now!

Bacancy
Lucky Media logo

Lucky Media

Get Lucky Now - the ideal choice for Laravel Development, with over a decade of experience!

Lucky Media
Lunar: Laravel E-Commerce logo

Lunar: Laravel E-Commerce

E-Commerce for Laravel. An open-source package that brings the power of modern headless e-commerce functionality to Laravel.

Lunar: Laravel E-Commerce
LaraJobs logo

LaraJobs

The official Laravel job board

LaraJobs
SaaSykit: Laravel SaaS Starter Kit logo

SaaSykit: Laravel SaaS Starter Kit

SaaSykit is a Multi-tenant Laravel SaaS Starter Kit that comes with all features required to run a modern SaaS. Payments, Beautiful Checkout, Admin Panel, User dashboard, Auth, Ready Components, Stats, Blog, Docs and more.

SaaSykit: Laravel SaaS Starter Kit
MongoDB logo

MongoDB

Enhance your PHP applications with the powerful integration of MongoDB and Laravel, empowering developers to build applications with ease and efficiency. Support transactional, search, analytics and mobile use cases while using the familiar Eloquent APIs. Discover how MongoDB's flexible, modern database can transform your Laravel applications.

MongoDB

The latest

View all →
Enum-Powered Route Permissions in Laravel image

Enum-Powered Route Permissions in Laravel

Read article
NativePHP for iOS image

NativePHP for iOS

Read article
Using Relative Date Helpers in Laravel's Query Builder image

Using Relative Date Helpers in Laravel's Query Builder

Read article
Fine-tuning Pagination Links in Laravel image

Fine-tuning Pagination Links in Laravel

Read article
Relative Date Helpers in Laravel 11.42 image

Relative Date Helpers in Laravel 11.42

Read article
Smart Route Detection in Laravel image

Smart Route Detection in Laravel

Read article