Lazy JSON Pages is a framework-agnostic API scraper that loads items from any paginated JSON API into a Laravel lazy collection. It is highly customizable, so you can make it work with any PHP project and any JSON API that has paginated results:
use Illuminate\Support\LazyCollection; $result = LazyCollection::fromJsonPages($source) ->totalPages('pagination.total_pages') ->async(requests: 3) ->throttle(requests: 100, perMinutes: 1) ->collect('data.*');Though Lazy JSON pages return data as a Laravel lazy collection, you can use this package with any PHP project as well. Since it uses Guzzle, you can easily customize client behavior via middleware needed for the target API you're consuming.
Main Features
- Framework-agnostic: Works independently of specific frameworks, but integrates smoothly with Laravel.
- Async HTTP Requests: Loads items from paginated JSON APIs using asynchronous HTTP requests.
- Lazy Collection Support: Seamlessly loads API data into Laravel’s lazy collections for efficient memory usage.
- Handles Pagination: Scrapes and processes paginated APIs without requiring explicit pagination management.
- Efficient Data Handling: Allows handling large datasets with minimal memory footprint.
You can learn more about this package, get full installation instructions, and view the source code on GitHub.
