Token-Optimized Object Notation encoder/decoder for Laravel with intelligent nested object handling

#Laravel TOON

Token-Optimized Object Notation encoder/decoder for Laravel with intelligent nested object handling.

TOON is a compact, YAML-like format designed to reduce token usage when sending data to LLMs. This package achieves 40-60% token reduction compared to JSON while maintaining full round-trip fidelity.

#Installation

composer require mischasigtermans/laravel-toon

#Quick Start

use MischaSigtermans\Toon\Facades\Toon;
 
$data = [
    'users' => [
        ['id' => 1, 'name' => 'Alice', 'role' => ['id' => 'admin', 'level' => 10]],
        ['id' => 2, 'name' => 'Bob', 'role' => ['id' => 'user', 'level' => 1]],
    ],
];
 
// Encode to TOON
$toon = Toon::encode($data);
 
// Decode back to array
$original = Toon::decode($toon);

Output:

users:
  items[2]{id,name,role.id,role.level}:
    1,Alice,admin,10
    2,Bob,user,1

#Why TOON?

When building MCP servers or LLM-powered applications, every token counts. JSON's verbosity wastes context window space with repeated keys and structural characters.

JSON (398 bytes):

{"orders":[{"id":"ord_1","status":"shipped","customer":{"id":"cust_1","name":"Alice"},"total":99.99},{"id":"ord_2","status":"pending","customer":{"id":"cust_2","name":"Bob"},"total":149.50}]}

TOON (186 bytes) - 53% smaller:

orders:
  items[2]{id,status,customer.id,customer.name,total}:
    ord_1,shipped,cust_1,Alice,99.99
    ord_2,pending,cust_2,Bob,149.5

#Benchmarks

Real-world benchmarks from a production application with 17,000+ records:

#Token Impact

For a typical paginated API response (50 records):

• JSON: ~3,274 tokens
• TOON: ~1,279 tokens
• Saved: ~2,000 tokens per request

#Features

#Nested Object Flattening

The key differentiator. Arrays containing objects with nested properties are automatically flattened using dot notation:

$data = [
    ['id' => 1, 'author' => ['name' => 'Jane', 'email' => 'jane@example.com']],
    ['id' => 2, 'author' => ['name' => 'John', 'email' => 'john@example.com']],
];
 
$toon = Toon::encode($data);
// items[2]{id,author.name,author.email}:
//   1,Jane,jane@example.com
//   2,John,john@example.com
 
$decoded = Toon::decode($toon);
// Returns original nested structure

#Multi-Level Nesting

Handles deeply nested structures:

$data = [
    [
        'id' => 1,
        'product' => [
            'name' => 'Widget',
            'category' => ['id' => 'cat_1', 'name' => 'Electronics'],
        ],
    ],
];
 
$toon = Toon::encode($data);
// items[1]{id,product.name,product.category.id,product.category.name}:
//   1,Widget,cat_1,Electronics

#Type Preservation

All scalar types are preserved through encode/decode:

$data = [
    'count' => 42,
    'price' => 19.99,
    'active' => true,
    'deleted' => false,
    'notes' => null,
];
 
$decoded = Toon::decode(Toon::encode($data));
// Types are preserved: int, float, bool, null

#Special Character Escaping

Commas, colons, and newlines in values are automatically escaped:

$data = ['message' => 'Hello, World: How are you?'];
$toon = Toon::encode($data);
// message: Hello\, World\: How are you?

#Configuration

Publish the config file:

php artisan vendor:publish --tag=toon-config

#Basic Options

// config/toon.php
return [
    // Arrays with fewer items use regular object format instead of tables
    'min_rows_for_table' => 2,
 
    // How deep to flatten nested objects (deeper = JSON string)
    'max_flatten_depth' => 3,
 
    // Escape style for special characters (comma, colon, newline)
    'escape_style' => 'backslash',
];

#Token-Saving Options

return [
    // Omit values to save tokens: 'null', 'empty', 'false', or 'all'
    'omit' => ['null', 'empty'],
 
    // Always skip these keys
    'omit_keys' => ['created_at', 'updated_at'],
 
    // Shorten verbose keys
    'key_aliases' => [
        'description' => 'desc',
        'organization_id' => 'org_id',
    ],
];

#Value Transformation

return [
    // Format dates (DateTime objects and ISO strings)
    'date_format' => 'Y-m-d',
 
    // Truncate long strings (adds ... suffix)
    'truncate_strings' => 100,
 
    // Limit decimal places for floats
    'number_precision' => 2,
];

#Utility Methods

#Measure Savings

$data = User::with('roles')->get()->toArray();
 
$diff = Toon::diff($data);
// [
//     'json_chars' => 12500,
//     'toon_chars' => 5200,
//     'saved_chars' => 7300,
//     'savings_percent' => 58.4,
// ]

#Encode Specific Keys Only

$users = User::all()->toArray();
 
// Only include id and name, exclude email, password, etc.
$toon = Toon::only($users, ['id', 'name']);

#Use Cases

#MCP Servers

Reduce token usage when returning data from MCP tool calls:

public function handle(): string
{
    $users = User::with('roles')->limit(100)->get();
 
    return Toon::encode([
        'count' => $users->count(),
        'users' => $users->toArray(),
    ]);
}

#LLM Context

Pack more data into your context window:

$context = Toon::encode([
    'conversation' => $messages,
    'user_profile' => $user->toArray(),
    'recent_orders' => $orders->toArray(),
]);
 
$response = $llm->chat([
    ['role' => 'system', 'content' => "Context:\n{$context}"],
    ['role' => 'user', 'content' => $question],
]);

#API Responses

Optional TOON responses for token-conscious clients:

public function index(Request $request)
{
    $data = Product::paginate()->toArray();
 
    if ($request->header('Accept') === 'application/toon') {
        return response(Toon::encode($data))
            ->header('Content-Type', 'application/toon');
    }
 
    return response()->json($data);
}

#Testing

composer test

#License

MIT