Try Live
Add Docs
Rankings
Pricing
Docs
Install
Install
Docs
Pricing
More...
More...
Try Live
Rankings
Enterprise
Create API Key
Add Docs
JsonMachine
https://github.com/halaxa/json-machine
Admin
A memory-efficient PHP library for parsing large JSON files or streams incrementally, offering
...
Tokens:
8,683
Snippets:
37
Trust Score:
7.6
Update:
5 months ago
Context
Skills
Chat
Benchmark
90.1
Suggestions
Latest
Show doc for...
Code
Info
Show Results
Context Summary (auto-generated)
Raw
Copy
Link
# JSON Machine JSON Machine is an efficient, easy-to-use PHP library for parsing extremely large JSON files and streams with constant memory footprint. The library solves the common problem of memory exhaustion when using `json_decode()` on large JSON documents by implementing a streaming parser that processes JSON incrementally, loading only one item at a time. Built on PHP generators, it provides a simple foreach-based API that works seamlessly with files, streams, and HTTP responses, making it ideal for processing multi-gigabyte JSON documents without memory constraints. The library supports advanced features including JSON Pointer navigation for parsing specific subtrees, recursive iteration for deeply nested structures, multiple decoder options, custom error handling, and integration with popular HTTP clients like GuzzleHttp and Symfony HttpClient. With O(2) memory complexity for streams and thorough test coverage, JSON Machine delivers production-ready performance for data processing, ETL pipelines, API consumption, and any scenario requiring efficient handling of large-scale JSON data. ## Parsing JSON Files with Items::fromFile() Parse large JSON files with constant memory usage by iterating items one at a time. ```php <?php use JsonMachine\Items; // Parse a large JSON file - only loads one item at a time $users = Items::fromFile('500MB-users.json'); foreach ($users as $id => $user) { // Process each user individually echo "User: {$user->name}, Email: {$user->email}\n"; } // Get arrays instead of objects using ExtJsonDecoder use JsonMachine\JsonDecoder\ExtJsonDecoder; $data = Items::fromFile('data.json', ['decoder' => new ExtJsonDecoder(true)]); foreach ($data as $key => $item) { // $item is now an associative array print_r($item); } ``` ## Parsing JSON Strings with Items::fromString() Process JSON strings already in memory with reduced memory footprint compared to json_decode(). ```php <?php use JsonMachine\Items; $jsonString = '{"apple": {"color": "red"}, "pear": {"color": "yellow"}}'; $fruits = Items::fromString($jsonString); foreach ($fruits as $name => $data) { echo "{$name}: {$data->color}\n"; } // Output: // apple: red // pear: yellow ``` ## Parsing Subtrees with JSON Pointer Target specific parts of a JSON document using JSON Pointer (RFC 6901) to iterate only the data you need. ```php <?php use JsonMachine\Items; // JSON: {"results": {"apple": {"color": "red"}, "pear": {"color": "yellow"}}} $fruits = Items::fromFile('fruits.json', ['pointer' => '/results']); foreach ($fruits as $name => $data) { echo "{$name}: {$data->color}\n"; } // Parse nested array values with wildcard // JSON: {"results": [{"name": "apple", "color": "red"}, {"name": "pear", "color": "yellow"}]} $colors = Items::fromFile('fruitsArray.json', ['pointer' => '/results/-/color']); foreach ($colors as $key => $value) { echo "Color: {$value}\n"; // Access current JSON pointer echo "Path: {$colors->getCurrentJsonPointer()}\n"; } // Output: // Color: red // Path: /results/0/color // Color: yellow // Path: /results/1/color ``` ## Parsing Multiple Subtrees Iterate over multiple document sections efficiently by specifying multiple JSON Pointers. ```php <?php use JsonMachine\Items; // JSON with berries and citruses arrays $fruits = Items::fromFile('fruits.json', [ 'pointer' => ['/berries', '/citruses'] ]); foreach ($fruits as $key => $value) { echo "Fruit: {$value['name']}, Color: {$value['color']}\n"; echo "Section: {$fruits->getCurrentJsonPointer()}\n"; } // Items are iterated in document order regardless of pointer order ``` ## Parsing Streams with Items::fromStream() Process streaming JSON data from network responses or other stream resources. ```php <?php use JsonMachine\Items; // Parse a stream resource $stream = fopen('https://api.example.com/data.json', 'r'); $items = Items::fromStream($stream); foreach ($items as $key => $item) { // Process streaming data as it arrives processItem($item); } fclose($stream); ``` ## GuzzleHttp Integration Parse JSON responses from GuzzleHttp without loading entire response into memory. ```php <?php use JsonMachine\Items; use GuzzleHttp\Client; use GuzzleHttp\Psr7\StreamWrapper; $client = new Client(); $response = $client->request('GET', 'https://api.example.com/large-dataset'); // Convert Guzzle PSR-7 stream to PHP stream resource $phpStream = StreamWrapper::getResource($response->getBody()); $items = Items::fromStream($phpStream, ['pointer' => '/data']); foreach ($items as $key => $value) { // Process each item from the API response echo "Processing item {$key}\n"; } ``` ## Symfony HttpClient Integration Parse streaming responses from Symfony HttpClient with native iterator support. ```php <?php use JsonMachine\Items; use Symfony\Component\HttpClient\HttpClient; function httpClientChunks($responseStream) { foreach ($responseStream as $chunk) { yield $chunk->getContent(); } } $client = HttpClient::create(); $response = $client->request('GET', 'https://api.example.com/data'); $jsonChunks = httpClientChunks($client->stream($response)); $items = Items::fromIterable($jsonChunks, ['pointer' => '/results']); foreach ($items as $key => $value) { // Process streaming API response var_dump($key, $value); } ``` ## Recursive Iteration with RecursiveItems Handle deeply nested JSON structures without loading entire objects into memory. ```php <?php use JsonMachine\RecursiveItems; // JSON: [{"username": "user1", "friends": [{"username": "friend1"}, {"username": "friend2"}]}] $users = RecursiveItems::fromFile('users.json'); foreach ($users as $user) { // $user is a RecursiveItems instance foreach ($user as $field => $value) { if ($field === 'friends') { // $value is also RecursiveItems foreach ($value as $friend) { // Convert to array when manageable $friendArray = $friend->toArray(); echo "Friend: {$friendArray['username']}\n"; } } elseif ($field === 'username') { echo "User: {$value}\n"; } } } ``` ## RecursiveItems Array Access and advanceToKey() Use array-like syntax and key navigation methods for convenient access to nested data. ```php <?php use JsonMachine\RecursiveItems; $users = RecursiveItems::fromFile('users.json'); foreach ($users as $user) { // Array-like access (syntax sugar for advanceToKey) $friends = $user['friends']; // equivalent to $user->advanceToKey('friends') foreach ($friends as $friend) { $friendData = $friend->toArray(); echo $friendData['username'] . "\n"; } } // Chain array access $users = RecursiveItems::fromFile('users.json'); $secondFriendUsername = $users[0]['friends'][1]['username']; echo $secondFriendUsername; // 'friend2' ``` ## Error Handling with ErrorWrappingDecoder Skip malformed JSON items instead of terminating on syntax errors. ```php <?php use JsonMachine\Items; use JsonMachine\JsonDecoder\DecodingError; use JsonMachine\JsonDecoder\ErrorWrappingDecoder; use JsonMachine\JsonDecoder\ExtJsonDecoder; $items = Items::fromFile('data-with-errors.json', [ 'decoder' => new ErrorWrappingDecoder(new ExtJsonDecoder()) ]); foreach ($items as $key => $item) { // Check if key or item is a decoding error if ($key instanceof DecodingError) { echo "Error in key: {$key->getErrorMessage()}\n"; echo "Raw JSON: {$key->getJsonValue()}\n"; continue; } if ($item instanceof DecodingError) { echo "Error in item {$key}: {$item->getErrorMessage()}\n"; echo "Raw JSON: {$item->getJsonValue()}\n"; continue; } // Process valid item processItem($key, $item); } ``` ## PassThruDecoder for Custom Parsing Receive raw JSON strings without automatic decoding for custom processing. ```php <?php use JsonMachine\Items; use JsonMachine\JsonDecoder\PassThruDecoder; $items = Items::fromFile('data.json', ['decoder' => new PassThruDecoder()]); foreach ($items as $key => $jsonString) { // $key and $jsonString are raw JSON strings // Parse with custom logic or external libraries $decoded = customJsonParser($jsonString); processItem($decoded); } ``` ## Tracking Parsing Progress Monitor progress through large JSON documents with debug mode enabled. ```php <?php use JsonMachine\Items; $fileSize = filesize('huge-file.json'); $items = Items::fromFile('huge-file.json', ['debug' => true]); foreach ($items as $name => $data) { $position = $items->getPosition(); $progress = intval($position / $fileSize * 100); echo "Progress: {$progress}% ({$position} / {$fileSize} bytes)\n"; // Process item processData($data); } // Note: getPosition() returns 0 when debug is disabled for performance ``` ## Parsing Single Scalar Values Extract individual scalar values from anywhere in a JSON document efficiently. ```php <?php use JsonMachine\Items; // JSON: {"lastModified": "2012-12-12", "data": {...large data...}} $items = Items::fromFile('metadata.json', ['pointer' => '/lastModified']); foreach ($items as $key => $value) { // Parser stops after finding the value - very efficient echo "Last modified: {$value}\n"; // "2012-12-12" break; } // Shortcut to get single value $items = Items::fromFile('metadata.json', ['pointer' => '/lastModified']); $lastModified = iterator_to_array($items)['lastModified']; ``` ## Custom Iterator from Iterable Parse JSON from any iterable that produces JSON chunks. ```php <?php use JsonMachine\Items; // Custom generator that yields JSON chunks function customJsonSource() { yield '{"users": ['; yield '{"id": 1, "name": "Alice"},'; yield '{"id": 2, "name": "Bob"}'; yield ']}'; } $items = Items::fromIterable(customJsonSource(), ['pointer' => '/users']); foreach ($items as $user) { echo "User {$user->id}: {$user->name}\n"; } // Output: // User 1: Alice // User 2: Bob ``` ## JSON Machine is a powerful tool for processing JSON data at scale, enabling developers to work with massive JSON documents, streaming APIs, and data pipelines without memory constraints. The library's primary use cases include ETL operations on large datasets, consuming paginated API responses, processing log files, importing/exporting data from databases, real-time data processing from webhooks, and analyzing multi-gigabyte JSON archives. By replacing `json_decode(file_get_contents())` with `Items::fromFile()`, applications immediately gain the ability to process JSON of any size with predictable, minimal memory usage. Integration is straightforward across various scenarios: use `Items::fromFile()` for local files, `Items::fromStream()` for network streams and API responses, `Items::fromString()` when JSON is already in memory, and `RecursiveItems` when individual items are too large to load completely. The library works seamlessly with modern HTTP clients through simple adapters, supports flexible JSON Pointer-based navigation to target specific data sections, and provides robust error handling to skip malformed items while continuing processing. With no production dependencies beyond optional ext-json, JSON Machine delivers production-grade performance with a simple, intuitive API that feels natural to PHP developers.