Skip to Content
Course content

113: XMLReader for Streaming XML

Click on the "Edit" button in the top corner of the screen to edit your slide content.

I've seen this happen a dozen times in production: a developer is tasked with importing a product catalog from a third-party supplier. The supplier provides a 500MB XML file. The developer, wanting to be efficient and clean, uses simplexml_load_file(). It works perfectly on their machine with a small test file. Then, they push to production, the script hits a 2GB XML file, and the server immediately throws a "Fatal error: Allowed memory size exhausted."

The Memory Trap of DOM-based Parsing

The problem is that SimpleXML and DOMDocument are "tree-based" parsers. They read the entire XML document and build a complete object representation of that tree in your RAM. If your XML file is 500MB on disk, the memory footprint of the resulting PHP object tree can easily be three or four times that. It's a luxury we can't afford when dealing with "big data" feeds.

// The "Naive" way - Great for small files, lethal for huge ones
$xml = simplexml_load_file('massive_catalog.xml');
foreach ($xml->product as $product) {
    echo $product->name;
}

When you're working with a file that might grow over time, relying on this approach is a ticking time bomb. You can bump your memory_limit in php.ini, but that's just delaying the inevitable. Eventually, the file will be larger than your available RAM.

Streaming with XMLReader

This is where XMLReader comes in. Instead of loading the whole tree, XMLReader acts like a cursor moving forward through the document. It only knows about the node it's currently pointing to. It doesn't care what happened ten nodes ago, and it has no idea what's coming next until it moves there. This means the memory usage remains constant regardless of whether the XML file is 10KB or 10GB.

The trade-off is that you lose the ability to jump around the document. You can't say "give me the third child of the second parent" without manually iterating through the stream to find it. You're essentially reading a tape deck rather than a CD.

$reader = new XMLReader();
$reader->open('massive_catalog.xml');

// Move to the first element
while ($reader->read()) {
    // We only care about the start of an element named 'product'
    if ($reader->nodeType == XMLReader::ELEMENT && $reader->name === 'product') {
        // We've found a product! Now we can extract its attributes or move inside
        echo "Processing product: " . $reader->getAttribute('id') . "\n";
        
        // Move the cursor past this element to keep the loop moving
        $reader->next();
    }
}
$reader->close();

The Hybrid Approach: The Best of Both Worlds

You might be thinking, "Iterating through every single node and checking nodeType feels tedious." I agree. Writing raw XMLReader code for complex nested structures is a chore. However, there's a professional trick: use XMLReader to stream to the right spot, and then use expand()` to turn just that one specific node into a DOMNode.

By doing this, you only ever have one "product" object in memory at a time. You get the convenience of DOM navigation for the individual item, but the memory efficiency of streaming for the overall file.

$reader = new XMLReader();
$reader->open('massive_catalog.xml');

while ($reader->read()) {
    if ($reader->nodeType == XMLReader::ELEMENT && $reader->name === 'product') {
        // Expand the current node into a DOMElement
        $node = $reader->expand();
        
        // Now we can use standard DOM methods on just this small fragment
        $name = $node->getElementsByTagName('name')->item(0)->nodeValue;
        $price = $node->getElementsByTagName('price')->item(0)->nodeValue;
        
        echo "Product: $name costs $price\n";
        
        // The $node object is destroyed in the next iteration, freeing memory
        $reader->next();
    }
}
$reader->close();

This pattern is how I handle almost every large-scale XML import. It separates the "navigation" (XMLReader) from the "data extraction" (DOM), giving you a script that is both maintainable and bulletproof against memory crashes.




📋 Practical Task

Exercise: Calculating High-Value Inventory from a Massive Book Feed

You have been provided with a simulated large XML file named library_export.xml. The file contains thousands of <book> entries, each with a <title> and a <price>.

Write a PHP script that uses XMLReader to stream through the file. Your script must:

  • Iterate through every <book> element without loading the entire file into memory.
  • Use the expand() method to convert the book node into a DOM element.
  • Check if the <price> of the book is greater than 50.00.
  • Maintain a running count of how many "high-value" books exist and print the final total to the console.

Constraint: Do not use simplexml_load_file or DOMDocument::load on the main file, as the goal is to keep memory usage under 2MB regardless of the XML file size.

Rating
0 0

There are no comments for now.

to be the first to leave a comment.