Testing Long-Running PHP Code with Generators

Sometimes, I am writing some longer-running process logic in PHP - often in the style of processing loops. This usually leads to a quite-long function which packs quite some functionality in it, but is also quite readable.

An example of such a processing loop is implemented in the new Flowpack.DecoupledContentStore - a two-stack CMS package for Neos we are currently developing. Inside, there is the NodeRenderOrchestrator whose job is to figure out what needs to be done in order to render all pages; and then check whether all pages were rendered successfully. An example of this control loop is roughly seen below:

function orchestrateRendering() {
  // try at most 10 major iterations
  for ($i = 0; $i < 10; $i++) {
 
    // find out what needs to be done in order to converge to a known-good state
    $pagesToRender = findUnrenderedPages();
 
    if (count($pagesToRender) === 0) {
      // we are finished!
      exit(0);
    }
    addRenderingJobsToJobQueue($pagesToRender);
 
    while (!allJobsProcessed()) {
      sleep(1);
    }
  }
}

This code has several difficulties which makes it hard to test:

It is running for quite a long time, and only terminates when everything is done.
The code has an exit() call inside, which would also terminate all testcases.
In line 14, it assumes that there are other workers which run in parallel to this code - which is hard to simulate in a single-threaded PHP environment.

Additionally, to provoke certain error scenarios, we often want to run the code only up to a certain position, then provoke an error, and then let it continue running and see if it deals with the error in a good way.

Initial Idea: Extract Some Functions

In a first quest to make this code better testable, it felt useful to extract some functions for some of the behavior above. It turns out to be rather difficult, because much of the logic is encoded in the outer-loop; so in the way how the different code sections above interact on a high level. Additionally, we needed to use Reflection to call these inner methods; or alternatively override quite some inner methods.

These solution ideas have a big drawback: We need to tear apart the control loop, just for the sake of testing.

So, what are our other options? Somehow, it would be good to say something like: "Please run until a certain point in time, then interrupt the execution. After I have done some stuff in my testcase, resume the execution at exactly this point in time."

Generators to the Rescue

It turns out this suspending and resuming behavior is possible in PHP (and other languages supporting Generators) using the Yield keyword. Let's see in a quick example how generators work:

<?php
function gen(): \Generator {
    echo "Gen: start\n";
    yield 1;
    echo "Gen: after yield 1\n";
    yield 2;
    echo "Gen: after yield 2\n";
}
 
echo "outer: begin\n";
$myGenerator = gen();
echo "outer: after initial function invocation\n";
$res = $myGenerator->current();
echo "Result: $res\n\n";
 
$myGenerator->next();
echo "outer: after next()\n";
$res = $myGenerator->current();
echo "Result: $res\n\n";
 
$myGenerator->next();
echo "outer: after next()\n";
$res = $myGenerator->current();
echo "Result: $res\n\n";

DISPLAYED OUTPUT:
 
outer: begin
outer: after initial function invocation
Gen: start
Result: 1
 
Gen: after yield 1
outer: after next()
Result: 2
 
Gen: after yield 2
outer: after next()
Result:

Take a moment to digest what you are seeing here:

When gen() is executed and the Generator is returned, the function is only executed up to (and including) the first yield statement. Then, the function invocation is suspended at the yield position (line 4).
The currently yielded value can be accessed using $myGenerator->current().
You can advance the generator (resume the suspended function) by calling $myGenerator->next(). This then invokes the suspended gen() function up to the next yield statement.

If this sounds like magic, it somehow is :-) Generators need to be supported by the language runtime itself, because they need a way to suspend a function and keeping the function's stack; so that it can be resumed at a later point in time.

Generators for Testing Control Loops

Now, back to our original question. We can emit some events from our processing logic, and we need an outside runtime to handle these events for us. Pretty much like this:

function orchestrateRendering(): \Generator {
  // try at most 10 major iterations
  for ($i = 0; $i < 10; $i++) {
    yield new BeginNextIterationEvent();
 
    // find out what needs to be done in order to converge to a known-good state
    $pagesToRender = findUnrenderedPages();
 
    if (count($pagesToRender) === 0) {
      // we are finished!
      yield new ExitEvent(0);
      return;
    }
    addRenderingJobsToJobQueue($pagesToRender);
 
    yield new RenderingQueueFilledEvent();
 
    while (!allJobsProcessed()) {
      sleep(1);
    }
  }
}

Now, our runtime to execute the generator can look like this:

In the simple (production) case, we pull all values (=Events) from the generator in a loop, as fast as possible.
In the testing case, we pull all values (=Events) from the generator in a loop until we hit our "breakpoint". This way, we can interrupt processing code at the exact position where we need it, then do our adjustments, and then resume it again.

public function run(\Generator $generator): void
{
    while ($generator->valid()) {
        $currentEvent = $generator->current();
        if ($currentEvent instanceof ExitEvent) {
            exit($currentEvent->getExitCode());
        }
        // try to read next event
        $generator->next();
    }
}
 
public function runUntilEventEncounteredForTesting(\Generator $generator, string $eventClassName): void
{
    while ($generator->valid()) {
        $currentEvent = $generator->current();
        if ($currentEvent instanceof ExitEvent) {
            // we do not exit here
            // stop iterating the iterator in all cases
            return;
        }
        if (is_a($currentEvent, $eventClassName)) {
            // stop here, can be continued lateron.
            return;
        }
        // try to read next event (if not stopped)
        $this->generator->next();
    }
}

Closing Thoughts

At first I was unsure whether this approach is a good one, because I have not seen it beforehand in practice. However, after using it for some days now, it still feels quite elegant to me - so I am quite happy with it so far. If you have further feedback, please let me know on twitter @skurfuerst :-)

You can check out the full source code of our long-running processes (part of Flowpack.DecoupledContentStore) inside NodeRenderOrchestrator.php and NodeRenderer.php - and the runtime to execute and interrupt these processes can be found in InterruptibleProcessRuntime.php.