Warm workers¶

The problem¶

After test selection, a mutant costs about 330 milliseconds. Only about 15 of those milliseconds are testing.

Roughly 95 percent of a run is setup that produces no information. No amount of smarter scheduling helps, because the cost is paid before scheduling begins.

How other tools avoid it¶

fork() clones a running process almost instantly, and the clone inherits everything the parent had already imported. A mutation tester can therefore import pytest once, then fork a fresh warm copy per mutant. Each copy starts hot and dies with its mess contained.

This is why mutmut is fast. It is also why mutmut refuses to run on Windows at all, telling you to use WSL, because Windows has no fork().

A rejected alternative¶

Python 3.14 added concurrent.interpreters through PEP 734. Subinterpreters are isolated, cheap, and work on Windows, which makes them an obvious candidate.

They were measured rather than assumed:

There is no saving. Isolation is precisely the point of a subinterpreter: each one gets its own sys.modules and re-executes every import. The value of fork() is inheriting imports, and PEP 734 deliberately does not do that.

This result is recorded so nobody repeats the experiment.

What Angelo does instead¶

Keep the process. Reset only the part that actually changed.

sequenceDiagram
    participant A as angelo
    participant P as pytest process
    A->>P: start once
    Note over P: import pytest, 170ms, paid once
    loop each mutant
        A->>A: patch the file
        A->>P: which tests to run
        Note over P: forget the project's own modules
        Note over P: run them
        P-->>A: exit code and failures
    end
    A->>P: kill when finished

Between mutants the worker deletes every module whose file lives inside the project copy. The mutated source is therefore re-imported on the next run, while pytest, the standard library and site-packages stay loaded. That is the entire saving.

Keeping it safe¶

A long lived process accumulating state is exactly the kind of thing that produces wrong verdicts, so three guards apply.

Any anomaly retires the worker. A timeout, a crash, or a reply Angelo cannot parse means the process is killed and the mutant is re-run in a fresh subprocess. Warm running can therefore only change the clock.

The process recycles. After warm_recycle_after runs, default 50, it is replaced. This bounds anything that survives the module purge.

It only applies to plain pytest commands. Anything else uses subprocesses.

Result¶

Two hundred mutants, eight workers.

Warm workers pay most where runs are many and each is cheap, which is the mirror image of test selection. Together with batching, the full stack takes a 48.1 second job down to 4.5 seconds.

Two bugs this cost¶

Both were silent, and both were caught by tests rather than by reading the code.

Purging __main__ broke pytest. The worker script itself is __main__, and it lives inside the project copy, so the module purge deleted it. Python's debugger imports __main__, so every mutant came back as an error.

pytest's output collided with the protocol. Both were writing to the same stdout, so the reader parsed "1 passed" as a reply. Replies now carry a marker prefix.

Limits¶

• Only for python -m pytest style commands.
• State can outlive a purge. C extension globals and third party caches holding project objects are the realistic cases. Recycling bounds this; it does not remove it.
• A hanging mutant costs a worker restart rather than just a process kill.
• If a score ever drifts between runs, set warm_workers = false first. This is the same risk class as the stale bytecode bug in benchmarks.