CPS Compilation

Why Pipeline Compiles to Continuation-Passing Style

When the controller shuts down, its threads die and their stacks, with every local variable and return address, vanish. To survive, a Pipeline's position and variables must live somewhere that can be written to disk. The Pipeline: Groovy plugin (workflow-cps, version 4383.v04fa_a_3d67b_d9 here) achieves this by compiling the Jenkinsfile into continuation-passing style (CPS): every method call, loop and assignment is rewritten into objects that say "do this, then continue with that." An interpreter, the groovy-cps library that CloudBees 21,188 wrote and later moved into the plugin's repository, walks those objects one small step at a time.

The same program as a thread stack and as heap objects the CPS interpreter can serialize
The same program as a thread stack and as heap objects the CPS interpreter can serialize

The pay-off is that, whenever the program is between steps (waiting for sh, input, sleep or an executor), its entire state is a graph of ordinary objects that can be serialized, and deserialized after a restart. The price is paid in three places the rest of this section demonstrates: everything in scope must be serializable, some Groovy idioms behave differently under the transformation, and the interpreter is slow.