Support Batch Execution Mode

[aglinxinyuan]

Discussed in #4149

^{Originally posted by aglinxinyuan January 8, 2026}
I would like to introduce the idea of supporting multiple runtime execution modes that users can choose from based on the requirements of their use case and the characteristics of their jobs.

The current (and default) execution behavior of our engine is what we call pipelined, or STREAMING, execution mode. In this mode, each operator performs continuous, incremental processing as data flows through the pipeline.

In addition, we plan to support a batch-style execution mode, referred to as BATCH execution mode. This mode executes jobs in a manner more reminiscent of traditional batch processing. We intend to enable this mode via a configuration flag.

Our unified approach to stream and batch processing ensures that applications executed over bounded inputs will produce the same final results regardless of the selected execution mode. Enabling BATCH execution allows the engine to apply additional optimizations that are only possible when operators know that their inputs are bounded.

Below, I will provide an example illustrating the differences between these execution modes:

[chenlica]

@aglinxinyuan I guess you created this issue manually and copied the content from the discussion. If so, notice that GitHub has a feature called "Create issue from discussion." It says: "The original post will be copied into a new issue, and the discussion will remain active." You can try to use this feature in the future.

[aglinxinyuan]

@aglinxinyuan I guess you created this issue manually and copied the content from the discussion. If so, notice that GitHub has a feature called "Create issue from discussion." It says: "The original post will be copied into a new issue, and the discussion will remain active." You can try to use this feature in the future.

I used "Create issue from discussion" to create this issue.