Ramen: Optimizing Operator Parallelism in Pipelined Dataflows with Dedicated Resources Using Reinforcement Learning

---

Overview: Ramen Implementation on Texera

This supplemental package contains the implementation of Ramen on Texera, including core algorithms, automation scripts, datasets, and workflows.

Environment Requirements

• Python 3.10+

• Texera

  • Java 11
  • Node.js (LTS version)
  • SBT
  • Yarn

• (Optional) MySQL
  The provided script uses MySQL to record and retrieve runtime statistics. Other storage methods are acceptable if they provide equivalent content.

• (Optional) Chrome
  The provided script automates workflow execution using Chrome. Other methods to trigger execution are acceptable.

To run Texera

First, install frontend packages by running this command in bash:

core/scripts/build.sh

To enable Python support in Texera, please install the required dependencies:

pip install -r core/amber/requirements.txt

Then, start both frontend and backend by running this command:

core/scripts/deploy-daemon.sh

We also provide a detailed description of how to deploy Texera in the texera folder.

Ramen

We include the following approaches:

• Estimator-based methods
• Greedy methods
• Reinforcement learning–based methods

All the implementations are under ramen folder. For each method, the implementation is organized in separate subfolders under the ramen directory.
We provide detailed instructions in their corresponding README.md files.