Edit existing workflows via AI chat

[taylordowns2000]

Currently, once a template is chosen via the "Create" button, the user is taken to the canvas and the chat panel disappears. From this point onwards, they're not able to modify their workflow using the AI chat.

But... the AI chat already takes the current workflow.yaml and the chat history... in other words, it's set up to edit workflows already and we should extend this functionality to the canvas.

But here's the catch: Right now, the workflow chat agent does not return job code. This is probably a good thing, as we don't want a high level workflow agent to mess with a user's existing job code until we have a nice code suggestion interface (coming next, like cursor) that allows the user to view each change and accept/reject it in the inspector.

So... for this issue, we'd want:

• the chat panel to stay available (but be collapsible) on the left side of the canvas
• the workflow chat to be able to modify YAML but not modify job code.
• to make sure that for existing jobs it preserves job code.
• to make sure that if you went back to a previous message in the chat you could still view the workflow as it looked at that point in time. (Without saving.) maybe related to Workflow Diagram: add undo/redo #2177

Note that deleting a step entirely will result in deleting job code, but that's OK as we'll have the previous version of the job code in the chat and/or in the snapshot history and/or in GitHub.

[josephjclark]

Please come and speak to me before starting this!

[stuartc]

What's been done:

Basic Toggle & Panel

• Added AI chat toggle button in workflow editor header
• Created WorkflowAiChatComponent for persistent AI chat panel
• Implemented panel show/hide functionality with proper event handling
• Panel reuses existing new workflow panel area with smooth transitions
• Added explicit close button (different from new workflow auto-dismiss behavior)

What's Next:

• Workflow state integration - Pass current workflow YAML to AI (not just from the last message in the chat history)
• Restore functionality - ensure users who edit a workflow without the AI assistant, are able to restore the workflow from the point they wrote a specific prompt

Draft PR here: #3319

[elias-ba]

This issue is blocked because currently the workflow assistant returns a YAML code that represents a whole new state of the workflow diagram, then we replace that with the previous state of diagram. The problem with that is it generates whole new UUIDs for the entities (jobs, edges, and triggers) resulting in anytime the user saves the workflow, Ecto deletes the previous entities in the database and creates new ones.

In the workflow creation flow, that is acceptable, but in edit mode we can't do that. There are many risks, for example: How would that affect previous runs of the workflow? What happens to audit trails and foreign key relationships?

While we could build complex ID-matching algorithms to guess which entities correspond to each other, this approach is error-prone and doesn't scale. We're already seeing issues with duplicate triggers and entity matching failures.

I think we should push toward the Assistant rendering JSON Patches (JSON Patch RFC 6902 or Diff/Patch like Git). That way we can do real edits of entities instead of replacing the whole workflow diagram. With JSON Patches, the AI would explicitly state its intent (update vs create vs delete), making the integration more reliable and the UX more predictable.

This would also align with how our frontend editor already communicates changes - using patches rather than full state replacement. The frontend sends patches, the AI should too.

Without this architectural change, we're building increasingly fragile workarounds that will eventually break and provide a poor user experience.

[josephjclark]

Call with @elias-ba and @hanna-paasivirta :

Get a patch (or set of patches) back from apollo

• aka ask apollo to give us commands or edits
• just like we're going to do with job chat
• might be hard to apply patches? Complex with many changes in one response?
• patches still need IDs to be applied? Unless we patch as text?

Send UUIDs to apollo

• aka ask apollo to give us a new structure
• model might forget or change uuids
• ID could be a simple fixed key
• Lightning can validate workflows and raise an error (then user can retry)
• what is the consequence of bad uuids? UUIDS of the same things might change across subsequent runs. Also we fill the database with junk. How will this affect auditability? Will the workflow history become unusable?
• We think this "id corruption" is probably fairly rare?

Solution

• Lightning will send UUIDs to Apollo
• Apollo will simplify/swap out those UUIDs before calling the model
• Apollo will restore the original UUIDs before sending them back to Lightning
• maybe later apollo will use a patch format, like new job chat, internally. This might reduce id churn (if it proves to be a problem)
• Can we add any heuristic lightning side to track how many new UUIDs come out of an edit?