Skip to content

Proposal: Jido Objects as a batteries-included durable object runtime for Phoenix apps #243

@mikehostetler

Description

@mikehostetler

Summary

We should put together a batteries-included Jido Objects story for app developers who want a Cloudflare Durable Objects-like experience in Elixir/Phoenix apps.

The important distinction is that durability already belongs in Jido core. This proposal is not about inventing a new durability layer. It is about packaging the existing durability, persistence, clustering, routing, and Phoenix integration into a coherent, easy-to-consume product story.

This use case has clear demand. I ran a Twitter poll and this was the number one requested feature/use case. We should treat that as a strong signal and shape a proposal that is flexible enough to fit the broader ecosystem without forcing the wrong abstractions too early.

Problem

Right now, the pieces are emerging, but the consumption story is still too fragmented:

  • jido owns core runtime semantics and durability
  • persistence already exists or is emerging via Postgres/Ecto and Bedrock
  • jido_cluster owns ownership/routing/failover semantics
  • Phoenix starter/demo apps show how the pieces can be embedded in a host app

That is a strong technical foundation, but it is not yet a simple answer to:

“How does a Phoenix app developer use Jido to get durable objects?”

We need a higher-level, batteries-included story that lets users stay inside their own Phoenix app rather than feeling like they need to run a separate Jido server/control plane.

Framing

I think the right framing is:

  • Jido core remains where durability, hibernate/thaw, checkpoints, journals, and persistence contracts live
  • storage adapters remain where Ecto/Postgres and Bedrock integrations live
  • jido_cluster remains where deterministic ownership, routing, and failover live
  • a higher-level package/repo should own the object runtime DX

That higher-level layer is what makes the story feel like “Durable Objects for Phoenix/Elixir apps”.

Proposal

Create a new product-level package/repo tentatively called jido_objects.

This package would sit above the core/runtime packages and provide a batteries-included object model backed by Jido agents.

Durable objects would be the first concrete use case

The first class of object should be durable objects:

  • keyed by logical object ID
  • single active owner per key
  • durable checkpoint/journal-backed state
  • hibernate/thaw on idle
  • failover/recovery through shared persistence
  • easy routing from Phoenix transports and app code

But the API should leave room for other object types later

This should not lock us into one object type forever. The object abstraction should leave room for additional types backed by Jido agents, for example:

  • local/in-memory objects
  • durable objects
  • region-affine objects
  • transient/session-scoped objects
  • other policy-driven object types as the ecosystem evolves

That gives us a strong use-case-specific product today without painting the ecosystem into a corner.

Where this fits in the ecosystem

Proposed layering:

  • jido
    • core runtime
    • durability semantics
    • hibernate/thaw
    • checkpoints/journals/persistence contracts
  • storage adapters
    • Postgres via Ecto
    • Bedrock
    • others as needed
  • jido_cluster
    • ownership
    • routing
    • failover/rebalancing
  • jido_objects
    • app-facing object runtime
    • object definitions and lifecycle ergonomics
    • batteries-included integration story
    • install generators/templates
    • Phoenix integration
  • starter/reference apps
    • complete examples
    • Fly.io deployment recipes
    • product/domain demos

Phoenix integration story

This needs to be very easy.

The intended model should be:

  • developers embed Jido Objects inside their own Phoenix app
  • they do not run a separate Jido Phoenix server as a separate product/service by default
  • Phoenix remains the host app and transport layer
  • Jido Objects provides the object runtime inside that app

That means jido_objects should make Phoenix integration feel first-class, likely through modules/helpers/macros along the lines of:

  • Channels integration
  • LiveView integration
  • PubSub topic conventions for object output
  • attach/detach lifecycle helpers
  • simple call/cast/get routing by object key

Whether that Phoenix integration lives directly in jido_objects or in a sibling jido_objects_phoenix package is still open, but the user story should feel unified and extremely simple.

Consumption story for end users

The desired story should look something like this:

  1. Add one dependency to a Phoenix app
  2. Run an install task/generator
  3. Configure storage backend (ecto/Postgres first, Bedrock optionally)
  4. Enable clustering for Fly or other multi-node deployment
  5. Define durable objects backed by Jido agents
  6. Use them from controllers, channels, or LiveViews
  7. Deploy the same Phoenix app to Fly

In other words, we want:

“I added Jido Objects to my Phoenix app and now I have durable, keyed, singleton object runtime semantics.”

That is the level of polish we should target.

Why this should likely be a product-level repo

This feels like the start of a broader pattern in the ecosystem:

  • core runtime packages stay focused and composable
  • higher-level packages can package strong use-case-specific stories
  • those packages should be flexible, opinionated where helpful, and easy to adopt

This use case is exactly the kind of thing that benefits from a product-level package/repo above the raw runtime primitives.

So I think this is a good candidate for a new pattern:

  • use-case-specific Jido packages
  • built on top of the ecosystem
  • batteries-included consumption story
  • still aligned with the core architecture

MVP scope

A sensible MVP for jido_objects would likely include:

  • durable objects as the first supported object type
  • keyed singleton semantics per object ID
  • Phoenix host-app integration story
  • Postgres/Ecto storage support as the easiest production path
  • Bedrock support as an advanced/first-party clustered storage path
  • jido_cluster integration for ownership/routing/failover
  • install task/generator for Phoenix apps
  • starter/reference app and Fly.io deployment documentation

Non-goals

For clarity, this proposal is not:

  • a replacement for durability in core Jido
  • a new storage engine
  • a requirement to run a separate Jido control plane/server
  • a commitment to making all object types fully generic on day one

Open questions

  • Is jido_objects the right name, with “durable objects” used as the primary tagline/story?
  • Should Phoenix integration live in the same package initially, or in a sibling package once the seams are proven?
  • Should the first production path be Postgres/Ecto-first, with Bedrock positioned as the advanced/native clustered backend?
  • How much of the object taxonomy should be explicit in v1 vs. leaving durable objects as the first preset and expanding later?
  • What should the install/generator story look like for Phoenix apps?

Why now

The main reason to do this now is not just technical readiness. It is that we now have:

  • clear demand signal
  • a recognizable market/problem framing
  • enough ecosystem pieces to tell a credible story
  • an opportunity to define a repeatable pattern for future Jido product-level packages

The goal should be to make this both sensible in the architecture and obvious to adopt for end users.

Metadata

Metadata

Assignees

No one assigned

    Labels

    parkedWorth keeping open, not worth acting on now

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions