Option to run periodic jobs without leader election?

[RobinLbt]

I have a specific use case where I need periodic jobs to run on every instance of our application, rather than just on the leader.

In our case, we're using these jobs to synchronize and populate in-memory caches with data on each instance.

Currently, River's periodic jobs use leader election to ensure only one worker manages periodic jobs at any time.

While this is great for most use cases, it doesn't work well when you need the job to execute on all instances independently.

Here's a simplified example of our current setup:

goCopyperiodicJobs := []*river.PeriodicJob{
    river.NewPeriodicJob(
        river.PeriodicInterval(10*time.Minute),
        func() (river.JobArgs, *river.InsertOpts) {
            return InMemoryDataSyncArgs{}, nil
        },
        &river.PeriodicJobOpts{RunOnStart: true},
    ),
}

Would it be possible to add an option to run periodic jobs without leader election? Something like:

goCopy&river.PeriodicJobOpts{
    RunOnStart: true,
    SkipLeaderElection: true,  // Hypothetical option
}

I understand this might be a niche use case, but it would be helpful for scenarios involving instance-specific tasks like cache warming, local data synchronization, or in-memory state management.

[bgentry]

@RobinLbt from your post I’m not sure if you’re aware that River does not have any sort of host affinity or way to ensure that a job runs on any particular worker. Once it gets enqueued the job will get picked up by whichever worker happens to get to it first (on the queue in question). As such there is no way to guarantee that a set of jobs is spread evenly across all hosts, regardless of them being periodic or not.

The only way you could ensure this is if each host was working a queue that was unique for itself, and you enqueued the job in each of these queues separately. I’m not sure this would be worth it though, it feels like trying to force something that goes against the model of horizontally scalable workers.

My recommendation is to use a different mechanism for these specific workloads. For example the cron library offers a way to run a task periodically on the specific host that’s running that logic, so you could have it on every instance if you want to. Its schedule interface is also the same as River’s so you could share schedules if you want.