single_threaded_executor needs refactoring

[thirtytwobits]

As I'm working though an official FreeRTOS integration I'm learning more about the design. Mostly things are going smoothly and integrating with FreeRTOS+TCP proceeds apace, however, the single_threaded_executor is wholly unusable in FreeRTOS. This would be okay (i.e. I can just implement IExecutor) except we have a lot of important logic in there to support scheduling that I can't reuse without copy-and-paste. Namely, IExecutor has no scheduling contract. This is the first thing: schedules should be defined as part of the design instead of a single implementation of IExecutor. It might be that we refactor the single_threaded_executor to utilize an AVL-tree-based priority queue that can be mixed into different executors. On FreeRTOS I want to have a single loop that is blocking on a FreeRTOS queue and consulting with something like this inner priority queue for libcyphal work based on I/O (i.e. it wakes when the ethernet peripheral signals new data). For this to work I need to know how long to block for before I need to wake up and service the executor.

I'm also wondering if Schedule should be part of IExecutor? When bootstrapping an integration one gets to a point where the media layer is ported, the transport is being created, and they are ready to test this out. Unless using single_threaded_executor the person (me in this case) is left to wonder "how is this supposed to work?". There's no indication of how an IExecutor is required to operate. This is somewhat per-design in that we wanted to leave this very flexible but it does seem like the ability to schedule work is an immutable requirement and executors should be required to support it. This schedule method would return a time to next scheduled job that integrations can use for blocking time outs or to schedule timers, etc.

Thoughts?

[serges147]

For this to work I need to know how long to block for before I need to wake up and service the executor.

SingleThreadedExecutor::spinOnce() returns SpinResult - its next_exec_time tells you how long to wait/sleep.

[serges147]

Namely, IExecutor has no scheduling contract.

This I can't get. What do you mean by "no scheduling contract"? Interface, schedule method, detailed description of Schedule::Variant, ... all there. Unless you mean something else.

[serges147]

...however, the single_threaded_executor is wholly unusable in FreeRTOS.

Quite possible. I also didn't see immediate and easy way to marry my SingleThreadedExecutor with Capn'tProto's KJ asyncs... although I didn't really tried (b/c of lack of time due to ocvsmd pressure).

So, I agree, to be sure that execution design is not only posix/epoll/kqueue capable we should add to agenda concrete tasks to try also with FreeRTOS and maybe c++20 coroutines.

[serges147]

Also, I would like repeat myself about my proposal (when Erik first started to ask questions about executor and RTOS integrations) to have a shared repo (or subdirectory at the existing demos repo) where we can productively share concrete use case (f.e. FreeRTOS on a STM32 target - it will cover both rtos specifics, maybe 32-bits specifics, as well as reveal early such things as unexpectedly big flash size I read about in the forum).

[thirtytwobits]

Namely, IExecutor has no scheduling contract.

This I can't get. What do you mean by "no scheduling contract"? Interface, schedule method, detailed description of Schedule::Variant, ... all there. Unless you mean something else.

By "contract" I mean that I can implement IExecutor without even realizing there are scheduling functions. The "pure virtual method not implemented" compiler errors in C++ provide a nice porting guide when one is starting out. If we had an IExecutor method that was "schedule" then I'd have to implement it if I were building my own executor (which I am). This would force me to evaluate how to implement the scheduling function for the platform I'm on.

I'm not sure that I'm right about this yet but it feels right. It might be that we need a bit more hierarchy in executor contracts. Perhaps we have IExecutor be the dumb base and we have something like ISchedulingExecutor for executors that have to be driven by external forces whereas something like an IMultiThreadedExecutor would simply have methods for starting and stopping the underlying thread pools that handled this scheduling?

Generally though, I agree that we need a bit more diversity then just Posix and FreeRTOS to get this all figured out. After FreeRTOS I think our next target has to be Zephr.