-
Notifications
You must be signed in to change notification settings - Fork 0
Linea Internals

Above image is what the topology looks like on the inside. Each Bolt (Spout's also a Bolt) is run using a BoltExecutor.
Bolt Executor is a wrapper to manage the Bolt instances and their execution in parallel. It is responsible for creating a fixed size thread pool to execute the bolt instances. Each Bolt instance is wrapped in BoltExecutorWrapper which is also the Disruptor EventHandler. Each Bolt instance gets it's own Disruptor (and RingBuffer) instance and 1 thread from the shared thread pool to execute the Bolt code for this instance. The diagram shows these different RingBuffers, one per Bolt Instance.
If a Bolt Instance is selected for processing an Event, the Bolt Executor's process method is called with the Instance ID and the Event. The Bolt Executor looks up it's internal hashmap to locate the RingBuffer of the requested Bolt Instance and publishes the Event into that buffer.
Spout is also run like other Bolt however there's a fundamental difference in the use and behavior of Disruptor. For emitting event Spout directly uses Collector, the Disruptor and it's RingBuffer is only used to notify the Spout of acks and fails of an Event. Therefore Disruptor is not used for Event emits in the case of Spout.
Ackers is also a Bolt, responsible for tracking the guaranteed processing of Events using the XOR Ledger design created by Nathan Marz in Apache Storm. Ackers receive ack events and emit out ack finish or ack failure events ack to the source Spouts. The diagram below shows how these function in a given topology:
