kthena-router: order boosted requests by session completion time

docs/kthena/docs/user-guide/session-boost.md

@@ -112,6 +112,7 @@ env:
 | `ENABLE_SESSION_BOOST`           | Enable the session-boost scheduling strategy                                     | `false`        | Mutually exclusive with `ENABLE_FAIRNESS_SCHEDULING`                                                                                                                                                                                                              |
 | `SESSION_BOOST_HEADER`           | HTTP header used to identify conversation sessions                               | `X-Session-ID` | Must match what your clients send                                                                                                                                                                                                                                 |
 | `SESSION_BOOST_MAX_SESSIONS`     | Max number of recently-completed sessions kept "warm" for boosting (LRU-bounded) | `4096`         | When the cache is full, the least-recently-used session is evicted automatically. Size it by the number of concurrent conversations you want to keep boosted—no time-based tuning required                                                                        |
+| `SESSION_BOOST_MAX_OVERTAKES`    | Max times a queued boosted request may be overtaken by newer completions         | `16`           | Bounds starvation: once a boosted request has been overtaken this many times it is "forced" and ranks ahead of newer completions, guaranteeing it runs before hitting `SESSION_BOOST_TIMEOUT`. Lower it to bound tail latency; raise it to favor prefix-cache reuse |
 | `SESSION_BOOST_INFLIGHT_PER_POD` | Inflight requests admitted per backend pod                                       | `16`           | The total inflight limit is this value multiplied by the number of backend pods. Size it from the per-pod concurrency (e.g., vLLM's `--max-num-seqs`)                                                                                                             |
 | `SESSION_BOOST_TIMEOUT`          | Maximum time a request may wait in the queue before it is rejected with 504      | `30s`          | Enabled by default. A request queued longer than this is rejected with `504 Gateway Timeout`. Set a non-positive duration (e.g. `0s`) to disable it. This is the only server-side queue-wait bound in session-boost mode; `FAIRNESS_QUEUE_TIMEOUT` does not apply |
 
@@ -169,7 +170,10 @@ This design is intentional: inference engines such as vLLM evict their prefix (K
 The session boost queue uses a simple two-level priority:
 
 1. **Boosted requests** (session tracked in the LRU cache) are always dequeued before non-boosted requests.
-2. **Within the same boost level**, requests are served in FIFO order (earliest arrival first).
+2. **Within the boosted group**, requests are ordered by their session's **completion time**: the session whose previous turn completed most recently is served first. Its prefix (KV) cache is the most likely to still be warm on the backend, so serving it first maximizes cache reuse. This ordering does not depend on when the follow-up request arrived, only on how recently the session's last turn finished; ties are broken FIFO by arrival time.
+3. **Among non-boosted requests**, ordering is FIFO (earliest arrival first).
+
+Ordering by completion time (rather than a plain LIFO on arrival) keeps the queue fair: a boosted request cannot be starved by newer arrivals, because a session's completion time is fixed once its turn finishes and does not keep getting pushed back. Every boosted request also still leapfrogs the entire non-boosted tier, so a follow-up turn is always prioritized over unrelated traffic.
 
 ### Backpressure Control
 
@@ -197,13 +201,13 @@ If a request waits in the queue longer than `SESSION_BOOST_TIMEOUT`, the router
 
 Session boost and user fairness are two mutually exclusive scheduling strategies for the per-model request queue; they configure that queue for different goals:
 
-| Aspect           | Session Boost                      | User Fairness                     |
-| ---------------- | ---------------------------------- | --------------------------------- |
-| Goal             | Maximize prefix cache hits         | Equitable resource allocation     |
-| Activation       | `ENABLE_SESSION_BOOST=true`        | `ENABLE_FAIRNESS_SCHEDULING=true` |
-| Requires user ID | No                                 | Yes                               |
-| Priority logic   | Boosted > non-boosted, FIFO within | Lower recent usage wins           |
-| Best for         | Multi-turn latency optimization    | Multi-tenant contention           |
+| Aspect           | Session Boost                                                | User Fairness                     |
+| ---------------- | ------------------------------------------------------------ | --------------------------------- |
+| Goal             | Maximize prefix cache hits                                   | Equitable resource allocation     |
+| Activation       | `ENABLE_SESSION_BOOST=true`                                  | `ENABLE_FAIRNESS_SCHEDULING=true` |
+| Requires user ID | No                                                           | Yes                               |
+| Priority logic   | Boosted > non-boosted; most-recently-completed session first | Lower recent usage wins           |
+| Best for         | Multi-turn latency optimization                              | Multi-tenant contention           |
 
 The two strategies are **mutually exclusive**; enabling both is a configuration error. Enable one or the other based on your primary scheduling concern.
 
@@ -286,9 +290,9 @@ The grace period is an **advanced, scenario-specific** tuning knob. It is **disa
 
 By default, when a request completes the queue immediately dequeues the next request. The grace period (`SESSION_BOOST_GRACE_PERIOD`) instead makes the queue **briefly hold the freed dequeue slot** after a completion, waiting for a follow-up from the *same* session to arrive so it can ride the still-warm prefix cache:
 
-- If a boosted (same-session) request arrives during the grace window, it is admitted first when the window ends, because boosted requests outrank others in the queue.
+- If a boosted (same-session) request arrives during the grace window, it is admitted first when the window ends, because boosted requests outrank others in the queue—and among boosted requests the one whose session completed most recently is at the head.
 - If no boosted request arrives before the window expires, the next non-boosted request proceeds as normal.
-- If the head of the queue is already a boosted request, the grace period is skipped entirely.
+- The queue always waits for the full grace window even when a boosted request is already at the head, because a still-newer same-session follow-up may yet arrive and should be the one to ride the warm prefix cache.
 
 ### When it might help