Skip to content

Commit 79910f9

Browse files
gustavocidornelaswhoseoyster
authored andcommitted
Fixes OPEN-5957 Completion tokens not being computed by OpenAIMonitor when stream=True
1 parent ab656a0 commit 79910f9

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

openlayer/llm_monitors.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,8 @@ def stream_chunks():
211211
raw_outputs.append(chunk.model_dump())
212212
if i == 0:
213213
first_token_time = time.time()
214+
if i > 0:
215+
num_of_completion_tokens = i + 1
214216

215217
delta = chunk.choices[0].delta
216218

@@ -236,8 +238,6 @@ def stream_chunks():
236238
] += delta.tool_calls[0].function.arguments
237239

238240
yield chunk
239-
if i > 0:
240-
num_of_completion_tokens = i + 1
241241
end_time = time.time()
242242
latency = (end_time - start_time) * 1000
243243
# pylint: disable=broad-except

0 commit comments

Comments
 (0)