Skip to content

Commit 414a518

Browse files
zybinmikhailMikhail Zybin
and
Mikhail Zybin
authored
Ndarray dtype fix, improve error wording (#1924)
1. I have noticed that if `response.statements` is an empty list, then the output of `verify_claims` function is `array([], dtype=float64)`, which raises a type error when `~` operation is applied to `reference_response` variable. This occurs in a rare case when hypothesis_list is empty (when there are no claims in the response). 2. A small change is the choice of words in `LLMDidNotFinishException` --------- Co-authored-by: Mikhail Zybin <[email protected]>
1 parent 1d4b2ff commit 414a518

File tree

2 files changed

+9
-4
lines changed

2 files changed

+9
-4
lines changed

src/ragas/exceptions.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ class LLMDidNotFinishException(RagasException):
3737
"""
3838

3939
def __init__(self):
40-
msg = "The LLM generation was not completed. Please increase try increasing the max_tokens and try again."
40+
msg = "The LLM generation was not completed. Please increase the max_tokens and try again."
4141
super().__init__(msg)
4242

4343

src/ragas/metrics/_factual_correctness.py

+8-3
Original file line numberDiff line numberDiff line change
@@ -233,7 +233,13 @@ async def verify_claims(
233233
response = await self.nli_prompt.generate(
234234
data=prompt_input, llm=self.llm, callbacks=callbacks
235235
)
236-
return np.array([bool(result.verdict) for result in response.statements])
236+
if response.statements:
237+
claim_verifications = np.array(
238+
[bool(result.verdict) for result in response.statements]
239+
)
240+
else:
241+
claim_verifications = np.array([], dtype=bool)
242+
return claim_verifications
237243

238244
async def _single_turn_ascore(
239245
self, sample: SingleTurnSample, callbacks: Callbacks
@@ -255,9 +261,8 @@ async def _single_turn_ascore(
255261
premise=response, hypothesis_list=reference_claims, callbacks=callbacks
256262
)
257263
else:
258-
response_reference = np.array([])
264+
response_reference = np.array([], dtype=bool)
259265

260-
response_reference = np.array(response_reference, dtype=bool)
261266
tp = sum(reference_response)
262267
fp = sum(~reference_response)
263268
if self.mode != "precision":

0 commit comments

Comments
 (0)