LlmFilter prompt fix

[dtecuci]

new LlmFilter prompt and score extraction. Still todo route the output of LlmFilter prompt into Doc element properties

[HenryL27]

if this works let's take it but I'm dubious on the formatting at the end (and it sounds like you are too)

(from slack)
Can I suggest something like

"justification_text": {{ {"type": "string", "description": "reason for the score"} }},
"score": {{ {"type": "int", "description": "confidence score (0-5) that the text satisfies the question", "constraint": ">= 0, <= 5" } }}

(I made up the constraint thing I have no idea if that works but it's an llm so maybe). Also you can specify a response format which gets used by openai (not anthropic tho and I think you have claude in finra right? although ig you have a special llm client so maybe you can incorporate it) like so:

class LlmFilterResponse(pydantic.BaseModel):
    justification_text: str # or some more complicated field specification
    score: int # or some more complicated field specification

LlmFilterMessagesJinjaPrompt = JinjaPrompt(
    ...,
    response_format = LlmFilterResponse
)

(can also be just a json schema dict thingy instead of pydantic)

for getting the scores if you can get semireliable json production you can use sycamote.utils.extract_json.extract_json (or something like that) to parse, probably more reliable than any regex