[BUG] Groundedness not working

[liambarstad]

Bug Description
groundedness_measure_with_cot_reasons is failing silently

To Reproduce

session = TruSession()

class RAG:
    @instrument
    def get_chunks(self, response_body: dict) -> list:
        return [ s['doc'] for s in response_body['documents']

    @instrument
    def query(self, input: str) -> str:
        response = requests.post('<<url>>', json={'query': input})
        if response.status_code == 200:
            chunks = self.get_chunks(response.json())
            return response.json()['response']

rag = RAG()

provider = Bedrock(model_id='<<model_id>>', region_name='<<region name>>')

f_groundedness = (
    Feedback(
        provider.groundedness_measure_with_cot_reasons, name='Groundedness'
    )
    .on(Select.RecordCalls.get_chunks.rets.collect() 
    # this does not work with rets[:] or just rets either
    .on_output()
)

# this works fine with answer and context relevance

tru_rag = TruCustomApp(
    rag,
    feedbacks=[f_groundedness
)

Expected behavior
I expect the metric to show up in my dashboard, since I'm following the sample code in the docs nearly to the letter

Relevant Logs/Tracebacks
When the dashboard is opened, "No feedbacks" is displayed, or groundedness is not shown. Besides that, it fails completely silently

Environment:

• OS: Windows
• Python Version: 3.12
• TruLens version: 1.2.6
• Versions of other relevant installed libraries

[sfc-gh-jreini]

Hey @liambarstad at least in this code snippet above you are missing some closing parenthesis and brackets that might lead to his error.

Here's your code fixed:

f_groundedness = (
    Feedback(
        provider.groundedness_measure_with_cot_reasons, name='Groundedness'
    )
    .on(Select.RecordCalls.get_chunks.rets.collect()) # added missing closing parenthesis
    .on_output()
)

# this works fine with answer and context relevance

tru_rag = TruCustomApp(
    rag,
    feedbacks=[f_groundedness] # added missing closing bracket
)

[liambarstad]

Those were typos on my part from copying the code over, those aren't part of the original

[julia940]

Those were typos on my part from copying the code over, those aren't part of the original

I also failed in the groundedness of this metric. Do you happen to have some methods to deal with this problem?

[julia940]

I also failed in the groundedness of this metric. Do you happen to have some methods to deal with this problem?
def f_groundedness():
provider = fOpenAI(model_engine='gpt-3.5-turbo')
return (
Feedback(provider.groundedness_measure_with_cot_reasons, name = "Groundedness")
.on(Select.RecordCalls.retrieve.rets.collect())
.on_output()
) This is my code, I also failed in the groundedness of this metric. Do you happen to have some methods to deal with this problem?