You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I tried to evaluate my RAG app with Groundedness, but there is an error: TypeError: Feedback.__init__() got an unexpected keyword argument 'groundedness_provider'
Hey @TedaLIEz - looks like this code is old. I'll get it updated.
Please try:
importnumpyasnpfromtrulens.apps.llamaindeximportTruLlamafromtrulens.coreimportFeedbackfromtrulens.providers.openaiimportOpenAI# Initialize provider classprovider=OpenAI()
# select context to be used in feedback. the location of context is app specific.context=TruLlama.select_context(query_engine)
# Define a groundedness feedback functionf_groundedness= (
Feedback(
provider.groundedness_measure_with_cot_reasons, name="Groundedness"
)
.on(context.collect()) # collect context chunks into a list
.on_output()
)
# Question/answer relevance between overall question and answer.f_answer_relevance=Feedback(
provider.relevance_with_cot_reasons, name="Answer Relevance"
).on_input_output()
# Question/statement relevance between question and each context chunk.f_context_relevance= (
Feedback(
provider.context_relevance_with_cot_reasons, name="Context Relevance"
)
.on_input()
.on(context)
.aggregate(np.mean)
)
Bug Description
I tried to evaluate my RAG app with Groundedness, but there is an error:
TypeError: Feedback.__init__() got an unexpected keyword argument 'groundedness_provider'
To Reproduce
Following doc: https://www.trulens.org/component_guides/evaluation_benchmarks/groundedness_benchmark/?h=groundedness#benchmarking-various-groundedness-feedback-function-providers-openai-gpt-35-turbo-vs-gpt-4-vs-huggingface
Expected behavior
No errors
Relevant Logs/Tracebacks
Environment:
trulens-apps-langchain==1.3.2
trulens-apps-llamaindex==1.3.2
trulens-core==1.3.2
trulens-dashboard==1.3.2
trulens-feedback==1.3.2
trulens-otel-semconv==1.3.2
trulens-providers-openai==1.3.2
The text was updated successfully, but these errors were encountered: