Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]In the Jupyter notebook, define feedback function error UnicodeEncodeError #1775

Open
hamster1111 opened this issue Feb 2, 2025 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@hamster1111
Copy link

Bug Description
I encountered a UnicodeEncodeError while defining the feedback function

To Reproduce

from trulens.core import TruSession
from trulens.core import Feedback
from trulens.apps.llamaindex import TruLlama
from trulens.providers.langchain import Langchain
from trulens.feedback.v2.feedback import Groundedness
from langchain_community.llms import SparkLLM

tru = TruSession()
tru.reset_database()
langc = Langchain(chain=spark_llm)

qa_relevance = (
    Feedback(langc.relevance_with_cot_reasons, name="Answer Relevance")
    .on_input_output()
)

qs_relevance = (
    Feedback(langc.relevance_with_cot_reasons, name="Context Relevance")
    .on_input()
    .on(TruLlama.select_source_nodes().node.text)
    .aggregate(np.mean)
)


groundedness = (
    Feedback(langc.groundedness_measure_with_cot_reasons, name="Groundedness")
    .on(TruLlama.select_source_nodes().node.text)
    .on_output()
)
feedbacks = [qa_relevance, qs_relevance, groundedness]
tru_recorder = TruLlama(
        query_engine,
        app_id=app_id,
        feedbacks=feedbacks
    )

Expected behavior
It should not report an error, but instead obtain a result similar to the following:

✅ In Groundedness, input source will be set to __record__.app.first.steps__.context.first.invoke.rets[:].page_content.collect() .
✅ In Groundedness, input statement will be set to __record__.main_output or `Select.RecordOutput` .
✅ In Answer Relevance, input prompt will be set to __record__.main_input or `Select.RecordInput` .
✅ In Answer Relevance, input response will be set to __record__.main_output or `Select.RecordOutput` .
✅ In Context Relevance, input question will be set to __record__.main_input or `Select.RecordInput` .
✅ In Context Relevance, input context will be set to __record__.app.first.steps__.context.first.invoke.rets[:].page_content .

Relevant Logs/Tracebacks
The error message is as follows:

UnicodeEncodeError                        Traceback (most recent call last)
Cell In[24], line 4
      1 
      2 qa_relevance = (
      3     Feedback(langc.relevance_with_cot_reasons, name="Answer Relevance")
----> 4     .on_input_output()
      5 )
      7 qs_relevance = (
      8     Feedback(langc.relevance_with_cot_reasons, name="Context Relevance")
      9     .on_input()
     10     .on(TruLlama.select_source_nodes().node.text)
     11     .aggregate(np.mean)
     12 )
     15 groundedness = (
     16     Feedback(langc.groundedness_measure_with_cot_reasons, name="Groundedness")
     17     .on(TruLlama.select_source_nodes().node.text)
     18     .on_output()
     19 )

File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Lib\site-packages\trulens\core\feedback\feedback.py:296, in Feedback.on_input_output(self)
    289 def on_input_output(self) -> Feedback:
    290     """
    291     Specifies that the feedback implementation arguments are to be the main
    292     app input and output in that order.
    293 
    294     Returns a new Feedback object with the specification.
    295     """
--> 296     return self.on_input().on_output()

File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Lib\site-packages\trulens\core\feedback\feedback.py:591, in Feedback.on_prompt(self, arg)
    589 if arg is None:
    590     arg = self._next_unselected_arg_name()
--> 591     self._print_guessed_selector(arg, select_schema.Select.RecordInput)
    593 new_selectors[arg] = select_schema.Select.RecordInput
    595 ret = self.model_copy()

File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Lib\site-packages\trulens\core\feedback\feedback.py:323, in Feedback._print_guessed_selector(self, par_name, par_path)
    320 else:
    321     alias_info = ""
--> 323 print(
    324     f"{text_utils.UNICODE_CHECK} In {self.supplied_name if self.supplied_name is not None else self.name}, "
    325     f"input {par_name} will be set to {par_path}{alias_info} ."
    326 )

File D:\work\hamster\pythonProject\reinforce_interview\RAG_reinforce\venv\Scripts\pywin32_postinstall.py:27, in Tee.write(self, what)
     25     except OSError:
     26         pass
---> 27 tee_f.write(what)

UnicodeEncodeError: 'gbk' codec can't encode character '\u2705' in position 0: illegal multibyte sequence

Environment:

  • OS: Windows
  • Python Version: 3.12
  • TruLens version:1.3.3
@hamster1111 hamster1111 added the bug Something isn't working label Feb 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants