Skip to content

Run the evaluation locally fails #70

@NanaXiong00

Description

@NanaXiong00

Describe the issue:
When executing python ./src/evaluate.py --evaluation-name assistant-dev --evaluation-data-path ./src/data/ground_truth_sample.jsonl --metrics similarity according to README.md, the error is as follows:
image
image

Repro Steps:

  1. Run git clone https://github.com/Azure-Samples/assistant-data-openai-python-promptflow
    and cd assistant-data-openai-python-promptflow
  2. Run python -m venv .venv and .venv\scripts\activate
  3. Run pip install -r ./src/requirements.txt
  4. Run azd auth login
  5. Run azd provision
  6. Run azd deploy
  7. Run pf flow test --flow ./src/copilot_sdk_flow/flow.flex.yaml --inputs chat_input="which month has peak sales in 2023"
  8. Run python ./src/evaluate.py --evaluation-name assistant-dev --evaluation-data-path ./src/data/ground_truth_sample.jsonl --metrics similarity

Environment:

  • Azd version: azd version 1.9.5 (commit cd2b7af9995d358aab33c782614f801ac1997dde)
  • OS: Windows desktop

Expected behavior:
Run python ./src/evaluate.py --evaluation-name assistant-dev --evaluation-data-path ./src/data/ground_truth_sample.jsonl --metrics similarity can pass.

@jongio and @jfomhover for notification.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions