Hi there,
just found a possible bug. The AnswerRelevancy scorer allows to pass an embedding_model parameter, which is used in the EmbeddingSimilarity call of the async runner, but not in the synchronous version, leading to wrong model usage when using the scorer synchronously.
EmbeddingSimilarity(client=self.client).eval_async(
output=q["question"], expected=input, model=self.embedding_model
)
vs.
EmbeddingSimilarity(client=self.client).eval(output=q["question"], expected=input, model=self.model)
created a PR: #167