TruLens Eval v0.19.2

Changelog

• Add port and server to Tru.run_dashboard (#686)
• Instrument langchain expression language based apps (#691) (#686)
• extract query_str from query bundle as main input for llama_index apps (#690)

Examples

• Change LangChain Quickstart to LCEL-RAG based example (#691)

Bug Fixes

• Better messages for optional imports (#688)

TruLens-Eval-0.19.1

Changelog:

• Remove cohere provider

TruLens-Eval v0.19.0

Changelog:

• Add support for Pydantic v2
• Improving handling/messages for openai client users
• Add Langchain Feedback Provider

Bug Fixes:

• Fix llama-index LLMBase import
• Use union to support python versions < 3.10

TruLens-Eval-0.18.3

Changelog:

• Migrate LiteLLM to v1 (#644)
• Migrate AzureOpenAI to v1 (#640)
• Expand groundedness smoke testing (#634)

Bug Fixes:

• Fix Langchain-based evals so that data is properly evaluated (#636)
• Fix escape unicode on records table (#632)
• Catch more endpoint exceptions when tracking costs (#627)
• Add wrapper for dynamically generated functions in boto3 (#626)
• Update AzureOpenAI example (#647)

TruLens-Eval-0.18.2

Changelog

• Unpin typing-extensions, typing-inspect (#590)
• Make CI Pipeline run daily (#599)
• Increase test coverage to all /quickstart* notebooks (#601)

Examples

• Add notebook to use for dev and debugging (#605)
• Add example for multimodal rag eval (#617)
• Add example for finetuning experiments in bedrock (#618)

Bug Fixes

• Fix helpfulness prompt (#594)
• Serialize OpenAI Client (#595)
• Removed extra reset cell in quickstart (#597)
• Fix langchain prompt template imports in examples (#602)
• Change model_id -> model_engine in Bedrock example (#612)
• Fix prompt swapping in model agreement feedback (#615)
• Fix > character in groundedness prompt (#623)

TruLens-Eval-0.18.0

Evaluate and Track LLM Applications

Changelog

• Migrate to OpenAI v1.

Known issues with async.

TruLens Eval v0.17.0

Changelog:

• Add criteria and improve chain of thought prompting for evals
• Allow feedback functions to be in different directions with appropriate coloring/emojis
• Filter leaderboard feedback function results to only those available for the given app id
• Add smoke testing/benchmarking for groundedness based on SummEval dataset

Bug Fixes:

• Fix issue with LiteLLM provider
• Allow Groundedness to run with any LLM provider

Examples

• Using Anthropic Claude to run feedback functions

TruLens Eval v0.16.0

Library containing evaluations of LLM Applications

Changelog

• [MLNN-1020] App runner UI updates by @walnutdust in #503
• Fix groundedness aggregation flakiness + incorrect 0 resolution. by @joshreini1 in #501
• [MLNN-1053] Add groundedness to Pinecone notebook by @ejisoo in #506
• [MLNN-1046] Example app with TruLens by @daniel-huang-1230 in #500
• [MLNN-1053] Update dependencies in pinecone notebook by @ejisoo in #507
• threading robustness and feedback retrieval by @piotrm0 in #480
• Merge generated docs and test files into main by @github-actions in #508
• updating JSONPath and features by @piotrm0 in #502
• handle no pii by @joshreini1 in #504
• Update Instrumentation Overview page: fix link, add trucustom by @joshreini1 in #505
• bugfixes by @piotrm0 in #510
• dashboard appui quickstart by @piotrm0 in #511
• Fix app UI showing the record twice by @walnutdust in #513
• Release branch 0.16.0 by @daniel-huang-1230 in #514

Bug Fixes

• Fix App UI, links, icons

New Contributors

• @ejisoo made their first contribution in #506
• @daniel-huang-1230 made their first contribution in #500

Full Changelog: trulens-eval-0.15.3...trulens-eval-0.16.0

TruLens Eval v0.15.3

Library containing evaluations of LLM Applications

Bug Fixes

• Fixed OpenAI provider issues for feedback functions

TruLens Eval v0.15.1

Library containing evaluations of LLM Applications

Changelog

• PII Detection Feedback Function
• Embedding Distance Feedback Function
• App UI Playground

Examples

• All new User Guides Docs Section
• Language Verification
• PII Detection
• Hallucination Detection
• Retrieval Quality

Bug Fixes

• Unicode Issue on Windows