From f1031a5e627b87dd25b534c6158d9f791eca8448 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Wed, 15 Jan 2025 15:59:47 +0800 Subject: [PATCH 01/14] Add a preview of v1.2 release notes. --- release_notes/v1.2.md | 368 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 368 insertions(+) create mode 100644 release_notes/v1.2.md diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md new file mode 100644 index 00000000..0d6c4f28 --- /dev/null +++ b/release_notes/v1.2.md @@ -0,0 +1,368 @@ +# OPEA Release Notes v1.2 +We are excited to announce the release of OPEA version 1.2, which includes significant contributions from the open-source community. This release addresses over 245(TODO: update the latest) pull requests. + +More information about how to get started with OPEA v1.2 can be found at [Getting Started](https://opea-project.github.io/latest/index.html) page. All project source code is maintained in the [repository](https://github.com/opea-project). To pull Docker images, please access the [Docker Hub](https://hub.docker.com/u/opea). For instructions on deploying Helm Charts, please refer to the [guide](https://github.com/opea-project/GenAIInfra/tree/v1.2/helm-charts#readme). + +## What's New in OPEA v1.2 +This release focuses on code refactoring for [GenAIComps](https://github.com/opea-project/GenAIComps), the epic efforts aimed at reduceing redundancy, addressing technical debt, and enhancing overall maintainability and code quality. As a result, OPEA users can expect a more robust and reliable OPEA with clearer guidance and improved documentation. + +OPEA v1.2 also introduces more scenarios with general availability, including: +- [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. +- Model Context Protocol(MCP) Support: Experimental support for MCP at Retriever. +- Helm Charts Deployment: Add supports for 3 additional GenAIExamples and their microservices. +- Enhanced Security: Istio Mutual TLS (mTLS) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). +- Enhancements for China Ecosystem: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). + + +### Highlights + +#### Code Factoring for GenAIComps +This is an epic task in v1.2. We refactored the entire [GenAIComps](https://github.com/opea-project/GenAIComps) codebase. This comprehensive effort focused on reducing redundancy, addressing accumulated technical debt, and enhancing the overall maintainability and code quality. The refactoring not only streamlined the architecture but also laid a stronger foundation for future scalability and development. + +At the architecture level, OPEA introduces `OpeaComponentRegistry` and `OpeaComponentLoader`. The OpeaComponentRegistry manages the lifecycle of component classes, including their registration and deregistration, while the OpeaComponentLoader instantiates components based on the classes in the registry and execute as needed. Unlike previous implementations, this approach ensures that the lifecycle of a component class is transparent to the user, and components are instantiated only when actively used. This design enhances efficiency, clarity, and flexibility in the system. + +At the component level, each OPEA component is structured into two layers: the `service wrapper` and the `service provider` (named as integrations in the code). The service wrapper, which is optional, manages service access, while the service provider delivers the actual functionality. This architecture allows components to be seamlessly integrated or removed without requiring code changes, enabling a modular and adaptable system. All the existing components have ported to the new architecture. + +Additionally, we reduced code redundancy, merged overlapping modules, and implemented adjustments to align with the new architectural changes. For users and contributors, please read the [documentation](#removals-merges-deprecations-and-behavior-changes) for the impactions from these changes. + + +#### New GenAI Components +- [Bedrock](https://github.com/opea-project/GenAIComps/pull/1031): OPEA LLM now supports [Amazon Bedrock](https://aws.amazon.com/bedrock/) as the backend of the text generation microservice. Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. +- [OpenSearch Vector Database](https://github.com/opea-project/GenAIComps/pull/1024): OPEA vectorstores now supports [AWS OpenSearch](https://aws.amazon.com/opensearch-service/). OpenSearch is an open-source, enterprise-grade search and observability suite that brings order to unstructured data at scale. +- [Elasticsearch Vector Database](https://github.com/opea-project/GenAIComps/pull/972): OPEA vectorestores now supports [Elasticsearch vector database](https://www.elastic.co/elasticsearch/vector-database), Elasticsearch's open source vector database offering an efficient way to create, store, and search vector embeddings. + + +#### Enhanced GenAI Components +- [Embedding and Reranking](https://github.com/opea-project/GenAIComps/issues/956) (TODO: update link if ready)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. +- Agent Microservice: + - SQL agent strategy: Take user question, hints (optional) and history (when available), and thinks step by step to solve the problem by interacting with a SQL database. OPEA currently has two types of [SQL agents](https://github.com/opea-project/GenAIComps/pull/1039): `sql_agent_llama` for using with open-source LLMs and `sql_agent`: for using with OpenAI models. + - Enabled user-customized tool subsets: Added support for [user-defined subsets of tools](https://github.com/opea-project/GenAIComps/issues/978) for the ChatCompletion API and Assistant APIs. + - Enabled persistence: Introduced Redis to persist Agent configurations and historical messages for Agent recovery and multi-turn conversations. +- [Standalone Microservice Deployment]() (TODO: update link if ready): Enabled the deployment of OPEA components as independent services, allowing for greater flexibility, scalability, and modularity in various application scenarios. + +#### Enhanced GenAI Examples +- ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203) (TODO: update link if ready). +- AgentQnA: Added [SQL worker agent](https://github.com/opea-project/GenAIExamples/pull/1370); Introduced a [Svelte-based](https://github.com/opea-project/GenAIExamples/pull/1389) GUI for ChatCompletion API for non-streaming interactions. + + +#### Newly Supported Models +- bge-base-zh-v1.5 +- Falcon2-40B/11B +- Falcon3 + +#### Newly Supported Hardware (TODO: update after PR merges) +- AMD® GPU: Enabled GenAI examples on AMD® GPUs using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AvatarChatbot](https://github.com/opea-project/GenAIExamples/pull/1288), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [DBQnA](https://github.com/opea-project/GenAIExamples/pull/1273), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). + +### Notable Changes (TODO: WIP) + +
GenAIExamples + +- Functionalities + - New GenAI Examples + - [AvatarChatbot] Initiate "AvatarChatbot" (audio) example ([cfffb4c](https://github.com/opea-project/GenAIExamples/commit/cfffb4c), [960805a](https://github.com/opea-project/GenAIExamples/commit/960805a)) + - [DBQnA] Adding DBQnA example in GenAIExamples ([c0643b7](https://github.com/opea-project/GenAIExamples/commit/c0643b7), [6b9a27d](https://github.com/opea-project/GenAIExamples/commit/6b9a27d)) + - [EdgeCraftRag] Add EdgeCraftRag as a GenAIExample ([c9088eb](https://github.com/opea-project/GenAIExamples/commit/c9088eb), [7949045](https://github.com/opea-project/GenAIExamples/commit/7949045), [096a37a](https://github.com/opea-project/GenAIExamples/commit/096a37a)) + - [GraphRAG] Add GraphRAG example [a65640b](https://github.com/opea-project/GenAIExamples/commit/a65640b) + - [Text2Image]: Add example for text2image [085d859](https://github.com/opea-project/GenAIExamples/commit/085d859) + - [WorkflowExecAgent] Add Workflow Executor Example [bf5c391](https://github.com/opea-project/GenAIExamples/commit/bf5c391) + - Enhanced GenAI Examples + - [AudioQnA] Add multi-language AudioQnA on Xeon [658867f](https://github.com/opea-project/GenAIExamples/commit/658867f) + - [AgentQnA] Update AgentQnA example for v1.1 release [5eb3d28](https://github.com/opea-project/GenAIExamples/commit/5eb3d28) + - [ChatQnA] Enable vLLM Profiling for ChatQnA ([00d9bb6](https://github.com/opea-project/GenAIExamples/commit/00d9bb6), [7adbba6](https://github.com/opea-project/GenAIExamples/commit/7adbba6)) + - [ChatQnA] Add Terraform and Ansible Modules information [7c9ed04](https://github.com/opea-project/GenAIExamples/commit/7c9ed04) + - [ChatQnA] Add chatqna wrapper for multiple model selection [fb514bb](https://github.com/opea-project/GenAIExamples/commit/fb514bb) + - [DocSum] Supported multimedia and added new GUI powered by gradio ([eb91d1f](https://github.com/opea-project/GenAIExamples/commit/eb91d1f), [0cdeb94](https://github.com/opea-project/GenAIExamples/commit/0cdeb94)) + - [DocSum] Support Chinese for Docsum [b0f7c9c](https://github.com/opea-project/GenAIExamples/commit/b0f7c9c) + - [DocIndexRetriever] Update DocIndexRetriever Example to allow user passing in retriever/reranker params [62e06a0](https://github.com/opea-project/GenAIExamples/commit/62e06a0) + - [MultimodalQnA] Image and Audio Support Phase 1 [bbc95bb](https://github.com/opea-project/GenAIExamples/commit/bbc95bb) + - [Text2Image] Add Text2Image UI, UI tests, Readme, and Docker support [c6fc92d](https://github.com/opea-project/GenAIExamples/commit/c6fc92d) + - update examples accuracy [088ab98](https://github.com/opea-project/GenAIExamples/commit/088ab98) + - Add one-button benchmark launcher ([5720cd4](https://github.com/opea-project/GenAIExamples/commit/5720cd4), [ced68e1](https://github.com/opea-project/GenAIExamples/commit/ced68e1)) + - Removed GenAI Pipelines + - [ChatQnA] remove ChatQnA vllm-on-ray [40386d9](https://github.com/opea-project/GenAIExamples/commit/40386d9) + - Changed Defaults + - [ChatQnA] Set no wrapper ChatQnA as default [619d941](https://github.com/opea-project/GenAIExamples/commit/619d941) + - [Codegen] Replace codegen default Model to Qwen/Qwen2.5-Coder-7B-Instruct. [2332d22](https://github.com/opea-project/GenAIExamples/commit/2332d22) + - [CodeTrans] update codetrans default model to Mistral-7B-Instruct-v0.3 [a2afce1](https://github.com/opea-project/GenAIExamples/commit/a2afce1) + +- Enhanced Security + - upgrade setuptools version to fix CVE-2024-6345 [2b2c7ee](https://github.com/opea-project/GenAIExamples/commit/2b2c7ee) + +- New Hardware Support + - [ChatQnA] Add compose example for ChatQnA AMD ROCm deployment [6d3a017](https://github.com/opea-project/GenAIExamples/commit/6d3a017) + - [CodeGen] Adding files to deploy CodeGen application on AMD GPU [83172e9](https://github.com/opea-project/GenAIExamples/commit/83172e9) + - [CodeTrans] Adding files to deploy CodeTrans application on AMD GPU [7e62175](https://github.com/opea-project/GenAIExamples/commit/7e62175) + - [DocSum] Add compose example for DocSum amd rocm deployment [b1bb6db](https://github.com/opea-project/GenAIExamples/commit/b1bb6db) + - [FaqGen] Add compose example for FaqGen AMD ROCm [5648839](https://github.com/opea-project/GenAIExamples/commit/5648839) + +- Dependency Versioning + - [gradio] Bump gradio from 4.44.0 to 5.0.0 in /MultimodalQnA/ui/gradio [f2f6c09](https://github.com/opea-project/GenAIExamples/commit/f2f6c09) + - [TGI-CPU] Update TGI CPU image to latest official release 2.4.0-intel-cpu [0306c62](https://github.com/opea-project/GenAIExamples/commit/0306c62) + - [TGI-Gaudi] Upgrade TGI Gaudi version to v2.0.6 [1ff85f6a](https://github.com/opea-project/GenAIExamples/commit/1ff85f6a852785a3abad7a5b286e020592dfd2b7) + - [TEI-Gaudi] Use fixed version(1.5.0) of TEI Gaudi for stability [9ff7df9](https://github.com/opea-project/GenAIExamples/commit/9ff7df9) + - [vLLM-Gaudi] align vllm hpu version to latest vllm-fork [e9b1645](https://github.com/opea-project/GenAIExamples/commit/e9b1645) + +- Deployment + - [ChatQnA] Add instructions of modifying reranking docker image for NVGPU [2587179](https://github.com/opea-project/GenAIExamples/commit/2587179) + - [ChatQnA] setup ollama service in aipc docker compose [def39cf](https://github.com/opea-project/GenAIExamples/commit/def39cf) + - [ChatQnA] Make rerank run on gaudi for hpu docker compose [3c164f3](https://github.com/opea-project/GenAIExamples/commit/3c164f3) + - [ChatQnA] Added the k8s yaml for vLLM support [e2f9037](https://github.com/opea-project/GenAIExamples/commit/e2f9037) + - [ChatQnA] manage your own ChatQnA pipelines. [d16c80e](https://github.com/opea-project/GenAIExamples/commit/d16c80e) + - [ChatQnA] docker install instruction for csp [75df2c9](https://github.com/opea-project/GenAIExamples/commit/75df2c9) + - [ChatQnA] ChatQnA with Remote Inference Endpoints (Kubernetes) [56f770c](https://github.com/opea-project/GenAIExamples/commit/56f770c) + - [ProductivitySuite] Simplify the deployment ProductivitySuite on kubernetes [afc39fa](https://github.com/opea-project/GenAIExamples/commit/afc39fa) + +- Fixed Issues + - [AvatarChatbot] Fix left issue of tgi version update [393367e](https://github.com/opea-project/GenAIExamples/commit/393367e) + - [ChatQnA] Fix the service connection issue on GPU and modify the emb backend [944ae47](https://github.com/opea-project/GenAIExamples/commit/944ae47) + - [ChatQnA] Fix AIPC docker container network issue [95b58b5](https://github.com/opea-project/GenAIExamples/commit/95b58b5) + - [ChatQnA] Fix top_n rerank docs [4a265ab](https://github.com/opea-project/GenAIExamples/commit/4a265ab) + - [ChatQnA] fix chatqna accuracy issue with incorrect penalty [b0487fe](https://github.com/opea-project/GenAIExamples/commit/b0487fe) + - [ChatQnA] Fix AIPC retriever and UI error [773c32b](https://github.com/opea-project/GenAIExamples/commit/773c32b) + - [DocSum] Fix docSum ui error in accessing parsed files [3744bb8](https://github.com/opea-project/GenAIExamples/commit/3744bb8) + - image build bug fix [82801d0](https://github.com/opea-project/GenAIExamples/commit/82801d0) + +- Documentation + - [AudioQnA] Update AudioQnA README.md for its workflow [63bad29](https://github.com/opea-project/GenAIExamples/commit/63bad29) + - [AudioQnA] Update AudioQnA README to add a couple usage details [184e9a4](https://github.com/opea-project/GenAIExamples/commit/184e9a4) + - [AgentQnA] Update Agent README.md for workflow [23b820e](https://github.com/opea-project/GenAIExamples/commit/23b820e) + - [AgentQnA] Update README.md for usage experience [a8f4245](https://github.com/opea-project/GenAIExamples/commit/a8f4245) + - [ChatQnA] Add steps to deploy opea services using minikube [6263b51](https://github.com/opea-project/GenAIExamples/commit/6263b51) + - [ChatQnA] Update ChatQnA Readme for LLM Endpoint [aa314f6](https://github.com/opea-project/GenAIExamples/commit/aa314f6) + - [ChatQnA] Update ChatQnA AIPC README [b056ce6](https://github.com/opea-project/GenAIExamples/commit/b056ce6) + - [CodeGen] Update CodeGen README for its workflow [12469c9](https://github.com/opea-project/GenAIExamples/commit/12469c9) + - [DocSum] Update DocSum README.md for its workflow [fbde15b](https://github.com/opea-project/GenAIExamples/commit/fbde15b) + - [FaqGen] Update FaqGen README.md for its workflow [0c6b044](https://github.com/opea-project/GenAIExamples/commit/0c6b044) + - [InstructionTuning] instruction finetune README improvement [644c3a6](https://github.com/opea-project/GenAIExamples/commit/644c3a6) + - [MultiModalQnA] Update MultiModal README.md for workflow [40800b0](https://github.com/opea-project/GenAIExamples/commit/40800b0) + - [ProductivitySuite] Update Productivity README.md for workflow [0edff26](https://github.com/opea-project/GenAIExamples/commit/0edff26) + - [DocIndexRetriever] Update DocIndexRetriever README.md for workflow [a3f9811](https://github.com/opea-project/GenAIExamples/commit/a3f9811) + - [SearchQnA] Update SearchQnA README.md for its workflow [bf28c7f](https://github.com/opea-project/GenAIExamples/commit/bf28c7f) + - [Translation] Update Translation README.md for workflow [35a4fef](https://github.com/opea-project/GenAIExamples/commit/35a4fef) + - [VideoQnA] Update VideoQnA README.md for workflow [1929dfd](https://github.com/opea-project/GenAIExamples/commit/1929dfd) + +- CI/CD/UT + - Add nightly image build and publish action [78331ee](https://github.com/opea-project/GenAIExamples/commit/78331ee) + - optimize hardware list for test [3b1a9fe](https://github.com/opea-project/GenAIExamples/commit/3b1a9fe) + - open manifest test in CI when dockerfile changed [620ef76](https://github.com/opea-project/GenAIExamples/commit/620ef76) + - Optimize path and link validity check. [7dec001](https://github.com/opea-project/GenAIExamples/commit/7dec001) +
+ +
GenAIComps + +- Functionalities + - New microservices: + - Add stable diffusion microservice [5d0c4367](https://github.com/opea-project/GenAIComps/commit/5d0c4367) + - Add image2video microservice (Stable Video Diffusion) [a03e7a55](https://github.com/opea-project/GenAIComps/commit/a03e7a55) + - Text to SQL microservice [827e3d40](https://github.com/opea-project/GenAIComps/commit/827e3d40) + - Add GPT-SoVITS microservice [6da7db9e](https://github.com/opea-project/GenAIComps/commit/6da7db9e) + - Add image2image microservice [52c1826f](https://github.com/opea-project/GenAIComps/commit/52c1826f) + - Initiate "animation" component [c26d37e7](https://github.com/opea-project/GenAIComps/commit/c26d37e7) + - GraphRAG with llama-index [19330ea2](https://github.com/opea-project/GenAIComps/commit/19330ea2) + - Enhanced microservices: + - Add DPO support in finetuning microservice [37f35140](https://github.com/opea-project/GenAIComps/commit/37f35140) + - Support Chinese for Docsum [9a00a3ea](https://github.com/opea-project/GenAIComps/commit/9a00a3ea) + - Support file upload summary for DocSum microservice [fa2ea642](https://github.com/opea-project/GenAIComps/commit/fa2ea642) + - Add support for Audio and Video summarization to Docsum [baafa402](https://github.com/opea-project/GenAIComps/commit/baafa402) + - vLLM support for FAQGen [f5c60f10](https://github.com/opea-project/GenAIComps/commit/f5c60f10) + - vLLM support for DocSum [550325d8](https://github.com/opea-project/GenAIComps/commit/550325d8) + - vLLM support for Codegen [24b9f03f](https://github.com/opea-project/GenAIComps/commit/24b9f03f) + - Enable vllm for Agent [4638c1d4](https://github.com/opea-project/GenAIComps/commit/4638c1d4) + - Multiple models and remote service support for langchain vLLM text-generation [e3812a74](https://github.com/opea-project/GenAIComps/commit/e3812a74) + - Set a higher default value(1.2) about repetition_penalty for codegen example to reduce repetition [5ed428f4](https://github.com/opea-project/GenAIComps/commit/5ed428f4) + - MultimodalQnA Image and Audio Support Phase 1 [29ef6426](https://github.com/opea-project/GenAIComps/commit/29ef6426) + - refine codetrans prompt, support parameter input [0bb019f8](https://github.com/opea-project/GenAIComps/commit/0bb019f8) + - add dynamic batching embedding/reranking [518cdfb6](https://github.com/opea-project/GenAIComps/commit/518cdfb6) + - Embedding compatible with OpenAI API [7bf1953c](https://github.com/opea-project/GenAIComps/commit/7bf1953c) + - Update RAGAgentLlama and ReActLlama [c8e36390](https://github.com/opea-project/GenAIComps/commit/c8e36390) + - [Agent] support custom prompt [3473bfb3](https://github.com/opea-project/GenAIComps/commit/3473bfb3) + - agent short & long term memory with langgraph. [e39b08f3](https://github.com/opea-project/GenAIComps/commit/e39b08f3) + - support faqgen upload file in UI [453ff726](https://github.com/opea-project/GenAIComps/commit/453ff726) + - Add E2E Prometheus metrics to applications [a6998a1d](https://github.com/opea-project/GenAIComps/commit/a6998a1d) + - Multiple models support for LLM TGI [e879366c](https://github.com/opea-project/GenAIComps/commit/e879366c) + - Add RAG agent and ReAct agent implemention for llama3.1 served by TGI-gaudi [e7fdf537](https://github.com/opea-project/GenAIComps/commit/e7fdf537) + - Support Llama3.2 vision and vision guard model [534c227a](https://github.com/opea-project/GenAIComps/commit/534c227a) + - Add Intel/toxic-prompt-roberta to toxicity detection microservice [f6f620a2](https://github.com/opea-project/GenAIComps/commit/f6f620a2) + - Refactor milvus dataprep and retriever [84374a57](https://github.com/opea-project/GenAIComps/commit/84374a57) + - Removed microservices + - Remove vllm ray [617e119f](https://github.com/opea-project/GenAIComps/commit/617e119f) + - Async support for microservices + - Support async for embedding micorservice [28672956](https://github.com/opea-project/GenAIComps/commit/28672956) + - TEI rerank microservice async support [9df4b3c0](https://github.com/opea-project/GenAIComps/commit/9df4b3c0) + - Async support for some microservices [f3746dc8](https://github.com/opea-project/GenAIComps/commit/f3746dc8) +- Performance + - Fix vllm microservice performance issue. [2159f9ad](https://github.com/opea-project/GenAIComps/commit/2159f9ad) + - [Dataprep] Reduce Upload File Time Consumption [71348998](https://github.com/opea-project/GenAIComps/commit/71348998) +- New Hardware Support + - Add vLLM ARC support with OpenVINO backend [a2b9d95f](https://github.com/opea-project/GenAIComps/commit/a2b9d95f) +- Enhanced Security + - Prediction Guard Guardrails components [4bbc7a2f](https://github.com/opea-project/GenAIComps/commit/4bbc7a2f) + - Add WildGuard Guardrail Microservice [5bb4046b](https://github.com/opea-project/GenAIComps/commit/5bb4046b)F + - upgrade setuptools version to fix CVE-2024-6345 [6518c0f0](https://github.com/opea-project/GenAIComps/commit/6518c0f0) + - Remote TGI/TGI services with OAuth Client Credentials authentication [74df6bb7](https://github.com/opea-project/GenAIComps/commit/74df6bb7) +- Validation + - Combine CI/CD docker compose. [23c99c11](https://github.com/opea-project/GenAIComps/commit/23c99c11704cfcd0b13b3b463819f633ba02b42d) +
+ +
GenAIEvals + +- New Benchmark + - Support HELMET [4c8f048](https://github.com/opea-project/GenAIEval/commit/4c8f048) + - Support Longbench [021193f](https://github.com/opea-project/GenAIEval/commit/021193f) +- Performance + - Add new constant loader & Fix poisson loader issue [e11588c](https://github.com/opea-project/GenAIEval/commit/e11588c) + - Support Poisson distributed requests for benchmark [7305ea3](https://github.com/opea-project/GenAIEval/commit/7305ea3) + - Support customized prompts and max new tokens in chatqna e2e test [79a4ad3](https://github.com/opea-project/GenAIEval/commit/79a4ad3) + - Add namespace support for k8s performance test [70697d1](https://github.com/opea-project/GenAIEval/commit/70697d1) + - Support sharegpt dataset in chatqna e2e test [028bf63](https://github.com/opea-project/GenAIEval/commit/028bf63) + - [Benchmark] Get benchmark reports. [946c439](https://github.com/opea-project/GenAIEval/commit/946c439) +- Accuracy + - Control the concurrent number of requests in codegen acc test. [84e077e](https://github.com/opea-project/GenAIEval/commit/84e077e) + - integrate deepeval metric with remote endpoint, like tgi server. [ffa65dc](https://github.com/opea-project/GenAIEval/commit/ffa65dc) + - Ragaaf - adding new metric 'context recall' [cc7cebd](https://github.com/opea-project/GenAIEval/commit/cc7cebd) + - Ragaaf - adding new metric 'context relevance' [f995c9c](https://github.com/opea-project/GenAIEval/commit/f995c9c) + - Ragaaf (RAG assessment annotation free) [2413e70](https://github.com/opea-project/GenAIEval/commit/2413e70) + - Adding new metrics to ragas offering [d1c1337](https://github.com/opea-project/GenAIEval/commit/d1c1337) + - add crud ragas evaluation. [f2bff45](https://github.com/opea-project/GenAIEval/commit/f2bff45) + - Minimize requirements for user data for OPEA ragas [f1593ea](https://github.com/opea-project/GenAIEval/commit/f1593ea) +- Monitoring + - Add node metrics Grafana dashboard [a19f42e](https://github.com/opea-project/GenAIEval/commit/a19f42e) + - Add CPU Grafana dashboard [38e69eb](https://github.com/opea-project/GenAIEval/commit/38e69eb) + - add the grafana dashboard json file for Gaudi metrics [6c9ae91](https://github.com/opea-project/GenAIEval/commit/6c9ae91) + - Enhance the Grafana JSON file [8653efb](https://github.com/opea-project/GenAIEval/commit/8653efb) +- Fixed Issues + - [ChatQnA Benchmark] Fixed the output token in chatqnafixed.py [2c8ca26](https://github.com/opea-project/GenAIEval/commit/2c8ca26) + - Fix test duration time inaccurate issue [9d76832](https://github.com/opea-project/GenAIEval/commit/9d76832) + - Fix llm output token length issue [99ef325](https://github.com/opea-project/GenAIEval/commit/99ef325) + - Fix llm serving benchmark issue [d6bafbd](https://github.com/opea-project/GenAIEval/commit/d6bafbd) + - Fix input token size(1024) [30adcbe](https://github.com/opea-project/GenAIEval/commit/30adcbe) + - Ragas fix for use of metrics argument [0cf3631](https://github.com/opea-project/GenAIEval/commit/0cf3631) + - fixed the number of ouput token & fixed the top_k=1 [4af0a62](https://github.com/opea-project/GenAIEval/commit/4af0a62) + - Fix JSON Return Format in getReqData Function [a4be366](https://github.com/opea-project/GenAIEval/commit/a4be366) +- Documentation + - Add setup guide of gaudi prometheus exporter [e9b8637](https://github.com/opea-project/GenAIEval/commit/e9b8637) + - Add README for running OPEA ragas using HF endpoint on Gaudi [0dff0d3](https://github.com/opea-project/GenAIEval/commit/0dff0d3) +
+ +
GenAIInfra + +- GMC + - Add manifests for new components [e51fd62](https://github.com/opea-project/GenAIInfra/commit/e51fd62) + +- HelmChart + - [AgentQnA] Helm Chart for AgentQnA [66de41c](https://github.com/opea-project/GenAIInfra/commit/66de41c) + - [AudioQnA] helm: Add audioQnA e2e helm chart [9efacee](https://github.com/opea-project/GenAIInfra/commit/9efacee) + - [AudioQnA] helm-charts: Add gpt-sovits support [1f55e1a](https://github.com/opea-project/GenAIInfra/commit/1f55e1a) + - [ChatQnA] Implement the nowrapper version chatqna [71c81d0](https://github.com/opea-project/GenAIInfra/commit/71c81d0) + - [FaqGen] Add FaqGen helm chart [f847e05](https://github.com/opea-project/GenAIInfra/commit/f847e05) + - [FaqGen] helm: Add llm-faqgen-tgi support [325126e](https://github.com/opea-project/GenAIInfra/commit/325126e) + - [HPA] helm/manifest: Sync HPA related k8s probe settings [c399578](https://github.com/opea-project/GenAIInfra/commit/c399578) + - [VisualQnA] Add helm chart for VisualQnA example [b077d44](https://github.com/opea-project/GenAIInfra/commit/b077d44) + - [UI] support variants for multiple examples [96af2ad](https://github.com/opea-project/GenAIInfra/commit/96af2ad) + - [Nginx] helm-chart: Make nginx service type configurable [a5c96ab](https://github.com/opea-project/GenAIInfra/commit/a5c96ab) + - [Milvus] Add milvus support for data-prep and retriever-usvc [d289b4e](https://github.com/opea-project/GenAIInfra/commit/d289b4e) + - Add helm chart for 3 components [881e2b5](https://github.com/opea-project/GenAIInfra/commit/881e2b5) + - accelerate also teirerank with Gaudi [620963f](https://github.com/opea-project/GenAIInfra/commit/620963f) + +- CSP + - terraform: add AWS/EKS deployment for ChatQnA [bdb9af9](https://github.com/opea-project/GenAIInfra/commit/bdb9af9) + +- Monitoring + - Add Grafana dashboard for monitoring OPEA application scaling in k8s [691bbc5](https://github.com/opea-project/GenAIInfra/commit/691bbc5) + - Add ServiceMonitors for rest of OPEA applications [fc6235a](https://github.com/opea-project/GenAIInfra/commit/fc6235a) + - Add monitoring option to (ChatQnA) Helm charts [dbd607e](https://github.com/opea-project/GenAIInfra/commit/dbd607e) + - Support alternative metrics on accelerated TGI / TEI instances [cdd3585](https://github.com/opea-project/GenAIInfra/commit/cdd3585) + - Expose options such as collector.interval of memory bandwidth exporter in k8s manifests and docker for user configuration. [2517e79](https://github.com/opea-project/GenAIInfra/commit/2517e79) + +- Dependency Versioning + - [TEI-Gaudi] Upgrade tei-gaudi version to 1.5.0 [c6a9c90](https://github.com/opea-project/GenAIInfra/commit/c6a9c90) + - [TGI-CPU] Update tgi cpu image version to 2.4.0-intel-cpu [f6c180e](https://github.com/opea-project/GenAIInfra/commit/f6c180e) + - [TGI-Gaudi] Upgrade tgi-gaudi to version 2.0.6 [915baa0](https://github.com/opea-project/GenAIInfra/commit/915baa0) + - Update the image version for ChatQnA examples [593458c](https://github.com/opea-project/GenAIInfra/commit/593458c) + +- Changed Defaults + - Change default model of codegen and codetrans [74476b7](https://github.com/opea-project/GenAIInfra/commit/74476b7) + +- Documentation + - Update observability README + fix typos [1d77b81](https://github.com/opea-project/GenAIInfra/commit/1d77b81) + - Monitoring, Observability and HPA doc improvements [14198fe](https://github.com/opea-project/GenAIInfra/commit/14198fe) + - Update GMC manifest changes and misc fixes [87dc673](https://github.com/opea-project/GenAIInfra/commit/87dc673) + - Improve Helm charts README [7b8c510](https://github.com/opea-project/GenAIInfra/commit/7b8c510) + - Create troubleshooting.md [d55ded4](https://github.com/opea-project/GenAIInfra/commit/d55ded4) + - Enhance helm chart repo usage in README [0de5535](https://github.com/opea-project/GenAIInfra/commit/0de5535) + +- CI/CD/UT + - Refactor CI scripts to support more components [e09270a](https://github.com/opea-project/GenAIInfra/commit/e09270a) + - Add github workflows to release helm chart [3910e3b](https://github.com/opea-project/GenAIInfra/commit/3910e3b) + - Fix link check failure (#481) (5 weeks ago) [fc87ef3](https://github.com/opea-project/GenAIInfra/commit/fc87ef3) + - Fix CI failures (#477) (5 weeks ago) [7e7b8ab](https://github.com/opea-project/GenAIInfra/commit/7e7b8ab) + - Optimize path and link validity check. [91bd163](https://github.com/opea-project/GenAIInfra/commit/91bd163) + -Enable image build process for memory-bandwidth-exporter [ddeac46](https://github.com/opea-project/GenAIInfra/commit/ddeac46) + - Add hyperlinks and paths validation. [d8cd3a1](https://github.com/opea-project/GenAIInfra/commit/d8cd3a1) +
+ +### Full Changelogs +- GenAIExamples: [v1.1...v1.2](https://github.com/opea-project/GenAIExamples/compare/v1.1...v1.2) +- GenAIComps: [v1.1...v1.2](https://github.com/opea-project/GenAIComps/compare/v1.1...v1.2) +- GenAIInfra: [v1.1...v1.2](https://github.com/opea-project/GenAIInfra/compare/v1.1...v1.2) +- GenAIEval: [v1.1...v1.2](https://github.com/opea-project/GenAIEval/compare/v1.1...v1.2) +- GenAIStudio: [v1.1...v1.2](https://github.com/opea-project/GenAIStudio/compare/v1.1...v1.2) + +## Removals, Merges, Deprecations and Behavior Changes + +### GenAIComps + +#### Removals +- Remove embedding microservices: `MOSEC` ([#1072](https://github.com/opea-project/GenAIComps/pull/1072)). +- Remove reranking microservices: `fastRAG`, `MOSEC` ([#1003](https://github.com/opea-project/GenAIComps/pull/1003)). +- Remove vector store microservices: `LanceDB`, `Chroma`. +- Remove `intent_detection` microservice. +- Remove module `cores/mega/gateway`. + +#### Merges +- `TGI`, `vLLM`, and `Ollama` integrations in LLM/text-generation have been merged into `LLM/text-generation/native`. + +#### Renamings +- Rename `comps/reranks` to `comps/rerankings`. + +### Docker Images + +#### Deprecations +The following Docker images are deprecated, and will be removed since OPEA v1.3: +- [opea/embedding-langchain-mosec](https://hub.docker.com/r/opea/embedding-langchain-mosec) +- [opea/embedding-langchain-mosec-endpoint](https://hub.docker.com/r/opea/embedding-langchain-mosec-endpoint) +- [opea/llm-vllm-hpu](https://hub.docker.com/r/opea/llm-vllm-hpu) +- [opea/llm-ray](https://hub.docker.com/r/opea/llm-ray) + +- [opea/reranking-langchain-mosec](https://hub.docker.com/r/opea/reranking-langchain-mosec) +- [opea/reranking-langchain-mosec-endpoint](https://hub.docker.com/r/opea/reranking-langchain-mosec-endpoint) +- [opea/reranking-fastrag](https://hub.docker.com/r/opea/reranking-fastrag) +- [opea/web-retriever-chroma](https://hub.docker.com/r/opea/web-retriever-chroma) + +#### Merges +The merged Docker images are deprecated, and will be removed since OPEA v1.3. Please use the new images instead. +- Merge [opea/embedding-tei](https://hub.docker.com/r/opea/embedding-tei) to [opea/embedding](https://hub.docker.com/r/opea/embedding) +- Merge [opea/llm-ollama](https://hub.docker.com/r/opea/llm-ollama) to []() (TODO: update if finalized) + +#### Renamings +The Docker images marked for renaming are now deprecated and will be removed since OPEA v1.3. Please use the renamed images instead. +- Rename [opea/embedding-multimodal-bridgetower-gaudi](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-gaudi) to [opea/embedding-multimodal-bridgetower-hpu](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-hpu) + +### GenAIExamples + +#### Behavior Changes +- ChatQnA: Use vLLM as the default serving framework on Gaudi to leverage its optimized performance characteristics, thereby improving throughput and reducing latency in inference tasks. + +## Contributors +This release would not have been possible without the contributions of the following organizations and individuals. + +### Contributing Organizations +- `Amazon`: AMD CPU/GPU support for GenAIExamples. +- `AMD`: Bedrock and OpenSearch vector database integration. +- `Infosys`: Elasticsearch vector database integration. +- `Intel`: Development and improvements to GenAI examples, components, infrastructure, and evaluation. + +### Individual Contributors +For a comprehensive list of individual contributors, please refer to the "Full Changelogs" section. \ No newline at end of file From 359f501633438e7ee4c5a073b6e23cbc73505d9f Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 14:05:35 +0800 Subject: [PATCH 02/14] Add notable changes for v1.2 release notes. --- release_notes/v1.2.md | 405 +++++++++++++++++++----------------------- 1 file changed, 180 insertions(+), 225 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 0d6c4f28..13f08f94 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -1,5 +1,5 @@ # OPEA Release Notes v1.2 -We are excited to announce the release of OPEA version 1.2, which includes significant contributions from the open-source community. This release addresses over 245(TODO: update the latest) pull requests. +We are excited to announce the release of OPEA version 1.2, which includes significant contributions from the open-source community. This release addresses over 263(TODO: update the latest) pull requests. More information about how to get started with OPEA v1.2 can be found at [Getting Started](https://opea-project.github.io/latest/index.html) page. All project source code is maintained in the [repository](https://github.com/opea-project). To pull Docker images, please access the [Docker Hub](https://hub.docker.com/u/opea). For instructions on deploying Helm Charts, please refer to the [guide](https://github.com/opea-project/GenAIInfra/tree/v1.2/helm-charts#readme). @@ -9,10 +9,9 @@ This release focuses on code refactoring for [GenAIComps](https://github.com/ope OPEA v1.2 also introduces more scenarios with general availability, including: - [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. - Model Context Protocol(MCP) Support: Experimental support for MCP at Retriever. -- Helm Charts Deployment: Add supports for 3 additional GenAIExamples and their microservices. - Enhanced Security: Istio Mutual TLS (mTLS) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). - Enhancements for China Ecosystem: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). - +- Helm Charts Deployment: Add supports for the examples Text2Image, SearchQnA, EdgeCraftRag and their microservices. ### Highlights @@ -21,284 +20,218 @@ This is an epic task in v1.2. We refactored the entire [GenAIComps](https://gith At the architecture level, OPEA introduces `OpeaComponentRegistry` and `OpeaComponentLoader`. The OpeaComponentRegistry manages the lifecycle of component classes, including their registration and deregistration, while the OpeaComponentLoader instantiates components based on the classes in the registry and execute as needed. Unlike previous implementations, this approach ensures that the lifecycle of a component class is transparent to the user, and components are instantiated only when actively used. This design enhances efficiency, clarity, and flexibility in the system. -At the component level, each OPEA component is structured into two layers: the `service wrapper` and the `service provider` (named as integrations in the code). The service wrapper, which is optional, manages service access, while the service provider delivers the actual functionality. This architecture allows components to be seamlessly integrated or removed without requiring code changes, enabling a modular and adaptable system. All the existing components have ported to the new architecture. +At the component level, each OPEA component is structured into two layers: the `service wrapper` and the `service provider` (named as integrations in the code). The service wrapper, which is optional, acts as a protocol hub and manages service access, while the service provider delivers the actual functionality. This architecture allows components to be seamlessly integrated or removed without requiring code changes, enabling a modular and adaptable system. All the existing components have ported to the new architecture. + +Additionally, we reduced code redundancy, merged overlapping modules, and implemented adjustments to align with the new architectural changes. -Additionally, we reduced code redundancy, merged overlapping modules, and implemented adjustments to align with the new architectural changes. For users and contributors, please read the [documentation](#removals-merges-deprecations-and-behavior-changes) for the impactions from these changes. +> [!NOTE] +> We suggest users and contributors to review the [documentation](#removals-merges-deprecations-and-behavior-changes) to understand the impacts of the code refactoring. #### New GenAI Components - [Bedrock](https://github.com/opea-project/GenAIComps/pull/1031): OPEA LLM now supports [Amazon Bedrock](https://aws.amazon.com/bedrock/) as the backend of the text generation microservice. Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. - [OpenSearch Vector Database](https://github.com/opea-project/GenAIComps/pull/1024): OPEA vectorstores now supports [AWS OpenSearch](https://aws.amazon.com/opensearch-service/). OpenSearch is an open-source, enterprise-grade search and observability suite that brings order to unstructured data at scale. - [Elasticsearch Vector Database](https://github.com/opea-project/GenAIComps/pull/972): OPEA vectorestores now supports [Elasticsearch vector database](https://www.elastic.co/elasticsearch/vector-database), Elasticsearch's open source vector database offering an efficient way to create, store, and search vector embeddings. +- [Guardrail Hallucination Detection](https://github.com/opea-project/GenAIComps/pull/1108): The capability of detecting `Hallucination` which spans a wide range of issues that can impact reliability, trustworthiness, and utility of AI-generated content. #### Enhanced GenAI Components -- [Embedding and Reranking](https://github.com/opea-project/GenAIComps/issues/956) (TODO: update link if ready)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. +- vLLM support for embeddings and rerankings](https://github.com/opea-project/GenAIComps/issues/956)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. - Agent Microservice: - SQL agent strategy: Take user question, hints (optional) and history (when available), and thinks step by step to solve the problem by interacting with a SQL database. OPEA currently has two types of [SQL agents](https://github.com/opea-project/GenAIComps/pull/1039): `sql_agent_llama` for using with open-source LLMs and `sql_agent`: for using with OpenAI models. - Enabled user-customized tool subsets: Added support for [user-defined subsets of tools](https://github.com/opea-project/GenAIComps/issues/978) for the ChatCompletion API and Assistant APIs. - Enabled persistence: Introduced Redis to persist Agent configurations and historical messages for Agent recovery and multi-turn conversations. -- [Standalone Microservice Deployment]() (TODO: update link if ready): Enabled the deployment of OPEA components as independent services, allowing for greater flexibility, scalability, and modularity in various application scenarios. +- Long-context Summarization: Supported multiple modes: [`auto`](https://github.com/opea-project/GenAIComps/commit/5aba3b25), [`stuff`, `truncate`, `map_reduce`, and `refine`](https://github.com/opea-project/GenAIComps/commit/45d00020). +- Standalone Microservice Deployment: Enabled the deployment of OPEA components as independent services, allowing for greater flexibility, scalability, and modularity in various application scenarios. +- [PDF Inputs Support](https://github.com/opea-project/GenAIComps/pull/1134): Support PDF inputs for dataprep, embeddings, lvms, and retrievers. #### Enhanced GenAI Examples -- ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203) (TODO: update link if ready). -- AgentQnA: Added [SQL worker agent](https://github.com/opea-project/GenAIExamples/pull/1370); Introduced a [Svelte-based](https://github.com/opea-project/GenAIExamples/pull/1389) GUI for ChatCompletion API for non-streaming interactions. - +- ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203), and Enable [Jaeger UI and OpenTelemetry tracing](https://github.com/opea-project/GenAIExamples/pull/1316) for TGI serving on HPU. +- AgentQnA: Added [SQL worker agent](https://github.com/opea-project/GenAIExamples/pull/1370) and introduced a [Svelte-based GUI](https://github.com/opea-project/GenAIExamples/pull/1389) for ChatCompletion API for non-streaming interactions. +- MultimodalQnA: Supported [PDF](https://github.com/opea-project/GenAIExamples/pull/1381) and [audio](https://github.com/opea-project/GenAIExamples/pull/1225) inputs. +- EdgeCraftRAG: Supported image/url data retrieval and display, display of LLM-used context sources in UI, pipeline remove operation in RESTful API and UI, RAG pipeline performance benchmark and display in UI. ([#GenAIExamples/1324](https://github.com/opea-project/GenAIExamples/pull/1324)) +- DocSum: Adding [URL summary option](https://github.com/opea-project/GenAIExamples/pull/1248) to Gradio-based UI. +- DocIndexRetriever: Add the pipeline without Reranking. #### Newly Supported Models - bge-base-zh-v1.5 - Falcon2-40B/11B - Falcon3 -#### Newly Supported Hardware (TODO: update after PR merges) -- AMD® GPU: Enabled GenAI examples on AMD® GPUs using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AvatarChatbot](https://github.com/opea-project/GenAIExamples/pull/1288), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [DBQnA](https://github.com/opea-project/GenAIExamples/pull/1273), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). +#### Newly Supported Hardware (TODO: update after code freeze) +- AMD® GPU: Enabled GenAI examples on AMD® GPUs using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA?](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). -### Notable Changes (TODO: WIP) +### Notable Changes
GenAIExamples - Functionalities - New GenAI Examples - - [AvatarChatbot] Initiate "AvatarChatbot" (audio) example ([cfffb4c](https://github.com/opea-project/GenAIExamples/commit/cfffb4c), [960805a](https://github.com/opea-project/GenAIExamples/commit/960805a)) - - [DBQnA] Adding DBQnA example in GenAIExamples ([c0643b7](https://github.com/opea-project/GenAIExamples/commit/c0643b7), [6b9a27d](https://github.com/opea-project/GenAIExamples/commit/6b9a27d)) - - [EdgeCraftRag] Add EdgeCraftRag as a GenAIExample ([c9088eb](https://github.com/opea-project/GenAIExamples/commit/c9088eb), [7949045](https://github.com/opea-project/GenAIExamples/commit/7949045), [096a37a](https://github.com/opea-project/GenAIExamples/commit/096a37a)) - - [GraphRAG] Add GraphRAG example [a65640b](https://github.com/opea-project/GenAIExamples/commit/a65640b) - - [Text2Image]: Add example for text2image [085d859](https://github.com/opea-project/GenAIExamples/commit/085d859) - - [WorkflowExecAgent] Add Workflow Executor Example [bf5c391](https://github.com/opea-project/GenAIExamples/commit/bf5c391) + - Enhanced GenAI Examples - - [AudioQnA] Add multi-language AudioQnA on Xeon [658867f](https://github.com/opea-project/GenAIExamples/commit/658867f) - - [AgentQnA] Update AgentQnA example for v1.1 release [5eb3d28](https://github.com/opea-project/GenAIExamples/commit/5eb3d28) - - [ChatQnA] Enable vLLM Profiling for ChatQnA ([00d9bb6](https://github.com/opea-project/GenAIExamples/commit/00d9bb6), [7adbba6](https://github.com/opea-project/GenAIExamples/commit/7adbba6)) - - [ChatQnA] Add Terraform and Ansible Modules information [7c9ed04](https://github.com/opea-project/GenAIExamples/commit/7c9ed04) - - [ChatQnA] Add chatqna wrapper for multiple model selection [fb514bb](https://github.com/opea-project/GenAIExamples/commit/fb514bb) - - [DocSum] Supported multimedia and added new GUI powered by gradio ([eb91d1f](https://github.com/opea-project/GenAIExamples/commit/eb91d1f), [0cdeb94](https://github.com/opea-project/GenAIExamples/commit/0cdeb94)) - - [DocSum] Support Chinese for Docsum [b0f7c9c](https://github.com/opea-project/GenAIExamples/commit/b0f7c9c) - - [DocIndexRetriever] Update DocIndexRetriever Example to allow user passing in retriever/reranker params [62e06a0](https://github.com/opea-project/GenAIExamples/commit/62e06a0) - - [MultimodalQnA] Image and Audio Support Phase 1 [bbc95bb](https://github.com/opea-project/GenAIExamples/commit/bbc95bb) - - [Text2Image] Add Text2Image UI, UI tests, Readme, and Docker support [c6fc92d](https://github.com/opea-project/GenAIExamples/commit/c6fc92d) - - update examples accuracy [088ab98](https://github.com/opea-project/GenAIExamples/commit/088ab98) - - Add one-button benchmark launcher ([5720cd4](https://github.com/opea-project/GenAIExamples/commit/5720cd4), [ced68e1](https://github.com/opea-project/GenAIExamples/commit/ced68e1)) - - Removed GenAI Pipelines - - [ChatQnA] remove ChatQnA vllm-on-ray [40386d9](https://github.com/opea-project/GenAIExamples/commit/40386d9) + - [AgentQnA] Add SQL agent to AgentQnA. ([287f03a8](https://github.com/opea-project/GenAIExamples/commit/287f03a8)) + - [ChatQnA] Enable OpenTelemetry Tracing for ChatQnA TGI serving on Gaudi. ([81022355](https://github.com/opea-project/GenAIExamples/commit/81022355)) + - [DocIndexRetriever] Enable the without-rerank flavor. ([a50e4e6f](https://github.com/opea-project/GenAIExamples/commit/a50e4e6f)) + - [DocSum] Support Long context for DocSum. ([50dd959d](https://github.com/opea-project/GenAIExamples/commit/50dd959d)) + - [DocSum] Adding URL summary option to DocSum Gradio-UI. ([84a6a6e9](https://github.com/opea-project/GenAIExamples/commit/84a6a6e9)) + - [EdgeCraftRAG] Add new feature and bug fix for EC-RAG. ([6169ea49](https://github.com/opea-project/GenAIExamples/commit/6169ea49)) + - [MultimodalQnA] Adds audio querying to MultimodalQ&A Example. ([c760cac2](https://github.com/opea-project/GenAIExamples/commit/c760cac2)) + - Changed Defaults - - [ChatQnA] Set no wrapper ChatQnA as default [619d941](https://github.com/opea-project/GenAIExamples/commit/619d941) - - [Codegen] Replace codegen default Model to Qwen/Qwen2.5-Coder-7B-Instruct. [2332d22](https://github.com/opea-project/GenAIExamples/commit/2332d22) - - [CodeTrans] update codetrans default model to Mistral-7B-Instruct-v0.3 [a2afce1](https://github.com/opea-project/GenAIExamples/commit/a2afce1) + - [DocSum] Changed Default UI to Gradio. ([00b526c8](https://github.com/opea-project/GenAIExamples/commit/00b526c8)) -- Enhanced Security - - upgrade setuptools version to fix CVE-2024-6345 [2b2c7ee](https://github.com/opea-project/GenAIExamples/commit/2b2c7ee) +- Performance + - [ChatQnA] Remove enforce-eager to enable HPU graphs for better vLLM perf. ([4c01e146](https://github.com/opea-project/GenAIExamples/commit/4c01e146)) - New Hardware Support - - [ChatQnA] Add compose example for ChatQnA AMD ROCm deployment [6d3a017](https://github.com/opea-project/GenAIExamples/commit/6d3a017) - - [CodeGen] Adding files to deploy CodeGen application on AMD GPU [83172e9](https://github.com/opea-project/GenAIExamples/commit/83172e9) - - [CodeTrans] Adding files to deploy CodeTrans application on AMD GPU [7e62175](https://github.com/opea-project/GenAIExamples/commit/7e62175) - - [DocSum] Add compose example for DocSum amd rocm deployment [b1bb6db](https://github.com/opea-project/GenAIExamples/commit/b1bb6db) - - [FaqGen] Add compose example for FaqGen AMD ROCm [5648839](https://github.com/opea-project/GenAIExamples/commit/5648839) + - Added compose example for MultimodalQnA deployment on AMD ROCm systems. ([236ea6bc](https://github.com/opea-project/GenAIExamples/commit/236ea6bc)) + - Added docker compose example for AgentQnA deployment on AMD ROCm. ([df7c1928](https://github.com/opea-project/GenAIExamples/commit/df7c1928)) + - Added compose example for VisualQnA deployment on AMD ROCm systems. ([77e640e2](https://github.com/opea-project/GenAIExamples/commit/77e640e2)) -- Dependency Versioning - - [gradio] Bump gradio from 4.44.0 to 5.0.0 in /MultimodalQnA/ui/gradio [f2f6c09](https://github.com/opea-project/GenAIExamples/commit/f2f6c09) - - [TGI-CPU] Update TGI CPU image to latest official release 2.4.0-intel-cpu [0306c62](https://github.com/opea-project/GenAIExamples/commit/0306c62) - - [TGI-Gaudi] Upgrade TGI Gaudi version to v2.0.6 [1ff85f6a](https://github.com/opea-project/GenAIExamples/commit/1ff85f6a852785a3abad7a5b286e020592dfd2b7) - - [TEI-Gaudi] Use fixed version(1.5.0) of TEI Gaudi for stability [9ff7df9](https://github.com/opea-project/GenAIExamples/commit/9ff7df9) - - [vLLM-Gaudi] align vllm hpu version to latest vllm-fork [e9b1645](https://github.com/opea-project/GenAIExamples/commit/e9b1645) - - Deployment - - [ChatQnA] Add instructions of modifying reranking docker image for NVGPU [2587179](https://github.com/opea-project/GenAIExamples/commit/2587179) - - [ChatQnA] setup ollama service in aipc docker compose [def39cf](https://github.com/opea-project/GenAIExamples/commit/def39cf) - - [ChatQnA] Make rerank run on gaudi for hpu docker compose [3c164f3](https://github.com/opea-project/GenAIExamples/commit/3c164f3) - - [ChatQnA] Added the k8s yaml for vLLM support [e2f9037](https://github.com/opea-project/GenAIExamples/commit/e2f9037) - - [ChatQnA] manage your own ChatQnA pipelines. [d16c80e](https://github.com/opea-project/GenAIExamples/commit/d16c80e) - - [ChatQnA] docker install instruction for csp [75df2c9](https://github.com/opea-project/GenAIExamples/commit/75df2c9) - - [ChatQnA] ChatQnA with Remote Inference Endpoints (Kubernetes) [56f770c](https://github.com/opea-project/GenAIExamples/commit/56f770c) - - [ProductivitySuite] Simplify the deployment ProductivitySuite on kubernetes [afc39fa](https://github.com/opea-project/GenAIExamples/commit/afc39fa) - -- Fixed Issues - - [AvatarChatbot] Fix left issue of tgi version update [393367e](https://github.com/opea-project/GenAIExamples/commit/393367e) - - [ChatQnA] Fix the service connection issue on GPU and modify the emb backend [944ae47](https://github.com/opea-project/GenAIExamples/commit/944ae47) - - [ChatQnA] Fix AIPC docker container network issue [95b58b5](https://github.com/opea-project/GenAIExamples/commit/95b58b5) - - [ChatQnA] Fix top_n rerank docs [4a265ab](https://github.com/opea-project/GenAIExamples/commit/4a265ab) - - [ChatQnA] fix chatqna accuracy issue with incorrect penalty [b0487fe](https://github.com/opea-project/GenAIExamples/commit/b0487fe) - - [ChatQnA] Fix AIPC retriever and UI error [773c32b](https://github.com/opea-project/GenAIExamples/commit/773c32b) - - [DocSum] Fix docSum ui error in accessing parsed files [3744bb8](https://github.com/opea-project/GenAIExamples/commit/3744bb8) - - image build bug fix [82801d0](https://github.com/opea-project/GenAIExamples/commit/82801d0) + - Use staged builds to minimize final image sizes. ([0eae391f](https://github.com/opea-project/GenAIExamples/commit/0eae391f)) + - Check duplicated dockerfile. ([aa5c91d7](https://github.com/opea-project/GenAIExamples/commit/aa5c91d7)) + - Add helm deployment instructions for GenAIExamples. ([c795ef22](https://github.com/opea-project/GenAIExamples/commit/c795ef22)) + - Add helm deployment instructions for codegen. ([5638075d](https://github.com/opea-project/GenAIExamples/commit/5638075d)) + +- Versioning + - Remove vllm hpu commit id limit. ([7d218b9f](https://github.com/opea-project/GenAIExamples/commit/7d218b9f)) + +- Bug Fixes + - FaqGen param fix. ([05365b61](https://github.com/opea-project/GenAIExamples/commit/05365b61)) - Documentation - - [AudioQnA] Update AudioQnA README.md for its workflow [63bad29](https://github.com/opea-project/GenAIExamples/commit/63bad29) - - [AudioQnA] Update AudioQnA README to add a couple usage details [184e9a4](https://github.com/opea-project/GenAIExamples/commit/184e9a4) - - [AgentQnA] Update Agent README.md for workflow [23b820e](https://github.com/opea-project/GenAIExamples/commit/23b820e) - - [AgentQnA] Update README.md for usage experience [a8f4245](https://github.com/opea-project/GenAIExamples/commit/a8f4245) - - [ChatQnA] Add steps to deploy opea services using minikube [6263b51](https://github.com/opea-project/GenAIExamples/commit/6263b51) - - [ChatQnA] Update ChatQnA Readme for LLM Endpoint [aa314f6](https://github.com/opea-project/GenAIExamples/commit/aa314f6) - - [ChatQnA] Update ChatQnA AIPC README [b056ce6](https://github.com/opea-project/GenAIExamples/commit/b056ce6) - - [CodeGen] Update CodeGen README for its workflow [12469c9](https://github.com/opea-project/GenAIExamples/commit/12469c9) - - [DocSum] Update DocSum README.md for its workflow [fbde15b](https://github.com/opea-project/GenAIExamples/commit/fbde15b) - - [FaqGen] Update FaqGen README.md for its workflow [0c6b044](https://github.com/opea-project/GenAIExamples/commit/0c6b044) - - [InstructionTuning] instruction finetune README improvement [644c3a6](https://github.com/opea-project/GenAIExamples/commit/644c3a6) - - [MultiModalQnA] Update MultiModal README.md for workflow [40800b0](https://github.com/opea-project/GenAIExamples/commit/40800b0) - - [ProductivitySuite] Update Productivity README.md for workflow [0edff26](https://github.com/opea-project/GenAIExamples/commit/0edff26) - - [DocIndexRetriever] Update DocIndexRetriever README.md for workflow [a3f9811](https://github.com/opea-project/GenAIExamples/commit/a3f9811) - - [SearchQnA] Update SearchQnA README.md for its workflow [bf28c7f](https://github.com/opea-project/GenAIExamples/commit/bf28c7f) - - [Translation] Update Translation README.md for workflow [35a4fef](https://github.com/opea-project/GenAIExamples/commit/35a4fef) - - [VideoQnA] Update VideoQnA README.md for workflow [1929dfd](https://github.com/opea-project/GenAIExamples/commit/1929dfd) + - Update README.md for add K8S cluster link for Gaudi. ([91ff520b](https://github.com/opea-project/GenAIExamples/commit/91ff520b)) + - Update README.md for support matrix. ([41374d86](https://github.com/opea-project/GenAIExamples/commit/41374d86)) + - Update README.md for quick start guide. ([00241d01](https://github.com/opea-project/GenAIExamples/commit/00241d01)) + - Add example for AudioQnA deploy in AMD ROCm. ([006c61bc](https://github.com/opea-project/GenAIExamples/commit/006c61bc)) - CI/CD/UT - - Add nightly image build and publish action [78331ee](https://github.com/opea-project/GenAIExamples/commit/78331ee) - - optimize hardware list for test [3b1a9fe](https://github.com/opea-project/GenAIExamples/commit/3b1a9fe) - - open manifest test in CI when dockerfile changed [620ef76](https://github.com/opea-project/GenAIExamples/commit/620ef76) - - Optimize path and link validity check. [7dec001](https://github.com/opea-project/GenAIExamples/commit/7dec001) + - CI: Add check for conflict image build definition. ([8182a833](https://github.com/opea-project/GenAIExamples/commit/8182a833)) + - Check image and service names and Dockerfile in build.yaml. ([e8cffc61](https://github.com/opea-project/GenAIExamples/commit/e8cffc61)) + - Detect dangerous command. ([736155ca](https://github.com/opea-project/GenAIExamples/commit/736155ca)) +
GenAIComps +- Code Refactoring + - Core & Components + - GenAIComps microservices refactor. ([f57e30dd](https://github.com/opea-project/GenAIComps/commit/f57e30dd)) + - Remove examples gateway. ([f5efaf1f](https://github.com/opea-project/GenAIComps/commit/f5efaf1f)) + - Refactor llm predictionguard. ([4c21738a](https://github.com/opea-project/GenAIComps/commit/4c21738a)) + - Refactor llm Docsum. ([88f93733](https://github.com/opea-project/GenAIComps/commit/88f93733)) + - Refactor lvms. ([feef30b0](https://github.com/opea-project/GenAIComps/commit/feef30b0)) + - Refactor FaqGen. ([ea72c943](https://github.com/opea-project/GenAIComps/commit/ea72c943)) + - Refine embedding naming and move dependency to 3rd_party. ([b91911a5](https://github.com/opea-project/GenAIComps/commit/b91911a5)) + - Finetuning code refactor. ([efd95780](https://github.com/opea-project/GenAIComps/commit/efd95780)) + - Text2image code refactor. ([2587a297](https://github.com/opea-project/GenAIComps/commit/2587a297)) + - Refactor prompt registry microservice. ([179b5da0](https://github.com/opea-project/GenAIComps/commit/179b5da0)) + - Feedback management microservice refactor. ([ec66b91c](https://github.com/opea-project/GenAIComps/commit/ec66b91c)) + - Refactor web retriever. ([962e0978](https://github.com/opea-project/GenAIComps/commit/962e0978)) + - Refactor guardrails microservice. ([631b5704](https://github.com/opea-project/GenAIComps/commit/631b5704)) + - Refactor reranking. ([267cad1f](https://github.com/opea-project/GenAIComps/commit/267cad1f)) + - Refine Component Interface. ([bf097395](https://github.com/opea-project/GenAIComps/commit/bf097395)) + - Refine agent directories. ([cf90932f](https://github.com/opea-project/GenAIComps/commit/cf90932f)) + - Refactor text2sql based on ERAG. ([2cfd014b](https://github.com/opea-project/GenAIComps/commit/2cfd014b)) + - Image2video code refactor. ([90a86345](https://github.com/opea-project/GenAIComps/commit/90a86345)) + - Refactor asr/tts components. ([a19c2226](https://github.com/opea-project/GenAIComps/commit/a19c2226)) + - Refactor image2image. ([10408750](https://github.com/opea-project/GenAIComps/commit/10408750)) + - Refactor Animation based on ERAG. ([a7888ab2](https://github.com/opea-project/GenAIComps/commit/a7888ab2)) + - [Reorg] Remove redundant file in retrievers/redis. ([f3aaaebf](https://github.com/opea-project/GenAIComps/commit/f3aaaebf)) + + - Deployment + - Add kubernetes deployment for GenAIComps. ([1cc4d211](https://github.com/opea-project/GenAIComps/commit/1cc4d211)) + - Functionalities - New microservices: - - Add stable diffusion microservice [5d0c4367](https://github.com/opea-project/GenAIComps/commit/5d0c4367) - - Add image2video microservice (Stable Video Diffusion) [a03e7a55](https://github.com/opea-project/GenAIComps/commit/a03e7a55) - - Text to SQL microservice [827e3d40](https://github.com/opea-project/GenAIComps/commit/827e3d40) - - Add GPT-SoVITS microservice [6da7db9e](https://github.com/opea-project/GenAIComps/commit/6da7db9e) - - Add image2image microservice [52c1826f](https://github.com/opea-project/GenAIComps/commit/52c1826f) - - Initiate "animation" component [c26d37e7](https://github.com/opea-project/GenAIComps/commit/c26d37e7) - - GraphRAG with llama-index [19330ea2](https://github.com/opea-project/GenAIComps/commit/19330ea2) + - Add opensearch integration for OPEA. ([8d6b4b0a](https://github.com/opea-project/GenAIComps/commit/8d6b4b0a)) + - Feature/elasticsearch vector store integration - Infosys. ([5ed041bd](https://github.com/opea-project/GenAIComps/commit/5ed041bd)) + - Build guardrail "Hallucination Detection" microservice. ([4db13298](https://github.com/opea-project/GenAIComps/commit/4db13298)) + - Enhanced microservices: - - Add DPO support in finetuning microservice [37f35140](https://github.com/opea-project/GenAIComps/commit/37f35140) - - Support Chinese for Docsum [9a00a3ea](https://github.com/opea-project/GenAIComps/commit/9a00a3ea) - - Support file upload summary for DocSum microservice [fa2ea642](https://github.com/opea-project/GenAIComps/commit/fa2ea642) - - Add support for Audio and Video summarization to Docsum [baafa402](https://github.com/opea-project/GenAIComps/commit/baafa402) - - vLLM support for FAQGen [f5c60f10](https://github.com/opea-project/GenAIComps/commit/f5c60f10) - - vLLM support for DocSum [550325d8](https://github.com/opea-project/GenAIComps/commit/550325d8) - - vLLM support for Codegen [24b9f03f](https://github.com/opea-project/GenAIComps/commit/24b9f03f) - - Enable vllm for Agent [4638c1d4](https://github.com/opea-project/GenAIComps/commit/4638c1d4) - - Multiple models and remote service support for langchain vLLM text-generation [e3812a74](https://github.com/opea-project/GenAIComps/commit/e3812a74) - - Set a higher default value(1.2) about repetition_penalty for codegen example to reduce repetition [5ed428f4](https://github.com/opea-project/GenAIComps/commit/5ed428f4) - - MultimodalQnA Image and Audio Support Phase 1 [29ef6426](https://github.com/opea-project/GenAIComps/commit/29ef6426) - - refine codetrans prompt, support parameter input [0bb019f8](https://github.com/opea-project/GenAIComps/commit/0bb019f8) - - add dynamic batching embedding/reranking [518cdfb6](https://github.com/opea-project/GenAIComps/commit/518cdfb6) - - Embedding compatible with OpenAI API [7bf1953c](https://github.com/opea-project/GenAIComps/commit/7bf1953c) - - Update RAGAgentLlama and ReActLlama [c8e36390](https://github.com/opea-project/GenAIComps/commit/c8e36390) - - [Agent] support custom prompt [3473bfb3](https://github.com/opea-project/GenAIComps/commit/3473bfb3) - - agent short & long term memory with langgraph. [e39b08f3](https://github.com/opea-project/GenAIComps/commit/e39b08f3) - - support faqgen upload file in UI [453ff726](https://github.com/opea-project/GenAIComps/commit/453ff726) - - Add E2E Prometheus metrics to applications [a6998a1d](https://github.com/opea-project/GenAIComps/commit/a6998a1d) - - Multiple models support for LLM TGI [e879366c](https://github.com/opea-project/GenAIComps/commit/e879366c) - - Add RAG agent and ReAct agent implemention for llama3.1 served by TGI-gaudi [e7fdf537](https://github.com/opea-project/GenAIComps/commit/e7fdf537) - - Support Llama3.2 vision and vision guard model [534c227a](https://github.com/opea-project/GenAIComps/commit/534c227a) - - Add Intel/toxic-prompt-roberta to toxicity detection microservice [f6f620a2](https://github.com/opea-project/GenAIComps/commit/f6f620a2) - - Refactor milvus dataprep and retriever [84374a57](https://github.com/opea-project/GenAIComps/commit/84374a57) + - [agent] Add tool choices for agent. ([3a7ccb0a](https://github.com/opea-project/GenAIComps/commit/3a7ccb0a)) + - [agent] Add SQL agent strategy. ([717c3c10](https://github.com/opea-project/GenAIComps/commit/717c3c10)) + - [llm] Modify Params to Support Falcon3 Model. ([6acefae7](https://github.com/opea-project/GenAIComps/commit/6acefae7)) + - [llm/summarization] Add auto mode for long context. ([45d00020](https://github.com/opea-project/GenAIComps/commit/45d00020)) + - Removed microservices - - Remove vllm ray [617e119f](https://github.com/opea-project/GenAIComps/commit/617e119f) - - Async support for microservices - - Support async for embedding micorservice [28672956](https://github.com/opea-project/GenAIComps/commit/28672956) - - TEI rerank microservice async support [9df4b3c0](https://github.com/opea-project/GenAIComps/commit/9df4b3c0) - - Async support for some microservices [f3746dc8](https://github.com/opea-project/GenAIComps/commit/f3746dc8) + - Remove ragas. ([3e6c5cd8](https://github.com/opea-project/GenAIComps/commit/3e6c5cd8)) + - Remove dataprep/multimedia2text. ([f006a3ee](https://github.com/opea-project/GenAIComps/commit/f006a3ee)) + - Performance - - Fix vllm microservice performance issue. [2159f9ad](https://github.com/opea-project/GenAIComps/commit/2159f9ad) - - [Dataprep] Reduce Upload File Time Consumption [71348998](https://github.com/opea-project/GenAIComps/commit/71348998) -- New Hardware Support - - Add vLLM ARC support with OpenVINO backend [a2b9d95f](https://github.com/opea-project/GenAIComps/commit/a2b9d95f) -- Enhanced Security - - Prediction Guard Guardrails components [4bbc7a2f](https://github.com/opea-project/GenAIComps/commit/4bbc7a2f) - - Add WildGuard Guardrail Microservice [5bb4046b](https://github.com/opea-project/GenAIComps/commit/5bb4046b)F - - upgrade setuptools version to fix CVE-2024-6345 [6518c0f0](https://github.com/opea-project/GenAIComps/commit/6518c0f0) - - Remote TGI/TGI services with OAuth Client Credentials authentication [74df6bb7](https://github.com/opea-project/GenAIComps/commit/74df6bb7) -- Validation - - Combine CI/CD docker compose. [23c99c11](https://github.com/opea-project/GenAIComps/commit/23c99c11704cfcd0b13b3b463819f633ba02b42d) + - Remove enforce-eager to enable HPU graphs for better vLLM perf. ([ddd372d3](https://github.com/opea-project/GenAIComps/commit/ddd372d3)) + +- Behavior Changes + - Exclude yield/reply time from first token latency metric. ([5663e168](https://github.com/opea-project/GenAIComps/commit/5663e168)) + +- Dependency Versioning + - [animations] Remove version restrictions. ([3f23bf58](https://github.com/opea-project/GenAIComps/commit/3f23bf58)) + - [asr] Add the dependency to pydantic. ([145f3fb8](https://github.com/opea-project/GenAIComps/commit/145f3fb8)) + +- Bug Fixes + - Fix docker compose health check issue. ([fe24decd](https://github.com/opea-project/GenAIComps/commit/fe24decd)) + - Fix OpenAI API compatiable issue: embedding. ([c955e5e4](https://github.com/opea-project/GenAIComps/commit/c955e5e4)) + - Fix OpenAI API compatiable issue: vllm comps support openai API ChatCompletionRequest. ([48ed5898](https://github.com/opea-project/GenAIComps/commit/48ed5898)) + - Fix OpenAI API compatiable issue: ASR. ([c3948ad5](https://github.com/opea-project/GenAIComps/commit/c3948ad5)) + +- CI/CD/UT + - Add dangerous cmd check. ([766c757f](https://github.com/opea-project/GenAIComps/commit/766c757f)) + - Enhance asr/tts tests. ([9a0d91a5](https://github.com/opea-project/GenAIComps/commit/9a0d91a5)) + - CI: Add check for conflict image build definition. ([0e94eecb](https://github.com/opea-project/GenAIComps/commit/0e94eecb))
-
GenAIEvals +
GenAIEval + +- Bug Fixes + - [FaqGen] Fix the metrics parse and statistics for benchmark. ([5d717e8](https://github.com/opea-project/GenAIEval/commit/5d717e8)) + - Update upload_file_no_rerank.txt. ([0155ec3](https://github.com/opea-project/GenAIEval/commit/0155ec3)) + - Update crag eval with benchmark results. ([6f7c3bc](https://github.com/opea-project/GenAIEval/commit/6f7c3bc)) + +- Changed Defaults + - Modify retrieval top_k parameter to 1 for benchmark. ([30e32ba](https://github.com/opea-project/GenAIEval/commit/30e32ba)) -- New Benchmark - - Support HELMET [4c8f048](https://github.com/opea-project/GenAIEval/commit/4c8f048) - - Support Longbench [021193f](https://github.com/opea-project/GenAIEval/commit/021193f) -- Performance - - Add new constant loader & Fix poisson loader issue [e11588c](https://github.com/opea-project/GenAIEval/commit/e11588c) - - Support Poisson distributed requests for benchmark [7305ea3](https://github.com/opea-project/GenAIEval/commit/7305ea3) - - Support customized prompts and max new tokens in chatqna e2e test [79a4ad3](https://github.com/opea-project/GenAIEval/commit/79a4ad3) - - Add namespace support for k8s performance test [70697d1](https://github.com/opea-project/GenAIEval/commit/70697d1) - - Support sharegpt dataset in chatqna e2e test [028bf63](https://github.com/opea-project/GenAIEval/commit/028bf63) - - [Benchmark] Get benchmark reports. [946c439](https://github.com/opea-project/GenAIEval/commit/946c439) -- Accuracy - - Control the concurrent number of requests in codegen acc test. [84e077e](https://github.com/opea-project/GenAIEval/commit/84e077e) - - integrate deepeval metric with remote endpoint, like tgi server. [ffa65dc](https://github.com/opea-project/GenAIEval/commit/ffa65dc) - - Ragaaf - adding new metric 'context recall' [cc7cebd](https://github.com/opea-project/GenAIEval/commit/cc7cebd) - - Ragaaf - adding new metric 'context relevance' [f995c9c](https://github.com/opea-project/GenAIEval/commit/f995c9c) - - Ragaaf (RAG assessment annotation free) [2413e70](https://github.com/opea-project/GenAIEval/commit/2413e70) - - Adding new metrics to ragas offering [d1c1337](https://github.com/opea-project/GenAIEval/commit/d1c1337) - - add crud ragas evaluation. [f2bff45](https://github.com/opea-project/GenAIEval/commit/f2bff45) - - Minimize requirements for user data for OPEA ragas [f1593ea](https://github.com/opea-project/GenAIEval/commit/f1593ea) -- Monitoring - - Add node metrics Grafana dashboard [a19f42e](https://github.com/opea-project/GenAIEval/commit/a19f42e) - - Add CPU Grafana dashboard [38e69eb](https://github.com/opea-project/GenAIEval/commit/38e69eb) - - add the grafana dashboard json file for Gaudi metrics [6c9ae91](https://github.com/opea-project/GenAIEval/commit/6c9ae91) - - Enhance the Grafana JSON file [8653efb](https://github.com/opea-project/GenAIEval/commit/8653efb) -- Fixed Issues - - [ChatQnA Benchmark] Fixed the output token in chatqnafixed.py [2c8ca26](https://github.com/opea-project/GenAIEval/commit/2c8ca26) - - Fix test duration time inaccurate issue [9d76832](https://github.com/opea-project/GenAIEval/commit/9d76832) - - Fix llm output token length issue [99ef325](https://github.com/opea-project/GenAIEval/commit/99ef325) - - Fix llm serving benchmark issue [d6bafbd](https://github.com/opea-project/GenAIEval/commit/d6bafbd) - - Fix input token size(1024) [30adcbe](https://github.com/opea-project/GenAIEval/commit/30adcbe) - - Ragas fix for use of metrics argument [0cf3631](https://github.com/opea-project/GenAIEval/commit/0cf3631) - - fixed the number of ouput token & fixed the top_k=1 [4af0a62](https://github.com/opea-project/GenAIEval/commit/4af0a62) - - Fix JSON Return Format in getReqData Function [a4be366](https://github.com/opea-project/GenAIEval/commit/a4be366) -- Documentation - - Add setup guide of gaudi prometheus exporter [e9b8637](https://github.com/opea-project/GenAIEval/commit/e9b8637) - - Add README for running OPEA ragas using HF endpoint on Gaudi [0dff0d3](https://github.com/opea-project/GenAIEval/commit/0dff0d3)
GenAIInfra -- GMC - - Add manifests for new components [e51fd62](https://github.com/opea-project/GenAIInfra/commit/e51fd62) - - HelmChart - - [AgentQnA] Helm Chart for AgentQnA [66de41c](https://github.com/opea-project/GenAIInfra/commit/66de41c) - - [AudioQnA] helm: Add audioQnA e2e helm chart [9efacee](https://github.com/opea-project/GenAIInfra/commit/9efacee) - - [AudioQnA] helm-charts: Add gpt-sovits support [1f55e1a](https://github.com/opea-project/GenAIInfra/commit/1f55e1a) - - [ChatQnA] Implement the nowrapper version chatqna [71c81d0](https://github.com/opea-project/GenAIInfra/commit/71c81d0) - - [FaqGen] Add FaqGen helm chart [f847e05](https://github.com/opea-project/GenAIInfra/commit/f847e05) - - [FaqGen] helm: Add llm-faqgen-tgi support [325126e](https://github.com/opea-project/GenAIInfra/commit/325126e) - - [HPA] helm/manifest: Sync HPA related k8s probe settings [c399578](https://github.com/opea-project/GenAIInfra/commit/c399578) - - [VisualQnA] Add helm chart for VisualQnA example [b077d44](https://github.com/opea-project/GenAIInfra/commit/b077d44) - - [UI] support variants for multiple examples [96af2ad](https://github.com/opea-project/GenAIInfra/commit/96af2ad) - - [Nginx] helm-chart: Make nginx service type configurable [a5c96ab](https://github.com/opea-project/GenAIInfra/commit/a5c96ab) - - [Milvus] Add milvus support for data-prep and retriever-usvc [d289b4e](https://github.com/opea-project/GenAIInfra/commit/d289b4e) - - Add helm chart for 3 components [881e2b5](https://github.com/opea-project/GenAIInfra/commit/881e2b5) - - accelerate also teirerank with Gaudi [620963f](https://github.com/opea-project/GenAIInfra/commit/620963f) + - helm chart: Add service account support. ([9bb7c3a](https://github.com/opea-project/GenAIInfra/commit/9bb7c3a)) + - Add vLLM support for DocSum. ([0943764](https://github.com/opea-project/GenAIInfra/commit/0943764)) + - Modify embedding-usvc to support multimodal embedding. ([ecb4866](https://github.com/opea-project/GenAIInfra/commit/ecb4866)) + - Add minimal resource requests for tgi. ([3b7f28b](https://github.com/opea-project/GenAIInfra/commit/3b7f28b)) + - Add text2image microservice support. ([7b35326](https://github.com/opea-project/GenAIInfra/commit/7b35326)) + - Adapt latest changes in asr/tts related components. ([9f9b1d5](https://github.com/opea-project/GenAIInfra/commit/9f9b1d5)) + - Add lvm releated microservices. ([b0c760f](https://github.com/opea-project/GenAIInfra/commit/b0c760f)) + - Adapt rerank/web-retriever to latest changes. ([386d6d6](https://github.com/opea-project/GenAIInfra/commit/386d6d6)) + - Adapt to latest changes in llm microservice famliy. ([70ad650](https://github.com/opea-project/GenAIInfra/commit/70ad650)) + - docsum: reduce microservices in docsum. ([68e7d06](https://github.com/opea-project/GenAIInfra/commit/68e7d06)) + - audioqna: reduce microservice numbers. ([07c163b](https://github.com/opea-project/GenAIInfra/commit/07c163b)) + - Add vLLM+HPA support to ChatQnA Helm chart. ([baed0b5](https://github.com/opea-project/GenAIInfra/commit/baed0b5)) + - Helm: Add audioqna UI support. ([7a26d06](https://github.com/opea-project/GenAIInfra/commit/7a26d06)) - CSP - - terraform: add AWS/EKS deployment for ChatQnA [bdb9af9](https://github.com/opea-project/GenAIInfra/commit/bdb9af9) + - Azure automated deployment for OPEA applications - Infosys. ([e9dc58a](https://github.com/opea-project/GenAIInfra/commit/e9dc58a)) - Monitoring - - Add Grafana dashboard for monitoring OPEA application scaling in k8s [691bbc5](https://github.com/opea-project/GenAIInfra/commit/691bbc5) - - Add ServiceMonitors for rest of OPEA applications [fc6235a](https://github.com/opea-project/GenAIInfra/commit/fc6235a) - - Add monitoring option to (ChatQnA) Helm charts [dbd607e](https://github.com/opea-project/GenAIInfra/commit/dbd607e) - - Support alternative metrics on accelerated TGI / TEI instances [cdd3585](https://github.com/opea-project/GenAIInfra/commit/cdd3585) - - Expose options such as collector.interval of memory bandwidth exporter in k8s manifests and docker for user configuration. [2517e79](https://github.com/opea-project/GenAIInfra/commit/2517e79) - -- Dependency Versioning - - [TEI-Gaudi] Upgrade tei-gaudi version to 1.5.0 [c6a9c90](https://github.com/opea-project/GenAIInfra/commit/c6a9c90) - - [TGI-CPU] Update tgi cpu image version to 2.4.0-intel-cpu [f6c180e](https://github.com/opea-project/GenAIInfra/commit/f6c180e) - - [TGI-Gaudi] Upgrade tgi-gaudi to version 2.0.6 [915baa0](https://github.com/opea-project/GenAIInfra/commit/915baa0) - - Update the image version for ChatQnA examples [593458c](https://github.com/opea-project/GenAIInfra/commit/593458c) + - Add monitoring for rest of ChatQnA + DocSum components. ([590991b](https://github.com/opea-project/GenAIInfra/commit/590991b)) - Changed Defaults - - Change default model of codegen and codetrans [74476b7](https://github.com/opea-project/GenAIInfra/commit/74476b7) + - docsum: Use docsum-gradio-ui by default. ([95d6398](https://github.com/opea-project/GenAIInfra/commit/95d6398)) + - Use default kubernetes imagePullPolicy. ([0f21681](https://github.com/opea-project/GenAIInfra/commit/0f21681)) - Documentation - - Update observability README + fix typos [1d77b81](https://github.com/opea-project/GenAIInfra/commit/1d77b81) - - Monitoring, Observability and HPA doc improvements [14198fe](https://github.com/opea-project/GenAIInfra/commit/14198fe) - - Update GMC manifest changes and misc fixes [87dc673](https://github.com/opea-project/GenAIInfra/commit/87dc673) - - Improve Helm charts README [7b8c510](https://github.com/opea-project/GenAIInfra/commit/7b8c510) - - Create troubleshooting.md [d55ded4](https://github.com/opea-project/GenAIInfra/commit/d55ded4) - - Enhance helm chart repo usage in README [0de5535](https://github.com/opea-project/GenAIInfra/commit/0de5535) + +- Bug Fixes + - [AgentQnA] Fix OpenAI compatiable issue: streaming -> stream. ([88a7b52](https://github.com/opea-project/GenAIInfra/commit/88a7b52)) + - Fix model-downloader and tgi in multi shard case. ([a4a96ab](https://github.com/opea-project/GenAIInfra/commit/a4a96ab)) - CI/CD/UT - - Refactor CI scripts to support more components [e09270a](https://github.com/opea-project/GenAIInfra/commit/e09270a) - - Add github workflows to release helm chart [3910e3b](https://github.com/opea-project/GenAIInfra/commit/3910e3b) - - Fix link check failure (#481) (5 weeks ago) [fc87ef3](https://github.com/opea-project/GenAIInfra/commit/fc87ef3) - - Fix CI failures (#477) (5 weeks ago) [7e7b8ab](https://github.com/opea-project/GenAIInfra/commit/7e7b8ab) - - Optimize path and link validity check. [91bd163](https://github.com/opea-project/GenAIInfra/commit/91bd163) - -Enable image build process for memory-bandwidth-exporter [ddeac46](https://github.com/opea-project/GenAIInfra/commit/ddeac46) - - Add hyperlinks and paths validation. [d8cd3a1](https://github.com/opea-project/GenAIInfra/commit/d8cd3a1) + +
+ +
GenAIStudio +
### Full Changelogs @@ -307,14 +240,15 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - GenAIInfra: [v1.1...v1.2](https://github.com/opea-project/GenAIInfra/compare/v1.1...v1.2) - GenAIEval: [v1.1...v1.2](https://github.com/opea-project/GenAIEval/compare/v1.1...v1.2) - GenAIStudio: [v1.1...v1.2](https://github.com/opea-project/GenAIStudio/compare/v1.1...v1.2) +- docs: [v1.1...v1.2](https://github.com/opea-project/docs/compare/v1.1...v1.2) ## Removals, Merges, Deprecations and Behavior Changes ### GenAIComps #### Removals -- Remove embedding microservices: `MOSEC` ([#1072](https://github.com/opea-project/GenAIComps/pull/1072)). -- Remove reranking microservices: `fastRAG`, `MOSEC` ([#1003](https://github.com/opea-project/GenAIComps/pull/1003)). +- Remove embedding microservices: `MOSEC`. +- Remove reranking microservices: `fastRAG`, `MOSEC`. - Remove vector store microservices: `LanceDB`, `Chroma`. - Remove `intent_detection` microservice. - Remove module `cores/mega/gateway`. @@ -325,6 +259,34 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem #### Renamings - Rename `comps/reranks` to `comps/rerankings`. +#### Versioning +- [animations] Remove fixed version constraints from all dependencies, and use the latest versions instead. +- Upgrade `HabanaAI/vllm-fork` to the latest. + +#### Behavior Changes +- [llm] Exclude yield/reply time from first token latency metric. + + +### GenAIExamples + +#### Behavior Changes +- ChatQnA: Use vLLM as the default serving framework on Gaudi to leverage its optimized performance characteristics, thereby improving throughput and reducing latency in inference tasks. +- DocSum: Change the default UI to a Gradio-based UI. + + +### GenAIEval + +#### Behavior Changes +- Retriever benchmark: Change the default top k from 4 to 1. + + +### GenAIInfra + +#### Behavior Changes +- Change the `imagePullPolicy` from `IfNotPresent` to `default`. + + + ### Docker Images #### Deprecations @@ -333,9 +295,6 @@ The following Docker images are deprecated, and will be removed since OPEA v1.3: - [opea/embedding-langchain-mosec-endpoint](https://hub.docker.com/r/opea/embedding-langchain-mosec-endpoint) - [opea/llm-vllm-hpu](https://hub.docker.com/r/opea/llm-vllm-hpu) - [opea/llm-ray](https://hub.docker.com/r/opea/llm-ray) - - [opea/reranking-langchain-mosec](https://hub.docker.com/r/opea/reranking-langchain-mosec) - [opea/reranking-langchain-mosec-endpoint](https://hub.docker.com/r/opea/reranking-langchain-mosec-endpoint) - [opea/reranking-fastrag](https://hub.docker.com/r/opea/reranking-fastrag) @@ -350,10 +309,6 @@ The merged Docker images are deprecated, and will be removed since OPEA v1.3. Pl The Docker images marked for renaming are now deprecated and will be removed since OPEA v1.3. Please use the renamed images instead. - Rename [opea/embedding-multimodal-bridgetower-gaudi](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-gaudi) to [opea/embedding-multimodal-bridgetower-hpu](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-hpu) -### GenAIExamples - -#### Behavior Changes -- ChatQnA: Use vLLM as the default serving framework on Gaudi to leverage its optimized performance characteristics, thereby improving throughput and reducing latency in inference tasks. ## Contributors This release would not have been possible without the contributions of the following organizations and individuals. @@ -365,4 +320,4 @@ This release would not have been possible without the contributions of the follo - `Intel`: Development and improvements to GenAI examples, components, infrastructure, and evaluation. ### Individual Contributors -For a comprehensive list of individual contributors, please refer to the "Full Changelogs" section. \ No newline at end of file +For a comprehensive list of individual contributors, please refer to the [Full Changelogs](#full-changelogs) section. \ No newline at end of file From 161d164fa4a85fdeb15b881ab60a13b26452143b Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 14:21:01 +0800 Subject: [PATCH 03/14] Update v1.2 release notes according to the status at 11/16. --- release_notes/v1.2.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 13f08f94..fec9b2a3 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -8,9 +8,9 @@ This release focuses on code refactoring for [GenAIComps](https://github.com/ope OPEA v1.2 also introduces more scenarios with general availability, including: - [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. -- Model Context Protocol(MCP) Support: Experimental support for MCP at Retriever. +- Model Context Protocol(MCP) Support: Experimental support for [MCP at Retriever](https://github.com/opea-project/GenAIComps/pull/1155). - Enhanced Security: Istio Mutual TLS (mTLS) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). -- Enhancements for China Ecosystem: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). +- Enhancements for GenAI Evaluation: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). - Helm Charts Deployment: Add supports for the examples Text2Image, SearchQnA, EdgeCraftRag and their microservices. ### Highlights @@ -58,7 +58,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - Falcon2-40B/11B - Falcon3 -#### Newly Supported Hardware (TODO: update after code freeze) +#### Newly Supported Hardware (TODO: finalize the list when code freeze) - AMD® GPU: Enabled GenAI examples on AMD® GPUs using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA?](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). ### Notable Changes @@ -299,11 +299,11 @@ The following Docker images are deprecated, and will be removed since OPEA v1.3: - [opea/reranking-langchain-mosec-endpoint](https://hub.docker.com/r/opea/reranking-langchain-mosec-endpoint) - [opea/reranking-fastrag](https://hub.docker.com/r/opea/reranking-fastrag) - [opea/web-retriever-chroma](https://hub.docker.com/r/opea/web-retriever-chroma) +- [opea/llm-ollama](https://hub.docker.com/r/opea/llm-ollama) #### Merges The merged Docker images are deprecated, and will be removed since OPEA v1.3. Please use the new images instead. - Merge [opea/embedding-tei](https://hub.docker.com/r/opea/embedding-tei) to [opea/embedding](https://hub.docker.com/r/opea/embedding) -- Merge [opea/llm-ollama](https://hub.docker.com/r/opea/llm-ollama) to []() (TODO: update if finalized) #### Renamings The Docker images marked for renaming are now deprecated and will be removed since OPEA v1.3. Please use the renamed images instead. From b7e62761d45862d94a76297ab026010415a030c4 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 14:35:00 +0800 Subject: [PATCH 04/14] Add PR link for Istio mTLS in v1.2 release notes. --- release_notes/v1.2.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index fec9b2a3..6b375000 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -9,7 +9,7 @@ This release focuses on code refactoring for [GenAIComps](https://github.com/ope OPEA v1.2 also introduces more scenarios with general availability, including: - [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. - Model Context Protocol(MCP) Support: Experimental support for [MCP at Retriever](https://github.com/opea-project/GenAIComps/pull/1155). -- Enhanced Security: Istio Mutual TLS (mTLS) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). +- Enhanced Security: [Istio Mutual TLS (mTLS)](https://github.com/opea-project/GenAIInfra/pull/702) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). - Enhancements for GenAI Evaluation: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). - Helm Charts Deployment: Add supports for the examples Text2Image, SearchQnA, EdgeCraftRag and their microservices. @@ -36,7 +36,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem #### Enhanced GenAI Components -- vLLM support for embeddings and rerankings](https://github.com/opea-project/GenAIComps/issues/956)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. +- [vLLM support for embeddings and rerankings](https://github.com/opea-project/GenAIComps/issues/956)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. - Agent Microservice: - SQL agent strategy: Take user question, hints (optional) and history (when available), and thinks step by step to solve the problem by interacting with a SQL database. OPEA currently has two types of [SQL agents](https://github.com/opea-project/GenAIComps/pull/1039): `sql_agent_llama` for using with open-source LLMs and `sql_agent`: for using with OpenAI models. - Enabled user-customized tool subsets: Added support for [user-defined subsets of tools](https://github.com/opea-project/GenAIComps/issues/978) for the ChatCompletion API and Assistant APIs. From 3aaabdca2971ccaa9a368d768fa226b00a730302 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 14:50:10 +0800 Subject: [PATCH 05/14] Add notable changes of GenAIStudio in v1.2 notes. --- release_notes/v1.2.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 6b375000..1204e9b5 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -32,7 +32,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - [Bedrock](https://github.com/opea-project/GenAIComps/pull/1031): OPEA LLM now supports [Amazon Bedrock](https://aws.amazon.com/bedrock/) as the backend of the text generation microservice. Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. - [OpenSearch Vector Database](https://github.com/opea-project/GenAIComps/pull/1024): OPEA vectorstores now supports [AWS OpenSearch](https://aws.amazon.com/opensearch-service/). OpenSearch is an open-source, enterprise-grade search and observability suite that brings order to unstructured data at scale. - [Elasticsearch Vector Database](https://github.com/opea-project/GenAIComps/pull/972): OPEA vectorestores now supports [Elasticsearch vector database](https://www.elastic.co/elasticsearch/vector-database), Elasticsearch's open source vector database offering an efficient way to create, store, and search vector embeddings. -- [Guardrail Hallucination Detection](https://github.com/opea-project/GenAIComps/pull/1108): The capability of detecting `Hallucination` which spans a wide range of issues that can impact reliability, trustworthiness, and utility of AI-generated content. +- [Guardrail Hallucination Detection](https://github.com/opea-project/GenAIComps/pull/1108): Added the capability of detecting `Hallucination` which spans a wide range of issues that can impact reliability, trustworthiness, and utility of AI-generated content. #### Enhanced GenAI Components @@ -46,7 +46,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - [PDF Inputs Support](https://github.com/opea-project/GenAIComps/pull/1134): Support PDF inputs for dataprep, embeddings, lvms, and retrievers. #### Enhanced GenAI Examples -- ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203), and Enable [Jaeger UI and OpenTelemetry tracing](https://github.com/opea-project/GenAIExamples/pull/1316) for TGI serving on HPU. +- ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203), and [Jaeger UI and OpenTelemetry tracing](https://github.com/opea-project/GenAIExamples/pull/1316) for TGI serving on HPU. - AgentQnA: Added [SQL worker agent](https://github.com/opea-project/GenAIExamples/pull/1370) and introduced a [Svelte-based GUI](https://github.com/opea-project/GenAIExamples/pull/1389) for ChatCompletion API for non-streaming interactions. - MultimodalQnA: Supported [PDF](https://github.com/opea-project/GenAIExamples/pull/1381) and [audio](https://github.com/opea-project/GenAIExamples/pull/1225) inputs. - EdgeCraftRAG: Supported image/url data retrieval and display, display of LLM-used context sources in UI, pipeline remove operation in RESTful API and UI, RAG pipeline performance benchmark and display in UI. ([#GenAIExamples/1324](https://github.com/opea-project/GenAIExamples/pull/1324)) @@ -232,6 +232,13 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem
GenAIStudio +- Add keycloak theme under assets. ([00da22d](https://github.com/opea-project/GenAIStudio/commit/00da22d)) +- Add new basic workflow after solving the bug. ([96f6590](https://github.com/opea-project/GenAIStudio/commit/96f6590)) +- Let initial inputs at least match one key for prompt. ([e6c4229](https://github.com/opea-project/GenAIStudio/commit/e6c4229)) +- Add more keywords, and retry another question. ([c2a6e70](https://github.com/opea-project/GenAIStudio/commit/c2a6e70)) +- Update openai version in studio-frontend. ([11ac0ba](https://github.com/opea-project/GenAIStudio/commit/11ac0ba)) +- Update readme and removed depreciated chromium version. ([62a35ea](https://github.com/opea-project/GenAIStudio/commit/62a35ea)) +
### Full Changelogs From 05f70e7a1cc0b21ad31e0c5f175ecae5529dc7af Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 14:55:01 +0800 Subject: [PATCH 06/14] Update v1.2 release notes according to Ke's comments. --- release_notes/v1.2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 1204e9b5..da46dbec 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -4,7 +4,7 @@ We are excited to announce the release of OPEA version 1.2, which includes signi More information about how to get started with OPEA v1.2 can be found at [Getting Started](https://opea-project.github.io/latest/index.html) page. All project source code is maintained in the [repository](https://github.com/opea-project). To pull Docker images, please access the [Docker Hub](https://hub.docker.com/u/opea). For instructions on deploying Helm Charts, please refer to the [guide](https://github.com/opea-project/GenAIInfra/tree/v1.2/helm-charts#readme). ## What's New in OPEA v1.2 -This release focuses on code refactoring for [GenAIComps](https://github.com/opea-project/GenAIComps), the epic efforts aimed at reduceing redundancy, addressing technical debt, and enhancing overall maintainability and code quality. As a result, OPEA users can expect a more robust and reliable OPEA with clearer guidance and improved documentation. +This release focuses on code refactoring for [GenAIComps](https://github.com/opea-project/GenAIComps), the epic efforts aimed at reducing redundancy, addressing technical debt, and enhancing overall maintainability and code quality. As a result, OPEA users can expect a more robust and reliable OPEA with clearer guidance and improved documentation. OPEA v1.2 also introduces more scenarios with general availability, including: - [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. @@ -321,8 +321,8 @@ The Docker images marked for renaming are now deprecated and will be removed sin This release would not have been possible without the contributions of the following organizations and individuals. ### Contributing Organizations -- `Amazon`: AMD CPU/GPU support for GenAIExamples. -- `AMD`: Bedrock and OpenSearch vector database integration. +- `Amazon`: Bedrock and OpenSearch vector database integration. +- `AMD`: AMD CPU/GPU support for GenAIExamples. - `Infosys`: Elasticsearch vector database integration. - `Intel`: Development and improvements to GenAI examples, components, infrastructure, and evaluation. From 51b8a3d6bc32f50e0805b48d491ea1b649be5e79 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 15:04:44 +0800 Subject: [PATCH 07/14] Fix typos in v1.2 release notes --- release_notes/v1.2.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index da46dbec..02c2c9ce 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -38,7 +38,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem #### Enhanced GenAI Components - [vLLM support for embeddings and rerankings](https://github.com/opea-project/GenAIComps/issues/956)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. - Agent Microservice: - - SQL agent strategy: Take user question, hints (optional) and history (when available), and thinks step by step to solve the problem by interacting with a SQL database. OPEA currently has two types of [SQL agents](https://github.com/opea-project/GenAIComps/pull/1039): `sql_agent_llama` for using with open-source LLMs and `sql_agent`: for using with OpenAI models. + - SQL agent strategy: Take user question, hints (optional) and history (when available), and think step by step to solve the problem by interacting with a SQL database. OPEA currently has two types of [SQL agents](https://github.com/opea-project/GenAIComps/pull/1039): `sql_agent_llama` for using with open-source LLMs and `sql_agent`: for using with OpenAI models. - Enabled user-customized tool subsets: Added support for [user-defined subsets of tools](https://github.com/opea-project/GenAIComps/issues/978) for the ChatCompletion API and Assistant APIs. - Enabled persistence: Introduced Redis to persist Agent configurations and historical messages for Agent recovery and multi-turn conversations. - Long-context Summarization: Supported multiple modes: [`auto`](https://github.com/opea-project/GenAIComps/commit/5aba3b25), [`stuff`, `truncate`, `map_reduce`, and `refine`](https://github.com/opea-project/GenAIComps/commit/45d00020). @@ -171,9 +171,9 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - Bug Fixes - Fix docker compose health check issue. ([fe24decd](https://github.com/opea-project/GenAIComps/commit/fe24decd)) - - Fix OpenAI API compatiable issue: embedding. ([c955e5e4](https://github.com/opea-project/GenAIComps/commit/c955e5e4)) - - Fix OpenAI API compatiable issue: vllm comps support openai API ChatCompletionRequest. ([48ed5898](https://github.com/opea-project/GenAIComps/commit/48ed5898)) - - Fix OpenAI API compatiable issue: ASR. ([c3948ad5](https://github.com/opea-project/GenAIComps/commit/c3948ad5)) + - Fix OpenAI API compatible issue: embedding. ([c955e5e4](https://github.com/opea-project/GenAIComps/commit/c955e5e4)) + - Fix OpenAI API compatible issue: vllm comps support openai API ChatCompletionRequest. ([48ed5898](https://github.com/opea-project/GenAIComps/commit/48ed5898)) + - Fix OpenAI API compatible issue: ASR. ([c3948ad5](https://github.com/opea-project/GenAIComps/commit/c3948ad5)) - CI/CD/UT - Add dangerous cmd check. ([766c757f](https://github.com/opea-project/GenAIComps/commit/766c757f)) @@ -202,9 +202,9 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - Add minimal resource requests for tgi. ([3b7f28b](https://github.com/opea-project/GenAIInfra/commit/3b7f28b)) - Add text2image microservice support. ([7b35326](https://github.com/opea-project/GenAIInfra/commit/7b35326)) - Adapt latest changes in asr/tts related components. ([9f9b1d5](https://github.com/opea-project/GenAIInfra/commit/9f9b1d5)) - - Add lvm releated microservices. ([b0c760f](https://github.com/opea-project/GenAIInfra/commit/b0c760f)) + - Add lvm related microservices. ([b0c760f](https://github.com/opea-project/GenAIInfra/commit/b0c760f)) - Adapt rerank/web-retriever to latest changes. ([386d6d6](https://github.com/opea-project/GenAIInfra/commit/386d6d6)) - - Adapt to latest changes in llm microservice famliy. ([70ad650](https://github.com/opea-project/GenAIInfra/commit/70ad650)) + - Adapt to latest changes in llm microservice family. ([70ad650](https://github.com/opea-project/GenAIInfra/commit/70ad650)) - docsum: reduce microservices in docsum. ([68e7d06](https://github.com/opea-project/GenAIInfra/commit/68e7d06)) - audioqna: reduce microservice numbers. ([07c163b](https://github.com/opea-project/GenAIInfra/commit/07c163b)) - Add vLLM+HPA support to ChatQnA Helm chart. ([baed0b5](https://github.com/opea-project/GenAIInfra/commit/baed0b5)) @@ -223,7 +223,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - Documentation - Bug Fixes - - [AgentQnA] Fix OpenAI compatiable issue: streaming -> stream. ([88a7b52](https://github.com/opea-project/GenAIInfra/commit/88a7b52)) + - [AgentQnA] Fix OpenAI compatible issue: streaming -> stream. ([88a7b52](https://github.com/opea-project/GenAIInfra/commit/88a7b52)) - Fix model-downloader and tgi in multi shard case. ([a4a96ab](https://github.com/opea-project/GenAIInfra/commit/a4a96ab)) - CI/CD/UT @@ -237,7 +237,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - Let initial inputs at least match one key for prompt. ([e6c4229](https://github.com/opea-project/GenAIStudio/commit/e6c4229)) - Add more keywords, and retry another question. ([c2a6e70](https://github.com/opea-project/GenAIStudio/commit/c2a6e70)) - Update openai version in studio-frontend. ([11ac0ba](https://github.com/opea-project/GenAIStudio/commit/11ac0ba)) -- Update readme and removed depreciated chromium version. ([62a35ea](https://github.com/opea-project/GenAIStudio/commit/62a35ea)) +- Update readme and removed deprecated chromium version. ([62a35ea](https://github.com/opea-project/GenAIStudio/commit/62a35ea))
From f6d2a4a2fded097266776e98c1e6facfff756b1c Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 16:42:01 +0800 Subject: [PATCH 08/14] Add CSP support in v1.2 release notes. --- release_notes/v1.2.md | 159 ++++++++++++++++++++++-------------------- 1 file changed, 83 insertions(+), 76 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 02c2c9ce..5115fb82 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -27,13 +27,10 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem > [!NOTE] > We suggest users and contributors to review the [documentation](#removals-merges-deprecations-and-behavior-changes) to understand the impacts of the code refactoring. +#### Supporting Cloud Service Providers +OPEA offers automated Terraform deployment using Intel® Optimized Cloud Modules for Terraform, available for major cloud platforms, including `Amazon Web Services (AWS)`, `Google Cloud Platform (GCP)`, and `Microsoft Azure`. To explore this option, check out the [Terraform deployment guide](https://opea-project.github.io/latest/GenAIExamples/ChatQnA/README.html#automated-terraform-deployment-using-intel-optimized-cloud-modules-for-terraform). -#### New GenAI Components -- [Bedrock](https://github.com/opea-project/GenAIComps/pull/1031): OPEA LLM now supports [Amazon Bedrock](https://aws.amazon.com/bedrock/) as the backend of the text generation microservice. Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. -- [OpenSearch Vector Database](https://github.com/opea-project/GenAIComps/pull/1024): OPEA vectorstores now supports [AWS OpenSearch](https://aws.amazon.com/opensearch-service/). OpenSearch is an open-source, enterprise-grade search and observability suite that brings order to unstructured data at scale. -- [Elasticsearch Vector Database](https://github.com/opea-project/GenAIComps/pull/972): OPEA vectorestores now supports [Elasticsearch vector database](https://www.elastic.co/elasticsearch/vector-database), Elasticsearch's open source vector database offering an efficient way to create, store, and search vector embeddings. -- [Guardrail Hallucination Detection](https://github.com/opea-project/GenAIComps/pull/1108): Added the capability of detecting `Hallucination` which spans a wide range of issues that can impact reliability, trustworthiness, and utility of AI-generated content. - +Additionally, OPEA supports manual deployment on virtual servers across `AWS`, `GCP`, `IBM Cloud`, `Azure`, and `Oracle Cloud Infrastructure (OCI)`. For detailed instructions, refer to the [manual deployment guide](https://opea-project.github.io/latest/getting-started/README.html#create-and-configure-a-virtual-server). #### Enhanced GenAI Components - [vLLM support for embeddings and rerankings](https://github.com/opea-project/GenAIComps/issues/956)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. @@ -45,6 +42,12 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - Standalone Microservice Deployment: Enabled the deployment of OPEA components as independent services, allowing for greater flexibility, scalability, and modularity in various application scenarios. - [PDF Inputs Support](https://github.com/opea-project/GenAIComps/pull/1134): Support PDF inputs for dataprep, embeddings, lvms, and retrievers. +#### New GenAI Components +- [Bedrock](https://github.com/opea-project/GenAIComps/pull/1031): OPEA LLM now supports [Amazon Bedrock](https://aws.amazon.com/bedrock/) as the backend of the text generation microservice. Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. +- [OpenSearch Vector Database](https://github.com/opea-project/GenAIComps/pull/1024): OPEA vectorstores now supports [AWS OpenSearch](https://aws.amazon.com/opensearch-service/). OpenSearch is an open-source, enterprise-grade search and observability suite that brings order to unstructured data at scale. +- [Elasticsearch Vector Database](https://github.com/opea-project/GenAIComps/pull/972): OPEA vectorestores now supports [Elasticsearch vector database](https://www.elastic.co/elasticsearch/vector-database), Elasticsearch's open source vector database offering an efficient way to create, store, and search vector embeddings. +- [Guardrail Hallucination Detection](https://github.com/opea-project/GenAIComps/pull/1108): Added the capability of detecting `Hallucination` which spans a wide range of issues that can impact reliability, trustworthiness, and utility of AI-generated content. + #### Enhanced GenAI Examples - ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203), and [Jaeger UI and OpenTelemetry tracing](https://github.com/opea-project/GenAIExamples/pull/1316) for TGI serving on HPU. - AgentQnA: Added [SQL worker agent](https://github.com/opea-project/GenAIExamples/pull/1370) and introduced a [Svelte-based GUI](https://github.com/opea-project/GenAIExamples/pull/1389) for ChatCompletion API for non-streaming interactions. @@ -59,9 +62,79 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - Falcon3 #### Newly Supported Hardware (TODO: finalize the list when code freeze) -- AMD® GPU: Enabled GenAI examples on AMD® GPUs using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA?](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). +- Intel® Gaudi® 3 AI Accelerator +- AMD® GPU using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA?](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). + + +## Deprecations and Behavior Changes + +### GenAIComps + +#### Removals +- Remove embedding microservices: `MOSEC`. +- Remove reranking microservices: `fastRAG`, `MOSEC`. +- Remove vector store microservices: `LanceDB`, `Chroma`. +- Remove `intent_detection` microservice. +- Remove module `cores/mega/gateway`. + +#### Merges +- `TGI`, `vLLM`, and `Ollama` integrations in LLM/text-generation have been merged into `LLM/text-generation/native`. + +#### Renamings +- Rename `comps/reranks` to `comps/rerankings`. + +#### Versioning +- [animations] Remove fixed version constraints from all dependencies, and use the latest versions instead. +- Upgrade `HabanaAI/vllm-fork` to the latest. + +#### Behavior Changes +- [llm] Exclude yield/reply time from first token latency metric. + -### Notable Changes +### GenAIExamples + +#### Behavior Changes +- ChatQnA: Use vLLM as the default serving framework on Gaudi to leverage its optimized performance characteristics, thereby improving throughput and reducing latency in inference tasks. +- DocSum: Change the default UI to a Gradio-based UI. + + +### GenAIEval + +#### Behavior Changes +- Retriever benchmark: Change the default top k from 4 to 1. + + +### GenAIInfra + +#### Behavior Changes +- Change the `imagePullPolicy` from `IfNotPresent` to `default`. + + + +### Docker Images + +#### Deprecations +The following Docker images are deprecated, and will be removed since OPEA v1.3: +- [opea/embedding-langchain-mosec](https://hub.docker.com/r/opea/embedding-langchain-mosec) +- [opea/embedding-langchain-mosec-endpoint](https://hub.docker.com/r/opea/embedding-langchain-mosec-endpoint) +- [opea/llm-vllm-hpu](https://hub.docker.com/r/opea/llm-vllm-hpu) +- [opea/llm-ray](https://hub.docker.com/r/opea/llm-ray) +- [opea/reranking-langchain-mosec](https://hub.docker.com/r/opea/reranking-langchain-mosec) +- [opea/reranking-langchain-mosec-endpoint](https://hub.docker.com/r/opea/reranking-langchain-mosec-endpoint) +- [opea/reranking-fastrag](https://hub.docker.com/r/opea/reranking-fastrag) +- [opea/web-retriever-chroma](https://hub.docker.com/r/opea/web-retriever-chroma) +- [opea/llm-ollama](https://hub.docker.com/r/opea/llm-ollama) + +#### Merges +The merged Docker images are deprecated, and will be removed since OPEA v1.3. Please use the new images instead. +- Merge [opea/embedding-tei](https://hub.docker.com/r/opea/embedding-tei) to [opea/embedding](https://hub.docker.com/r/opea/embedding) + +#### Renamings +The Docker images marked for renaming are now deprecated and will be removed since OPEA v1.3. Please use the renamed images instead. +- Rename [opea/embedding-multimodal-bridgetower-gaudi](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-gaudi) to [opea/embedding-multimodal-bridgetower-hpu](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-hpu) + + +## Notable Changes
GenAIExamples @@ -241,7 +314,8 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem
-### Full Changelogs + +## Full Changelogs - GenAIExamples: [v1.1...v1.2](https://github.com/opea-project/GenAIExamples/compare/v1.1...v1.2) - GenAIComps: [v1.1...v1.2](https://github.com/opea-project/GenAIComps/compare/v1.1...v1.2) - GenAIInfra: [v1.1...v1.2](https://github.com/opea-project/GenAIInfra/compare/v1.1...v1.2) @@ -249,73 +323,6 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem - GenAIStudio: [v1.1...v1.2](https://github.com/opea-project/GenAIStudio/compare/v1.1...v1.2) - docs: [v1.1...v1.2](https://github.com/opea-project/docs/compare/v1.1...v1.2) -## Removals, Merges, Deprecations and Behavior Changes - -### GenAIComps - -#### Removals -- Remove embedding microservices: `MOSEC`. -- Remove reranking microservices: `fastRAG`, `MOSEC`. -- Remove vector store microservices: `LanceDB`, `Chroma`. -- Remove `intent_detection` microservice. -- Remove module `cores/mega/gateway`. - -#### Merges -- `TGI`, `vLLM`, and `Ollama` integrations in LLM/text-generation have been merged into `LLM/text-generation/native`. - -#### Renamings -- Rename `comps/reranks` to `comps/rerankings`. - -#### Versioning -- [animations] Remove fixed version constraints from all dependencies, and use the latest versions instead. -- Upgrade `HabanaAI/vllm-fork` to the latest. - -#### Behavior Changes -- [llm] Exclude yield/reply time from first token latency metric. - - -### GenAIExamples - -#### Behavior Changes -- ChatQnA: Use vLLM as the default serving framework on Gaudi to leverage its optimized performance characteristics, thereby improving throughput and reducing latency in inference tasks. -- DocSum: Change the default UI to a Gradio-based UI. - - -### GenAIEval - -#### Behavior Changes -- Retriever benchmark: Change the default top k from 4 to 1. - - -### GenAIInfra - -#### Behavior Changes -- Change the `imagePullPolicy` from `IfNotPresent` to `default`. - - - -### Docker Images - -#### Deprecations -The following Docker images are deprecated, and will be removed since OPEA v1.3: -- [opea/embedding-langchain-mosec](https://hub.docker.com/r/opea/embedding-langchain-mosec) -- [opea/embedding-langchain-mosec-endpoint](https://hub.docker.com/r/opea/embedding-langchain-mosec-endpoint) -- [opea/llm-vllm-hpu](https://hub.docker.com/r/opea/llm-vllm-hpu) -- [opea/llm-ray](https://hub.docker.com/r/opea/llm-ray) -- [opea/reranking-langchain-mosec](https://hub.docker.com/r/opea/reranking-langchain-mosec) -- [opea/reranking-langchain-mosec-endpoint](https://hub.docker.com/r/opea/reranking-langchain-mosec-endpoint) -- [opea/reranking-fastrag](https://hub.docker.com/r/opea/reranking-fastrag) -- [opea/web-retriever-chroma](https://hub.docker.com/r/opea/web-retriever-chroma) -- [opea/llm-ollama](https://hub.docker.com/r/opea/llm-ollama) - -#### Merges -The merged Docker images are deprecated, and will be removed since OPEA v1.3. Please use the new images instead. -- Merge [opea/embedding-tei](https://hub.docker.com/r/opea/embedding-tei) to [opea/embedding](https://hub.docker.com/r/opea/embedding) - -#### Renamings -The Docker images marked for renaming are now deprecated and will be removed since OPEA v1.3. Please use the renamed images instead. -- Rename [opea/embedding-multimodal-bridgetower-gaudi](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-gaudi) to [opea/embedding-multimodal-bridgetower-hpu](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-hpu) - ## Contributors This release would not have been possible without the contributions of the following organizations and individuals. From 5b8e06cbc35351b123acf997de46118c290a8557 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 16 Jan 2025 16:51:55 +0800 Subject: [PATCH 09/14] Add CSP support in What's new of v1.2 release notes. --- release_notes/v1.2.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 5115fb82..acaca49c 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -9,6 +9,7 @@ This release focuses on code refactoring for [GenAIComps](https://github.com/ope OPEA v1.2 also introduces more scenarios with general availability, including: - [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. - Model Context Protocol(MCP) Support: Experimental support for [MCP at Retriever](https://github.com/opea-project/GenAIComps/pull/1155). +- Cloud Service Providers(CSP) Support: Supported automated Terraform deployment using Intel® Optimized Cloud Modules for Terraform, available for major cloud platforms, including `Amazon Web Services (AWS)`, `Google Cloud Platform (GCP)`, and `Microsoft Azure`. - Enhanced Security: [Istio Mutual TLS (mTLS)](https://github.com/opea-project/GenAIInfra/pull/702) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). - Enhancements for GenAI Evaluation: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). - Helm Charts Deployment: Add supports for the examples Text2Image, SearchQnA, EdgeCraftRag and their microservices. @@ -28,7 +29,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem > We suggest users and contributors to review the [documentation](#removals-merges-deprecations-and-behavior-changes) to understand the impacts of the code refactoring. #### Supporting Cloud Service Providers -OPEA offers automated Terraform deployment using Intel® Optimized Cloud Modules for Terraform, available for major cloud platforms, including `Amazon Web Services (AWS)`, `Google Cloud Platform (GCP)`, and `Microsoft Azure`. To explore this option, check out the [Terraform deployment guide](https://opea-project.github.io/latest/GenAIExamples/ChatQnA/README.html#automated-terraform-deployment-using-intel-optimized-cloud-modules-for-terraform). +OPEA offers automated Terraform deployment using Intel® Optimized Cloud Modules for Terraform, available for major cloud platforms, including `AWS`, `GCP`, and `Azure`. To explore this option, check out the [Terraform deployment guide](https://opea-project.github.io/latest/GenAIExamples/ChatQnA/README.html#automated-terraform-deployment-using-intel-optimized-cloud-modules-for-terraform). Additionally, OPEA supports manual deployment on virtual servers across `AWS`, `GCP`, `IBM Cloud`, `Azure`, and `Oracle Cloud Infrastructure (OCI)`. For detailed instructions, refer to the [manual deployment guide](https://opea-project.github.io/latest/getting-started/README.html#create-and-configure-a-virtual-server). From 8743db19ae6464ff172a4a439e363e2215ff9b04 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Fri, 17 Jan 2025 20:14:56 +0800 Subject: [PATCH 10/14] Remove a hyperlink of docker image to make check pass. --- release_notes/v1.2.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index acaca49c..fb93cbb0 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -1,5 +1,5 @@ # OPEA Release Notes v1.2 -We are excited to announce the release of OPEA version 1.2, which includes significant contributions from the open-source community. This release addresses over 263(TODO: update the latest) pull requests. +We are excited to announce the release of OPEA version 1.2, which includes significant contributions from the open-source community. This release addresses over 290 pull requests. More information about how to get started with OPEA v1.2 can be found at [Getting Started](https://opea-project.github.io/latest/index.html) page. All project source code is maintained in the [repository](https://github.com/opea-project). To pull Docker images, please access the [Docker Hub](https://hub.docker.com/u/opea). For instructions on deploying Helm Charts, please refer to the [guide](https://github.com/opea-project/GenAIInfra/tree/v1.2/helm-charts#readme). @@ -28,6 +28,7 @@ Additionally, we reduced code redundancy, merged overlapping modules, and implem > [!NOTE] > We suggest users and contributors to review the [documentation](#removals-merges-deprecations-and-behavior-changes) to understand the impacts of the code refactoring. + #### Supporting Cloud Service Providers OPEA offers automated Terraform deployment using Intel® Optimized Cloud Modules for Terraform, available for major cloud platforms, including `AWS`, `GCP`, and `Azure`. To explore this option, check out the [Terraform deployment guide](https://opea-project.github.io/latest/GenAIExamples/ChatQnA/README.html#automated-terraform-deployment-using-intel-optimized-cloud-modules-for-terraform). @@ -57,12 +58,15 @@ Additionally, OPEA supports manual deployment on virtual servers across `AWS`, ` - DocSum: Adding [URL summary option](https://github.com/opea-project/GenAIExamples/pull/1248) to Gradio-based UI. - DocIndexRetriever: Add the pipeline without Reranking. +#### Enhanced GenAIStudio +In this release, GenAI Studio enables keycloak for multi user management, supporting sandbox environment for multi workflow execution and enables Grafana based visualization dashboards with build-in performance metric on Prometheus for model evaluation and functional nodes performance. + #### Newly Supported Models - bge-base-zh-v1.5 - Falcon2-40B/11B - Falcon3 -#### Newly Supported Hardware (TODO: finalize the list when code freeze) +#### Newly Supported Hardware (TODO: finalize the list after code freeze) - Intel® Gaudi® 3 AI Accelerator - AMD® GPU using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA?](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). @@ -81,6 +85,9 @@ Additionally, OPEA supports manual deployment on virtual servers across `AWS`, ` #### Merges - `TGI`, `vLLM`, and `Ollama` integrations in LLM/text-generation have been merged into `LLM/text-generation/native`. +#### Movements +- Move `redis`, `milvus`, `elasticsearch`, `opensearch`, `` + #### Renamings - Rename `comps/reranks` to `comps/rerankings`. @@ -132,7 +139,7 @@ The merged Docker images are deprecated, and will be removed since OPEA v1.3. Pl #### Renamings The Docker images marked for renaming are now deprecated and will be removed since OPEA v1.3. Please use the renamed images instead. -- Rename [opea/embedding-multimodal-bridgetower-gaudi](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-gaudi) to [opea/embedding-multimodal-bridgetower-hpu](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-hpu) +- Rename [opea/embedding-multimodal-bridgetower-gaudi](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-gaudi) to `opea/embedding-multimodal-bridgetower-hpu`. ## Notable Changes From 04c25aef4fd6c474c9fbba260391db5420d5710b Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Mon, 20 Jan 2025 17:42:06 +0800 Subject: [PATCH 11/14] update v1.2 release notes to fit the status of ww4.1. --- release_notes/v1.2.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index fb93cbb0..f7fd7312 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -1,5 +1,5 @@ # OPEA Release Notes v1.2 -We are excited to announce the release of OPEA version 1.2, which includes significant contributions from the open-source community. This release addresses over 290 pull requests. +We are excited to announce the release of OPEA version 1.2, which includes significant contributions from the open-source community. This release addresses over 320 pull requests. More information about how to get started with OPEA v1.2 can be found at [Getting Started](https://opea-project.github.io/latest/index.html) page. All project source code is maintained in the [repository](https://github.com/opea-project). To pull Docker images, please access the [Docker Hub](https://hub.docker.com/u/opea). For instructions on deploying Helm Charts, please refer to the [guide](https://github.com/opea-project/GenAIInfra/tree/v1.2/helm-charts#readme). @@ -8,7 +8,7 @@ This release focuses on code refactoring for [GenAIComps](https://github.com/ope OPEA v1.2 also introduces more scenarios with general availability, including: - [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. -- Model Context Protocol(MCP) Support: Experimental support for [MCP at Retriever](https://github.com/opea-project/GenAIComps/pull/1155). +- Model Context Protocol(MCP) Support: Experimental support for MCP at Retriever. - Cloud Service Providers(CSP) Support: Supported automated Terraform deployment using Intel® Optimized Cloud Modules for Terraform, available for major cloud platforms, including `Amazon Web Services (AWS)`, `Google Cloud Platform (GCP)`, and `Microsoft Azure`. - Enhanced Security: [Istio Mutual TLS (mTLS)](https://github.com/opea-project/GenAIInfra/pull/702) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). - Enhancements for GenAI Evaluation: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). @@ -66,9 +66,9 @@ In this release, GenAI Studio enables keycloak for multi user management, suppor - Falcon2-40B/11B - Falcon3 -#### Newly Supported Hardware (TODO: finalize the list after code freeze) +#### Newly Supported Hardware - Intel® Gaudi® 3 AI Accelerator -- AMD® GPU using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [SearchQnA?](https://github.com/opea-project/GenAIExamples/pull/1193), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). +- AMD® GPU using AMD® ROCm™ for [AgentQnA](https://github.com/opea-project/GenAIExamples/pull/1166), [AudioQnA](https://github.com/opea-project/GenAIExamples/pull/1147), [MultimodalQnA](https://github.com/opea-project/GenAIExamples/pull/1233), [VisualQnA](https://github.com/opea-project/GenAIExamples/pull/1201), [Translation](https://github.com/opea-project/GenAIExamples/pull/1191). ## Deprecations and Behavior Changes @@ -83,10 +83,10 @@ In this release, GenAI Studio enables keycloak for multi user management, suppor - Remove module `cores/mega/gateway`. #### Merges -- `TGI`, `vLLM`, and `Ollama` integrations in LLM/text-generation have been merged into `LLM/text-generation/native`. +- `TGI`, `vLLM`, and `Ollama` integrations in LLM/text-generation have been merged into `LLM/text-generation/integrations/native`. #### Movements -- Move `redis`, `milvus`, `elasticsearch`, `opensearch`, `` +- [vectore stores] Move `redis`, `milvus`, `elasticsearch`, `opensearch`, `pathway`, `pgvector` to `comps/third_parties`. #### Renamings - Rename `comps/reranks` to `comps/rerankings`. @@ -98,9 +98,18 @@ In this release, GenAI Studio enables keycloak for multi user management, suppor #### Behavior Changes - [llm] Exclude yield/reply time from first token latency metric. +#### API Changes +- Dataprep + - Ingest a file: change from `v1/dataprep` to `v1/dataprep/ingest`. + - Get a file: change from `v1/dataprep/get_file` to `v1/dataprep/ingest/get`. + - Delete a file: change from `v1/dataprep/delete_file` to `v1/dataprep/delete` + ### GenAIExamples +#### Versioning +- Upgrade `tgi-gaudi` to 2.3.1. + #### Behavior Changes - ChatQnA: Use vLLM as the default serving framework on Gaudi to leverage its optimized performance characteristics, thereby improving throughput and reducing latency in inference tasks. - DocSum: Change the default UI to a Gradio-based UI. From 52b424cb49e2d56faed17c26d6cc4c516358180a Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Thu, 23 Jan 2025 08:38:09 +0800 Subject: [PATCH 12/14] Update v1.2 release notes for the ww4.3 updates --- release_notes/v1.2.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index f7fd7312..15bad56c 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -7,12 +7,12 @@ More information about how to get started with OPEA v1.2 can be found at [Gettin This release focuses on code refactoring for [GenAIComps](https://github.com/opea-project/GenAIComps), the epic efforts aimed at reducing redundancy, addressing technical debt, and enhancing overall maintainability and code quality. As a result, OPEA users can expect a more robust and reliable OPEA with clearer guidance and improved documentation. OPEA v1.2 also introduces more scenarios with general availability, including: -- [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and [LangChain](https://github.com/opea-project/GenAIComps/pull/1128) integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. +- [LlamaIndex](https://github.com/run-llama/llama_index/pull/16666) and LangChain Integration: Enabling OPEA as a backend. LlamaIndex integration currently supports [ChatQnA](https://github.com/opea-project/GenAIExamples/tree/v1.2/ChatQnA) only. - Model Context Protocol(MCP) Support: Experimental support for MCP at Retriever. - Cloud Service Providers(CSP) Support: Supported automated Terraform deployment using Intel® Optimized Cloud Modules for Terraform, available for major cloud platforms, including `Amazon Web Services (AWS)`, `Google Cloud Platform (GCP)`, and `Microsoft Azure`. - Enhanced Security: [Istio Mutual TLS (mTLS)](https://github.com/opea-project/GenAIInfra/pull/702) and [OIDC (Open ID Connect) based Authentication with APISIX](https://github.com/opea-project/GenAIInfra/pull/312). - Enhancements for GenAI Evaluation: [Specialized evaluation benchmarks](https://github.com/opea-project/GenAIEval/pull/141) tailored for Chinese language models, focusing on their performance and accuracy within [Chinese dataset](https://github.com/opea-project/GenAIEval/pull/48). -- Helm Charts Deployment: Add supports for the examples Text2Image, SearchQnA, EdgeCraftRag and their microservices. +- Helm Charts Deployment: Add supports for the examples Text2Image, SearchQnA and their microservices. ### Highlights @@ -35,31 +35,31 @@ OPEA offers automated Terraform deployment using Intel® Optimized Cloud Modules Additionally, OPEA supports manual deployment on virtual servers across `AWS`, `GCP`, `IBM Cloud`, `Azure`, and `Oracle Cloud Infrastructure (OCI)`. For detailed instructions, refer to the [manual deployment guide](https://opea-project.github.io/latest/getting-started/README.html#create-and-configure-a-virtual-server). #### Enhanced GenAI Components -- [vLLM support for embeddings and rerankings](https://github.com/opea-project/GenAIComps/issues/956)：Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. +- [vLLM support for embeddings and rerankings](https://github.com/opea-project/GenAIComps/issues/956): Integrate vLLM as a serving framework to enhance the performance and scalability of embedding and reranking models. - Agent Microservice: - SQL agent strategy: Take user question, hints (optional) and history (when available), and think step by step to solve the problem by interacting with a SQL database. OPEA currently has two types of [SQL agents](https://github.com/opea-project/GenAIComps/pull/1039): `sql_agent_llama` for using with open-source LLMs and `sql_agent`: for using with OpenAI models. - Enabled user-customized tool subsets: Added support for [user-defined subsets of tools](https://github.com/opea-project/GenAIComps/issues/978) for the ChatCompletion API and Assistant APIs. - Enabled persistence: Introduced Redis to persist Agent configurations and historical messages for Agent recovery and multi-turn conversations. - Long-context Summarization: Supported multiple modes: [`auto`](https://github.com/opea-project/GenAIComps/commit/5aba3b25), [`stuff`, `truncate`, `map_reduce`, and `refine`](https://github.com/opea-project/GenAIComps/commit/45d00020). - Standalone Microservice Deployment: Enabled the deployment of OPEA components as independent services, allowing for greater flexibility, scalability, and modularity in various application scenarios. -- [PDF Inputs Support](https://github.com/opea-project/GenAIComps/pull/1134): Support PDF inputs for dataprep, embeddings, lvms, and retrievers. +- [PDF Inputs Support](https://github.com/opea-project/GenAIComps/pull/1134): Support PDF inputs for dataprep, embeddings, LVMs, and retrievers. #### New GenAI Components - [Bedrock](https://github.com/opea-project/GenAIComps/pull/1031): OPEA LLM now supports [Amazon Bedrock](https://aws.amazon.com/bedrock/) as the backend of the text generation microservice. Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. - [OpenSearch Vector Database](https://github.com/opea-project/GenAIComps/pull/1024): OPEA vectorstores now supports [AWS OpenSearch](https://aws.amazon.com/opensearch-service/). OpenSearch is an open-source, enterprise-grade search and observability suite that brings order to unstructured data at scale. - [Elasticsearch Vector Database](https://github.com/opea-project/GenAIComps/pull/972): OPEA vectorestores now supports [Elasticsearch vector database](https://www.elastic.co/elasticsearch/vector-database), Elasticsearch's open source vector database offering an efficient way to create, store, and search vector embeddings. -- [Guardrail Hallucination Detection](https://github.com/opea-project/GenAIComps/pull/1108): Added the capability of detecting `Hallucination` which spans a wide range of issues that can impact reliability, trustworthiness, and utility of AI-generated content. +- [Guardrail Hallucination Detection](https://github.com/opea-project/GenAIComps/pull/1108): Added the capability of detecting Hallucination which spans a wide range of issues that can impact reliability, trustworthiness, and utility of AI-generated content. #### Enhanced GenAI Examples - ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203), and [Jaeger UI and OpenTelemetry tracing](https://github.com/opea-project/GenAIExamples/pull/1316) for TGI serving on HPU. - AgentQnA: Added [SQL worker agent](https://github.com/opea-project/GenAIExamples/pull/1370) and introduced a [Svelte-based GUI](https://github.com/opea-project/GenAIExamples/pull/1389) for ChatCompletion API for non-streaming interactions. - MultimodalQnA: Supported [PDF](https://github.com/opea-project/GenAIExamples/pull/1381) and [audio](https://github.com/opea-project/GenAIExamples/pull/1225) inputs. - EdgeCraftRAG: Supported image/url data retrieval and display, display of LLM-used context sources in UI, pipeline remove operation in RESTful API and UI, RAG pipeline performance benchmark and display in UI. ([#GenAIExamples/1324](https://github.com/opea-project/GenAIExamples/pull/1324)) -- DocSum: Adding [URL summary option](https://github.com/opea-project/GenAIExamples/pull/1248) to Gradio-based UI. +- DocSum: Added [URL summary option](https://github.com/opea-project/GenAIExamples/pull/1248) to Gradio-based UI. - DocIndexRetriever: Add the pipeline without Reranking. #### Enhanced GenAIStudio -In this release, GenAI Studio enables keycloak for multi user management, supporting sandbox environment for multi workflow execution and enables Grafana based visualization dashboards with build-in performance metric on Prometheus for model evaluation and functional nodes performance. +In this release, GenAI Studio enables Keycloak for multi-user management, supporting sandbox environment for multi-workflow execution and enables Grafana based visualization dashboards with built-in performance metric on Prometheus for model evaluation and functional nodes performance. #### Newly Supported Models - bge-base-zh-v1.5 @@ -163,9 +163,9 @@ The Docker images marked for renaming are now deprecated and will be removed sin - [ChatQnA] Enable OpenTelemetry Tracing for ChatQnA TGI serving on Gaudi. ([81022355](https://github.com/opea-project/GenAIExamples/commit/81022355)) - [DocIndexRetriever] Enable the without-rerank flavor. ([a50e4e6f](https://github.com/opea-project/GenAIExamples/commit/a50e4e6f)) - [DocSum] Support Long context for DocSum. ([50dd959d](https://github.com/opea-project/GenAIExamples/commit/50dd959d)) - - [DocSum] Adding URL summary option to DocSum Gradio-UI. ([84a6a6e9](https://github.com/opea-project/GenAIExamples/commit/84a6a6e9)) + - [DocSum] Added URL summary option to DocSum Gradio-UI. ([84a6a6e9](https://github.com/opea-project/GenAIExamples/commit/84a6a6e9)) - [EdgeCraftRAG] Add new feature and bug fix for EC-RAG. ([6169ea49](https://github.com/opea-project/GenAIExamples/commit/6169ea49)) - - [MultimodalQnA] Adds audio querying to MultimodalQ&A Example. ([c760cac2](https://github.com/opea-project/GenAIExamples/commit/c760cac2)) + - [MultimodalQnA] Add audio querying to MultimodalQ&A Example. ([c760cac2](https://github.com/opea-project/GenAIExamples/commit/c760cac2)) - Changed Defaults - [DocSum] Changed Default UI to Gradio. ([00b526c8](https://github.com/opea-project/GenAIExamples/commit/00b526c8)) @@ -191,8 +191,8 @@ The Docker images marked for renaming are now deprecated and will be removed sin - FaqGen param fix. ([05365b61](https://github.com/opea-project/GenAIExamples/commit/05365b61)) - Documentation - - Update README.md for add K8S cluster link for Gaudi. ([91ff520b](https://github.com/opea-project/GenAIExamples/commit/91ff520b)) - - Update README.md for support matrix. ([41374d86](https://github.com/opea-project/GenAIExamples/commit/41374d86)) + - Update README.md for adding K8S cluster link for Gaudi. ([91ff520b](https://github.com/opea-project/GenAIExamples/commit/91ff520b)) + - Update README.md for supporting matrix. ([41374d86](https://github.com/opea-project/GenAIExamples/commit/41374d86)) - Update README.md for quick start guide. ([00241d01](https://github.com/opea-project/GenAIExamples/commit/00241d01)) - Add example for AudioQnA deploy in AMD ROCm. ([006c61bc](https://github.com/opea-project/GenAIExamples/commit/006c61bc)) @@ -322,7 +322,7 @@ The Docker images marked for renaming are now deprecated and will be removed sin
GenAIStudio -- Add keycloak theme under assets. ([00da22d](https://github.com/opea-project/GenAIStudio/commit/00da22d)) +- Add Keycloak theme under assets. ([00da22d](https://github.com/opea-project/GenAIStudio/commit/00da22d)) - Add new basic workflow after solving the bug. ([96f6590](https://github.com/opea-project/GenAIStudio/commit/96f6590)) - Let initial inputs at least match one key for prompt. ([e6c4229](https://github.com/opea-project/GenAIStudio/commit/e6c4229)) - Add more keywords, and retry another question. ([c2a6e70](https://github.com/opea-project/GenAIStudio/commit/c2a6e70)) From c145d47b2c8944c51fc3be29486b079a1e23bb06 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Fri, 24 Jan 2025 13:36:19 +0800 Subject: [PATCH 13/14] Update release notes/v1.2 based on dmsuehir's comments Co-authored-by: Dina Suehiro Jones --- release_notes/v1.2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 15bad56c..a28b2118 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -53,7 +53,7 @@ Additionally, OPEA supports manual deployment on virtual servers across `AWS`, ` #### Enhanced GenAI Examples - ChatQnA: Enabled [embedding and reranking on vLLM](https://github.com/opea-project/GenAIExamples/issues/1203), and [Jaeger UI and OpenTelemetry tracing](https://github.com/opea-project/GenAIExamples/pull/1316) for TGI serving on HPU. - AgentQnA: Added [SQL worker agent](https://github.com/opea-project/GenAIExamples/pull/1370) and introduced a [Svelte-based GUI](https://github.com/opea-project/GenAIExamples/pull/1389) for ChatCompletion API for non-streaming interactions. -- MultimodalQnA: Supported [PDF](https://github.com/opea-project/GenAIExamples/pull/1381) and [audio](https://github.com/opea-project/GenAIExamples/pull/1225) inputs. +- MultimodalQnA: Added support for [PDF](https://github.com/opea-project/GenAIExamples/pull/1381) ingestion, and [image](https://github.com/opea-project/GenAIExamples/pull/1381)/[audio](https://github.com/opea-project/GenAIExamples/pull/1225) queries. - EdgeCraftRAG: Supported image/url data retrieval and display, display of LLM-used context sources in UI, pipeline remove operation in RESTful API and UI, RAG pipeline performance benchmark and display in UI. ([#GenAIExamples/1324](https://github.com/opea-project/GenAIExamples/pull/1324)) - DocSum: Added [URL summary option](https://github.com/opea-project/GenAIExamples/pull/1248) to Gradio-based UI. - DocIndexRetriever: Add the pipeline without Reranking. From 286a0e146837490cdbfcd91ea39b72ae6c0e38e7 Mon Sep 17 00:00:00 2001 From: Yi Yao Date: Sat, 25 Jan 2025 22:15:00 +0800 Subject: [PATCH 14/14] Update v1.2 notes to remove part of renamings. --- release_notes/v1.2.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/release_notes/v1.2.md b/release_notes/v1.2.md index 15bad56c..632ca659 100644 --- a/release_notes/v1.2.md +++ b/release_notes/v1.2.md @@ -146,11 +146,6 @@ The following Docker images are deprecated, and will be removed since OPEA v1.3: The merged Docker images are deprecated, and will be removed since OPEA v1.3. Please use the new images instead. - Merge [opea/embedding-tei](https://hub.docker.com/r/opea/embedding-tei) to [opea/embedding](https://hub.docker.com/r/opea/embedding) -#### Renamings -The Docker images marked for renaming are now deprecated and will be removed since OPEA v1.3. Please use the renamed images instead. -- Rename [opea/embedding-multimodal-bridgetower-gaudi](https://hub.docker.com/r/opea/embedding-multimodal-bridgetower-gaudi) to `opea/embedding-multimodal-bridgetower-hpu`. - - ## Notable Changes
GenAIExamples