diff --git a/docs/README.md b/docs/README.md index 67d60a06f..2eaa5ef38 100644 --- a/docs/README.md +++ b/docs/README.md @@ -33,6 +33,8 @@ See the full documentation at [`../README.md`](../README.md) or browse sub-pages ## Overview +[Overview](https://lightspeed-core.github.io/lightspeed-stack/overview.html) + [Architecture](https://lightspeed-core.github.io/lightspeed-stack/ARCHITECTURE.html) [Architecture diagram](https://lightspeed-core.github.io/lightspeed-stack/architecture.svg) diff --git a/docs/index.md b/docs/index.md index f0e61edae..01bdec762 100644 --- a/docs/index.md +++ b/docs/index.md @@ -40,6 +40,8 @@ product questions using backend LLM services, agents, and RAG databases. ## Overview +[Overview](https://lightspeed-core.github.io/lightspeed-stack/overview.html) + [Architecture](https://lightspeed-core.github.io/lightspeed-stack/ARCHITECTURE.html) [Architecture diagram](https://lightspeed-core.github.io/lightspeed-stack/architecture.svg) diff --git a/docs/overview.md b/docs/overview.md new file mode 100644 index 000000000..981866f07 --- /dev/null +++ b/docs/overview.md @@ -0,0 +1,27 @@ +# Overview + +## Introduction + +### What is Lightspeed Core Stack? + +**Lightspeed Core Stack (LCore)** is an enterprise-grade middleware service that provides a robust layer between client applications and AI Large Language Model (LLM) backends. It adds essential enterprise features such as authentication, authorization, quota management, caching, and observability to LLM interactions. + +Current version of LCore is built on **Llama Stack** - open-source framework that provides standardized APIs for building LLM applications. Llama Stack offers a unified interface for models, RAG (vector stores), tools, and safety (shields) across different providers. LCore communicates with Llama Stack to orchestrate all LLM operations. + +To enhance LLM responses, LCore leverages **RAG (Retrieval-Augmented Generation)**, which retrieves relevant context from vector databases before generating answers. Llama Stack manages the vector stores, and LCore queries them to inject relevant documentation, knowledge bases, or previous conversations into the LLM prompt. + +### Key Features + +- **Multi-Provider Support**: Works with multiple LLM providers (Ollama, OpenAI, Watsonx, etc.) +- **Enterprise Security**: Authentication, authorization (RBAC), and secure credential management +- **Resource Management**: Token-based quota limits and usage tracking +- **Conversation Management**: Multi-turn conversations with history and caching +- **RAG Integration**: Retrieval-Augmented Generation for context-aware responses +- **Tool Orchestration**: Model Context Protocol (MCP) server integration +- **Observability**: Prometheus metrics, structured logging, and health checks +- **Agent-to-Agent**: A2A protocol support for multi-agent collaboration + +### Components Overview + +![Overview Stack as service](./overview.svg) + diff --git a/docs/overview.svg b/docs/overview.svg new file mode 100644 index 000000000..02dd918d4 --- /dev/null +++ b/docs/overview.svg @@ -0,0 +1,599 @@ + + + + + + + + + + + + + + + + + + +
+
+
Lightspeed to Dataverse exporter
+
+
+ + + + + + + + + + + + + + + +
+
+
Prometheus
+
+
+ + + + + + + + + + + + + + + + +
+
+
REST API
interface
+
+
+ + + + + + + + + + + + + + + + + +
+
+
Metrics
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
Lightspeed
Core Stack (LCS) service
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
Client
+
API
+
+
+
+ + + + + + + + + + + + + + +
+
+
+
Llama Stack
+
(library or
+
service mode)
+
+
+
+ + + + + + + + + + + + + + +
+
+
Agents
+
+
+ + + + + + + + + + + + + + +
+
+
+
RAG
+
+
+
+ + + + + + + + + + + + + + + + + + + + +
+
+
Configuration
+
+
+ + + + + + + + + + + + + + +
+
+
Conversation cache
+
+
+ + + + + + + + + + + + + + +
+
+
OKP
+
+
+ + + + + + + + + + + + + + +
+
+
Transcripts
+
+
+ + + + + + + + + + + + + + +
+
+
Feedback
+
+
+ + + + + + + + + + + + + + +
+
+
Human in loop controller
+
+
+ + + + + + + + + + + + + + +
+
+
Question validators
+
+
+ + + + + + + + + + + + + + +
+
+
Answer redactors
+
+
+ + + + + + + + + + + + + + +
+
+
Summarizers
+
+
+ + + + + + + + + + + + + + +
+
+
MCP servers
+
+
+ + + + + + + + + + + + + + + +
+
+
Authentication
(k8s, SSO...)
+
+
+ + + + + + + + + + + + + + + + + + + + + +
+
+
Llama Stack
Configuration
+
+
+ + + + + + + + + + + + + + + +
+
+
Authorization (RBAC)
+
+
+ + + + + + + + + + + + + + + + +
+
+
Sidecar
runners
+
+
+ + + + + + + + + + + + + + + + +
+
+
Conversation
history
+
+
+ + + + + + + + + + + + + + + + +
+
+
Quota management
+
+
+ + + + + + + + + + + + + + + + +
+
+
+
configuration
+
generator
+
+
+
+ + + + + + + + + + + + + + + +
+
+
+
Azure key
+
generator
+
on startup
+
+
+
+ + + + + + + + + +