LLM on OpenShift

In this repo you will find resources, demos, recipes,... to work with LLMs on OpenShift with OpenShift Data Science or Open Data Hub.

Content

Inference Servers deployment

Two different Inference Servers deployment instructions are available:

• Caikit-TGIS-Serving (external): How to deploy the Caikit-TGIS-Serving stack, from OpenDataHub.
• Hugging Face Text Generation Inference: How to deploy the Text Generation Inference server from Hugging Face.

Vector Databases deployment

Deployments of different databases that can be used as a Vector Store are available:

• PostgreSQL+pgvector: Full recipe to create an instance of PostgreSQL with the pgvector extension, making it usable as a vector store.
• Redis: Full recipe to deploy Redis, create a Cluster and a suitable Database for a Vector Store.

Inference and application examples

• Caikit: Basic example demonstrating how to work with Caikit+TGIS for LLM serving.
• Langchain examples: Various notebooks demonstrating how to work with Langchain. Examples include both HFTGI and Caikit+TGIS Serving.
• Langflow examples: Various examples demonstrating how to work with Langflow.
• UI examples: Various examples on how to create and deploy a UI to interact with your LLM.