Mx Cloud GenAI Refactoring #10307
Mx Cloud GenAI Refactoring #10307
Conversation
Added a 'How to get started' section
Added Mistral Connector to the list of available models and provided links for easier access to the Mendix Portal and documentation.
Corrected formatting and improved clarity in the instructions for using the Mendix Cloud GenAI Connector.
Reintroduced sections on Mendix Portal and Mendix Cloud GenAI Connector with updated formatting.
Updated the Mendix Cloud GenAI portal documentation to clarify resource pack access and added new sections for resource overview and metadata usage.
Reformatted metadata example for clarity and structure.
Correct grammatical errors and improve clarity in the introduction and metadata examples.
Corrected formatting and removed redundant information regarding Cross-Region Inference (CRI).
Removed typical use cases section detailing text generation, embeddings generation, and knowledge base functionalities from the Mendix Cloud GenAI Connector documentation.
Expanded the document to include typical use cases for large language models (LLMs), detailing applications in text generation, embeddings generation, and knowledge base utilization.
Added alert about embedding generation in the Mendix Cloud GenAI Connector documentation.
Refine examples of LLM use cases and clarify content creation.
Updated the Mistral Connector documentation to include a link for detailed use cases.
Updated the Mistral Connector documentation to clarify use cases and integrate links to relevant APIs.
Updated section headers and removed redundant features description.
Expanded the description of the OpenAI Connector to include support for knowledge bases from other providers like Mendix Cloud and added a reference to typical use cases.
Updated the Mendix Cloud GenAI portal documentation to include navigation details and resource overview. Added notes on new screenshots and clarified cross-region inference profile provisioning.
Updated the resource details section to clarify navigation and available features.
Updated section titles and improved clarity of token consumption monitoring description.
Added additional details for knowledge base resources and keys, clarifying the relationship between knowledge base and embeddings resource keys.
removed one 'the'
added mention of DeployedKnowledgeBase input param
- added mention of DeplyoedKnowledgeBase
- clarified use case - collection example
Clarified Metadata section
| The module enables tailoring generated responses to specific contexts by grounding them in data inside of a collection belonging to a Mendix Cloud GenAI knowledge base resource. This allows for the secure use of private company data or other non-public information when interacting with GenAI models within the Mendix app. It provides a low-code solution to store discrete data (commonly called chunks) in the knowledge base and retrieves relevant information for end-user actions or application processes. | ||
|
|
||
| Knowledge bases are often used for: | ||
| The Mendix Cloud GenAI Connector module generates embeddings internally when interacting with a knowledge base. This means that you do not need to create embedding keys yourself when interacting with a Mendix Cloud knowledge base. Pure embedding operations are only required if additional processes, such as using the generated vectors instead of text, are needed. For example, a similar search algorithm could use vector distances to calculate relatedness. |
There was a problem hiding this comment.
I would use "Direct embedding operations" over "Pure embedding operations"
| * Complete the runtime setup of Mendix Cloud GenAI configuration by navigating to the page mentioned above. Import a key generated in the [portal](https://genai.home.mendix.com) or provided to you and click **Test Key** to validate its functionality. Note that this key establishes a connection between the Mendix Cloud resources and your application. It contains all the information required to set up the connection. | ||
|
|
||
| {{% alert color="info" %}} | ||
| When using an Embeddings Mdel Resource in combination with a Knowledge Base Resource, there is no need to import both keys: the connection details for the embeddings generation model will be generated automatically on import of the Knowledge Base Resource key. |
There was a problem hiding this comment.
There is a typo here. "Mdel". I think it should be "Model"
|
|
||
| #### How to get the Deployed Model in scope | ||
|
|
||
| The DeployedModel will be created automatically when importing keys at runtime and needs to be retrieved from the database. |
There was a problem hiding this comment.
Is this a single word name "DeployedModel"?
There was a problem hiding this comment.
It's the entity's name ... I will write DeployedModel object to make it clearer
Reference to example for collections in metadata section
| 2. [Retrieval of knowledge (Nearest neighbor)](#knowledge-base-retrieval) | ||
|
|
||
| You do not need to manually add embeddings to a chunk, as the connector handles this internally. To see all existing knowledge bases for a configuration, go to the **Knowledge Base** tab on the [Mendix Cloud GenAI Configuration](#configuration) page and refresh the view on the right. Alternatively, use the `Get Collections` action to retrieve a synchronized list of collections inside of your knowledge base resource to include in your module. Lastly, you can delete a collection using the `Delete Collection` action. | ||
| You do not need to manually add embeddings to a chunk, as the connector handles this internally. To see all existing collections for a knowledge base configuration, go to the **Knowledge Base** tab on the [Mendix Cloud GenAI Configuration](#configuration) page and refresh the view on the right. Alternatively, use the `Get Collections` action to retrieve a synchronized list of collections inside of your knowledge base resource to include in your module. Lastly, you can delete a collection using the `Delete Collection` action. |
There was a problem hiding this comment.
I don't understand this statement. You do not need to manually add embeddings to a chunk . We never add embeddings to chunk, during the indexing phase we chunk the document - Split into smaller pieces (e.g., 512 tokens, with optional overlap) then generate embeddings - Calculate one embedding vector per chunk. Then during the retrieval phase, Embed the query - User question → embedding vector and then Similarity search - Find chunks with closest embeddings to query. what do you mean by adding embedding to chunk?
|
|
||
| The Mendix Portal allows easy access to manage the resources, through the GenAI Resources section [here](https://genai.home.mendix.com/). | ||
|
|
||
| * Get insight into the consumption of input/output tokens against entitlements for Models |
There was a problem hiding this comment.
This statement confuses me. We display input/output against to a TEXT or embedding resource not specifically against to a model. Then we display token consumption against to the bundle month, not specifically against to entitlements for models.
There was a problem hiding this comment.
I changed it to 'Get insight into the consumption of input/output tokens for Text and Embeddings Generation Resources' for now
| When you are looking at the knowledge base resource settings, you will also see details of the associated embeddings resource and vice versa. To learn more about embeddings, see the [Embedding vector](/appstore/modules/genai/rag/#embedding-vector) section of *RAG in a Mendix App*. | ||
|
|
||
| #### Cross-Region Inference (CRI) | ||
| \* Cross-region inference (CRI) allows a model to redirect requests to another region, helping to distribute the load across multiple regions. Connecting to a cross-region inference profile does not change how the request is sent; the redirection happens on the server side, determining the region to handle the request to get the fastest response, see also [Increase throughput with cross-Region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html). If applicable, CRI profiles are selected during provisioning of a model resource. |
There was a problem hiding this comment.
I think it is good to mention that the redirect happen only within Europe. No request go outside the EU.
| Metadata consists of key-value pairs and serves as additional information connected to the data, though it is not part of the vectorization itself. | ||
|
|
||
| For example, a GenAI Knowledge Base could be used to store customer support tickets. Each ticket may have associated metadata such as Ticket Type, Status, and Priority. Instead of relying solely on similarity-based searches of ticket descriptions, users can filter for specific tickets, such as 'bug' tickets with the status set to 'solved'. | ||
| For example, a GenAI Knowledge Base could be used to store customer support tickets. Each ticket may have associated metadata such as |
There was a problem hiding this comment.
CRITICAL: This isn't a good use case. Meta data must be static content. If you use Status/ Priority as meta data, the problem is status/ Priority of a ticket is more likely to change during the lifecycle of a ticket. Thus this KB data representation will be obsolete. To keep that in sync, u need to perform manual re-indexing, which is very expensive.
Good example can be a legal document processing agent backed by a knowledge base. where each legal doc may have following meta data tags
"document_type": "contract",
"contract_category": "vendor_agreement",
"signing_year": "2023",
"jurisdiction": "EU",
"practice_area": "commercial",
"confidentiality_level": "attorney_eyes_only"
There was a problem hiding this comment.
But 'historical tickets' are static
I'll add that term
I agree it might not be the best use case and that if it is not historical the data is dynamic and it makes more sense to use the app database for it like we do for our Support Assistant Starter App
| {{% alert color="info" %}} The **Consumption** tab is available for Model resources only.{{% /alert %}} | ||
|
|
||
| The **Token Consumption Monitor** shows detailed graphs of the token consumption used by the GenAI resource. Use this overview to see the current usage, insights on the usage per day, and to compare the current month with previous months. | ||
| The **Consumption** section provides outcomones of token consumption monitoring for each GenAI resource in a graphical way. Use this overview to see the current usage, insights on the usage per day, and to compare the current month with previous months. |
There was a problem hiding this comment.
I think it will be making sense to mention about the new bundle month. you can explain the bundle month as below.
The bundle month represents the period during which token consumption is tracked, beginning on the date of your last GenAI Resource plan entitlement reset and ending on the next reset date. This creates a recurring monthly cycle based on your plan activation date, not the calendar month.
Made sentence more precise
Specified Search -> Semantic Search
removed 'against entitlements' - because we do not do that yet
CRI clarification based on PV's feedback
bundle months clarification
Clarified the term 'Model resources'
Data Separation through Metadata
added diagram for data separation through metadata
added diagram for data separation through metadata
4 participants
No description provided.