diff --git a/docs/docs/extraction/audio.md b/docs/docs/extraction/audio.md index 9d664d0b29..6e3b2b0d84 100644 --- a/docs/docs/extraction/audio.md +++ b/docs/docs/extraction/audio.md @@ -27,7 +27,7 @@ to transcribe speech to text, which is then embedded by using the NeMo Retriever !!! important - Due to limitations in available VRAM controls in the current release, the parakeet-1-1b-ctc-en-us ASR NIM microservice must run on a [dedicated additional GPU](support-matrix.md). For the full list of requirements, refer to [Support Matrix](support-matrix.md). + Due to limitations in available VRAM controls in the current release, the RIVA ASR NIM microservice must run on a [dedicated additional GPU](support-matrix.md). For the full list of requirements, refer to [Support Matrix](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/support-matrix/support-matrix.html). This pipeline enables users to retrieve speech files at the segment level. diff --git a/docs/docs/extraction/benchmarking.md b/docs/docs/extraction/benchmarking.md index 8a18ce62ab..6c3df50e7f 100644 --- a/docs/docs/extraction/benchmarking.md +++ b/docs/docs/extraction/benchmarking.md @@ -35,7 +35,7 @@ Before you use this documentation, you need the following: ### Run Your First Test ```bash -# 1. Navigate to the nv-ingest-harness directory +# 1. Navigate to the harness directory cd tools/harness # 2. Install dependencies @@ -181,8 +181,8 @@ Example: # YAML active section has api_version: v2 # Dataset bo767 has extract_images: false # Override via environment variable (highest priority) -EXTRACT_IMAGES=true API_VERSION=v1 uv run nv-ingest-harness-run --case=e2e --dataset=bo767 -# Result: Uses bo767 path, but extract_images=true (env override) and api_version=v1 (env override) +EXTRACT_IMAGES=true API_VERSION=v2 uv run nv-ingest-harness-run --case=e2e --dataset=bo767 +# Result: Uses bo767 path, but extract_images=true (env override) and api_version=v2 (env override) ``` **Precedence Details:** @@ -636,7 +636,7 @@ tools/harness/artifacts/__UTC/ Enable per-document element breakdown: ```bash -uv run nv-ingest-harness-run --case=e2e --dataset=bo767 --doc-analysis +uv run nv-ingest-harness-run --case=e2e --doc-analysis ``` **Sample Output:** diff --git a/docs/docs/extraction/content-metadata.md b/docs/docs/extraction/content-metadata.md index 5a55b9e6a7..cb4bec7fa0 100644 --- a/docs/docs/extraction/content-metadata.md +++ b/docs/docs/extraction/content-metadata.md @@ -164,7 +164,7 @@ Describes the structural location of content within a document. | `span` | `int` | `-1` | Span identifier within a line, for finer granularity. | | `nearby_objects` | `NearbyObjectsSchema` | `NearbyObjectsSchema()` | Information about objects (text, images, structured data) near the current content. See [NearbyObjectsSchema](#nearbyobjectsschema). | -### `NearbyObjectsSchema` (Currently Unused) +### `NearbyObjectsSchema` (Currently Unused) {#nearbyobjectsschema} Container for different types of nearby objects. | Field | Type | Default Value | Description | @@ -243,7 +243,7 @@ Specific metadata for audio content. | `audio_transcript` | `str` | `""` | Transcript of the audio content. | | `audio_type` | `str` | `""` | Type or format of the audio (e.g., `mp3`, `wav`). | -### `ErrorMetadataSchema` (Currently Unused) +### `ErrorMetadataSchema` (Currently Unused) {#errormetadataschema} Metadata describing errors encountered during processing. | Field | Type | Default Value | Description | @@ -253,7 +253,7 @@ Metadata describing errors encountered during processing. | `source_id` | `str` | `""` | Identifier of the source item that caused the error, if applicable. | | `error_msg` | `str` | *Required* | The error message. | -### `InfoMessageMetadataSchema` (Currently Unused) +### `InfoMessageMetadataSchema` (Currently Unused) {#infomessagemetadataschema} Informational messages related to processing. | Field | Type | Default Value | Description | diff --git a/docs/docs/extraction/custom-metadata.md b/docs/docs/extraction/custom-metadata.md index 805aef6289..4256caf5c1 100644 --- a/docs/docs/extraction/custom-metadata.md +++ b/docs/docs/extraction/custom-metadata.md @@ -60,7 +60,7 @@ For more information about the `Ingestor` class, see [Use the Python API](nv-ing For more information about the `vdb_upload` method, see [Upload Data](data-store.md). ```python -from nv_ingest_client.client import Ingestor +from nv_ingest_client.client.interface import Ingestor hostname="localhost" collection_name = "nv_ingest_collection" @@ -142,7 +142,7 @@ you can use the `content_metadata` field to filter search results. The following example uses a filter expression to narrow results by department. ```python -from nv_ingest_client.util.milvus import nvingest_retrieval +from nv_ingest_client.util.vdb.milvus import nvingest_retrieval hostname="localhost" collection_name = "nv_ingest_collection" @@ -157,14 +157,14 @@ q_results = [] for que in queries: q_results.append( nvingest_retrieval( - [que], - collection_name, - milvus_uri=f"http://{hostname}:19530", - embedding_endpoint=f"http://{hostname}:8012/v1", - hybrid=sparse, - top_k=top_k, - model_name=model_name, - gpu_search=False, + [que], + collection_name=collection_name, + milvus_uri=f"http://{hostname}:19530", + embedding_endpoint=f"http://{hostname}:8012/v1", + hybrid=sparse, + top_k=top_k, + model_name=model_name, + gpu_search=False, _filter=filter_expr ) ) diff --git a/docs/docs/extraction/faq.md b/docs/docs/extraction/faq.md index 40b5ea3765..3f3e763f7c 100644 --- a/docs/docs/extraction/faq.md +++ b/docs/docs/extraction/faq.md @@ -82,7 +82,7 @@ For more information, refer to [Extract Specific Elements from PDFs](nv-ingest-p ```python Ingestor(client=client) .files("data/multimodal_test.pdf") - .extract( + .extract( extract_text=True, extract_tables=True, extract_charts=True, diff --git a/docs/docs/extraction/prerequisites.md b/docs/docs/extraction/prerequisites.md index bdc6318664..b5cf408d3f 100644 --- a/docs/docs/extraction/prerequisites.md +++ b/docs/docs/extraction/prerequisites.md @@ -11,6 +11,7 @@ Before you begin using [NeMo Retriever Library](overview.md), ensure the followi ## Software Requirements - Linux operating systems (Ubuntu 22.04 or later recommended) +- **Python 3.12 or later** (required for NeMo Retriever Library packages; see note below) - [Docker](https://docs.docker.com/engine/install/) - [Docker Compose](https://docs.docker.com/compose/install/) - [Docker Buildx](https://docs.docker.com/build/concepts/overview/#buildx) `>= 0.17` (Compose 2.40+ enforces this) @@ -22,7 +23,7 @@ Before you begin using [NeMo Retriever Library](overview.md), ensure the followi !!! note - When you use UV, create the environment with Python 3.12 or later — for example, `uv venv --python 3.12`. This matches the `requires-python` metadata in the library packages. + Install **Python 3.12 or later** before creating your environment. Using Python 3.10 or 3.11 will cause dependency resolution failures when installing NeMo Retriever Library packages. diff --git a/docs/docs/extraction/python-api-reference.md b/docs/docs/extraction/python-api-reference.md index f3296118f0..5c2749b167 100644 --- a/docs/docs/extraction/python-api-reference.md +++ b/docs/docs/extraction/python-api-reference.md @@ -80,7 +80,7 @@ The caption task can call a vision-language model (VLM) with the following optio Example: ```python -from nemo_retriever.client.interface import Ingestor +from nv_ingest_client.client.interface import Ingestor ingestor = ( Ingestor() @@ -224,7 +224,7 @@ The `extract` method enables different types of data to be extracted. Use the following code to extract a single PDF file. ```python -from nemo_retriever.client.interface import Ingestor +from nv_ingest_client.client.interface import Ingestor # Initialize Ingestor with a local PDF file ingestor = Ingestor().files("path/to/document.pdf") @@ -527,7 +527,7 @@ The caption task can call a VLM with optional prompt and reasoning overrides: Example: ```python -from nemo_retriever.client.interface import Ingestor +from nv_ingest_client.client.interface import Ingestor ingestor = ( Ingestor() @@ -662,7 +662,7 @@ For more information on environment variables, refer to [Environment Variables]( Use the following code to extract mp3 audio content. ```python -from nemo_retriever.client import Ingestor +from nv_ingest_client.client.interface import Ingestor ingestor = Ingestor().files("audio_file.mp3") diff --git a/docs/docs/extraction/quickstart-guide.md b/docs/docs/extraction/quickstart-guide.md index 8905301bdc..e053c73c14 100644 --- a/docs/docs/extraction/quickstart-guide.md +++ b/docs/docs/extraction/quickstart-guide.md @@ -184,7 +184,8 @@ The following examples demonstrate how to extract text, charts, tables, and imag ```python import logging, os, time -from nv_ingest_client.client import Ingestor, NvIngestClient +from nv_ingest_client.client.interface import Ingestor +from nv_ingest_client.client import NvIngestClient from nv_ingest_client.util.process_json_files import ingest_json_results_to_blob client = NvIngestClient( message_client_port=7670, @@ -535,7 +536,7 @@ docker compose \ ## Specify MIG slices for NIM models -When you deploy the pipeline with NIM models on MIG‑enabled GPUs, MIG device slices are requested and scheduled through the `values.yaml` file for the corresponding NIM microservice. For IBM Content-Aware Storage (CAS) deployments, this allows NIM pods to land only on nodes that expose the desired MIG profiles [raw.githubusercontent](https://raw.githubusercontent.com/NVIDIA/nv-ingest/main/helm/README.md). +When you deploy NeMo Retriever Library with NIM models on MIG‑enabled GPUs, MIG device slices are requested and scheduled through the `values.yaml` file for the corresponding NIM microservice. For IBM Content-Aware Storage (CAS) deployments, this allows NeMo Retriever Library NIM pods to land only on nodes that expose the desired MIG profiles [raw.githubusercontent](https://raw.githubusercontent.com/NVIDIA/NeMo-Retriever/main/helm/README.md).​ To target a specific MIG profile—for example, a 3g.20gb slice on an A100, which is a hardware-partitioned virtual GPU instance that gives your workload a fixed mid-sized share of the A100’s compute plus 20 GB of dedicated GPU memory and behaves like a smaller independent GPU—for a given NIM, configure the `resources` and `nodeSelector` under that NIM’s values path in `values.yaml`. diff --git a/docs/docs/extraction/quickstart-library-mode.md b/docs/docs/extraction/quickstart-library-mode.md index 75a401066f..c3331a662e 100644 --- a/docs/docs/extraction/quickstart-library-mode.md +++ b/docs/docs/extraction/quickstart-library-mode.md @@ -13,10 +13,11 @@ The primary Python entry point for launching the Ray-based ingestion pipeline in ```python import time -from nemo_retriever.framework.orchestration.ray.util.pipeline.pipeline_runners import run_pipeline -from nemo_retriever.client import Ingestor, NemoRetrieverClient -from nemo_retriever.util.message_brokers.simple_message_broker import SimpleClient -from nemo_retriever.util.process_json_files import ingest_json_results_to_blob +from nv_ingest.framework.orchestration.ray.util.pipeline.pipeline_runners import run_pipeline +from nv_ingest_client.client.interface import Ingestor +from nv_ingest_client.client import NvIngestClient +from nv_ingest_api.util.message_brokers.simple_message_broker import SimpleClient +from nv_ingest_client.util.process_json_files import ingest_json_results_to_blob def main(): # Start the pipeline subprocess for library mode @@ -122,7 +123,7 @@ To query for relevant snippets of the ingested content, and use them with an LLM ```python import os from openai import OpenAI -from nemo_retriever.util.milvus import query +from nv_ingest_client.util.vdb.milvus import nvingest_retrieval milvus_uri = "milvus.db" collection_name = "test" @@ -130,16 +131,16 @@ sparse=False queries = ["Which animal is responsible for the typos?"] -retrieved_docs = query( +retrieved_docs = nvingest_retrieval( queries, - collection_name, + collection_name=collection_name, milvus_uri=milvus_uri, hybrid=sparse, top_k=1, ) # simple generation example -extract = retrieved_docs[0][0]["entity"]["text"] +extract = retrieved_docs[0][0].get("entity", retrieved_docs[0][0]).get("text", "") client = OpenAI( base_url = "https://integrate.api.nvidia.com/v1", api_key = os.environ["NVIDIA_API_KEY"] @@ -239,8 +240,8 @@ It listens for ingestion requests on port `7671` from an external client. import logging import os -from nemo_retriever.framework.orchestration.ray.util.pipeline.pipeline_runners import run_pipeline -from nemo_retriever.util.logging.configuration import configure_logging as configure_local_logging +from nv_ingest.framework.orchestration.ray.util.pipeline.pipeline_runners import run_pipeline +from nv_ingest_api.util.logging.configuration import configure_logging as configure_local_logging # Configure the logger logger = logging.getLogger(__name__) @@ -285,11 +286,11 @@ import logging import os import time -from nemo_retriever.framework.orchestration.ray.util.pipeline.pipeline_runners import run_pipeline -from nemo_retriever.util.logging.configuration import configure_logging as configure_local_logging -from nemo_retriever.util.message_brokers.simple_message_broker import SimpleClient -from nemo_retriever.client import Ingestor -from nemo_retriever.client import NemoRetrieverClient +from nv_ingest.framework.orchestration.ray.util.pipeline.pipeline_runners import run_pipeline +from nv_ingest_api.util.logging.configuration import configure_logging as configure_local_logging +from nv_ingest_api.util.message_brokers.simple_message_broker import SimpleClient +from nv_ingest_client.client.interface import Ingestor +from nv_ingest_client.client import NvIngestClient # Configure the logger logger = logging.getLogger(__name__) diff --git a/docs/docs/extraction/support-matrix.md b/docs/docs/extraction/support-matrix.md index b109be6918..0df56fa5c3 100644 --- a/docs/docs/extraction/support-matrix.md +++ b/docs/docs/extraction/support-matrix.md @@ -7,6 +7,11 @@ Before you begin using [NeMo Retriever Library](overview.md), ensure that you ha NVIDIA Ingest (nv-ingest) has been renamed NeMo Retriever Library. +## Software Requirements + +- **Python**: 3.12 or later. The NeMo Retriever Library core and harness require Python 3.12+; the client supports Python 3.11+. Using Python 3.10 or earlier will cause dependency resolution failures. For details, see [Prerequisites](prerequisites.md). + + ## Core and Advanced Pipeline Features The Nemo Retriever Library extraction core pipeline features run on a single A10G or better GPU. @@ -39,6 +44,7 @@ This includes the following: NeMo Retriever Library supports the following GPU hardware. - [RTX Pro 6000 Blackwell Server Edition](https://www.nvidia.com/en-us/data-center/rtx-pro-6000-blackwell-server-edition/) +- [RTX PRO 4500 Blackwell](https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-4500/) - [DGX B200](https://www.nvidia.com/en-us/data-center/dgx-b200/) - [H200 NVL](https://www.nvidia.com/en-us/data-center/h200/) - [H100 Tensor Core GPU](https://www.nvidia.com/en-us/data-center/h100/) diff --git a/docs/docs/extraction/user-defined-functions.md b/docs/docs/extraction/user-defined-functions.md index aef2c09c3a..e9ba7dffa1 100644 --- a/docs/docs/extraction/user-defined-functions.md +++ b/docs/docs/extraction/user-defined-functions.md @@ -304,7 +304,7 @@ UDFs can be executed at different stages of the pipeline by specifying the `targ - `broker_response` - Response message handling - `otel_tracer` - OpenTelemetry tracing -> **Note:** For the complete and up-to-date list of pipeline stages, see the [default_pipeline.yaml](../../../config/default_pipeline.yaml) configuration file. +> **Note:** For the complete and up-to-date list of pipeline stages, see the [default_pipeline.yaml](https://github.com/NVIDIA/nv-ingest/blob/main/config/default_pipeline.yaml) configuration file. #### Target Stage Selection Examples diff --git a/docs/docs/extraction/vlm-embed.md b/docs/docs/extraction/vlm-embed.md index 03d089e8c5..0e8a632718 100644 --- a/docs/docs/extraction/vlm-embed.md +++ b/docs/docs/extraction/vlm-embed.md @@ -1,7 +1,6 @@ # Use Multimodal Embedding with NeMo Retriever Library -This documentation describes how to use [NeMo Retriever Library](overview.md) -with the multimodal embedding model [Llama 3.2 NeMo Retriever Multimodal Embedding 1B](https://build.nvidia.com/nvidia/llama-3_2-nemoretriever-1b-vlm-embed-v1). +This guide explains how to use the [NeMo Retriever Library](overview.md) with the multimodal embedding model [Llama Nemotron Embed VL 1B v2](https://build.nvidia.com/nvidia/llama-nemotron-embed-vl-1b-v2). The `Llama 3.2 NeMo Retriever Multimodal Embedding 1B` model is optimized for multimodal question-answering retrieval. The model can embed documents in the form of an image, text, or a combination of image and text.