diff --git a/docs/docs/extraction/custom-metadata.md b/docs/docs/extraction/custom-metadata.md index 6ab5d96d51..ea721b63e6 100644 --- a/docs/docs/extraction/custom-metadata.md +++ b/docs/docs/extraction/custom-metadata.md @@ -4,7 +4,7 @@ You can upload custom metadata for documents during ingestion. By uploading custom metadata you can attach additional information to documents, and use it for filtering results during retrieval operations. For example, you can add author metadata to your documents, and filter by author when you retrieve results. -To create filters at query time, use predicates supported by [LanceDB SQL](https://lancedb.github.io/lancedb/sql/) against your table schema (custom fields are serialized into the `metadata` column with your ingested chunks). For a worked example, see the repository notebook linked at the end of this page. +To create filters at query time, use predicates supported by [LanceDB SQL](https://docs.lancedb.com/search/sql) against your table schema (custom fields are serialized into the `metadata` column with your ingested chunks). For a worked example, see the repository notebook linked at the end of this page. Use this documentation to use custom metadata to filter search results when you work with [NeMo Retriever Library](overview.md). @@ -177,4 +177,4 @@ print(f"{q_results}") ## Related Content - For a notebook that uses the CLI to add custom metadata and filter query results, see [metadata_and_filtered_search.ipynb -](https://github.com/NVIDIA/nv-ingest/blob/main/examples/metadata_and_filtered_search.ipynb). +](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/metadata_and_filtered_search.ipynb). diff --git a/docs/docs/extraction/faq.md b/docs/docs/extraction/faq.md index 3f3e763f7c..c2bcc4dc08 100644 --- a/docs/docs/extraction/faq.md +++ b/docs/docs/extraction/faq.md @@ -12,8 +12,8 @@ This documentation contains the Frequently Asked Questions (FAQ) for [NeMo Retri You can use the CLI or Python APIs to perform extraction only, and then consume the results. Using the Python API, `results` is a list object with one entry. -For code examples, see the Jupyter notebooks [Multimodal RAG with LlamaIndex](https://github.com/NVIDIA/nv-ingest/blob/main/examples/llama_index_multimodal_rag.ipynb) -and [Multimodal RAG with LangChain](https://github.com/NVIDIA/nv-ingest/blob/main/examples/langchain_multimodal_rag.ipynb). +For code examples, see the Jupyter notebooks [Multimodal RAG with LlamaIndex](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/llama_index_multimodal_rag.ipynb) +and [Multimodal RAG with LangChain](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/langchain_multimodal_rag.ipynb). @@ -61,7 +61,7 @@ For production environments, you should use the provided Helm charts. For [libra For advanced scenarios, you might want to use library mode with self-hosted NIM instances. You can set custom endpoints for each NIM. -For examples of `*_ENDPOINT` variables, refer to [nv-ingest/docker-compose.yaml](https://github.com/NVIDIA/nv-ingest/blob/main/docker-compose.yaml). +For examples of `*_ENDPOINT` variables, refer to [nv-ingest/docker-compose.yaml](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/docker-compose.yaml). diff --git a/docs/docs/extraction/notebooks.md b/docs/docs/extraction/notebooks.md index 1a4e7363cb..08fd432637 100644 --- a/docs/docs/extraction/notebooks.md +++ b/docs/docs/extraction/notebooks.md @@ -9,7 +9,7 @@ To get started using [NeMo Retriever Library](overview.md), you can try one of t ## Dataset Downloads for Benchmarking -If you plan to run benchmarking or evaluation tests, you must download the [Benchmark Datasets (Bo20, Bo767, Bo10k)](https://github.com/NVIDIA/nv-ingest/blob/main/evaluation/digital_corpora_download.ipynb) from Digital Corpora. This is a prerequisite for all benchmarking operations. +If you plan to run benchmarking or evaluation tests, you must download the [Benchmark Datasets (Bo20, Bo767, Bo10k)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/evaluation/digital_corpora_download.ipynb) from Digital Corpora. This is a prerequisite for all benchmarking operations. ## Getting Started @@ -17,17 +17,17 @@ To get started with the basics, try one of the following guides or notebooks: - [CLI Client Quick Start Guide](quickstart-guide.md#ingest_cli_example) — walkthrough for `nv-ingest-cli` / `retriever` with the self-hosted stack - [Python Client Quick Start Guide](quickstart-guide.md#ingest_python_example) — walkthrough for the Python client with the self-hosted stack -- [How to add metadata to your documents and filter searches](https://github.com/NVIDIA/nv-ingest/blob/main/examples/metadata_and_filtered_search.ipynb) -- [How to reindex a collection](https://github.com/NVIDIA/nv-ingest/blob/main/examples/reindex_example.ipynb) +- [How to add metadata to your documents and filter searches](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/metadata_and_filtered_search.ipynb) +- [How to reindex a collection](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/reindex_example.ipynb) For more advanced scenarios, try one of the following notebooks: -- [Build a Custom Vector Database Operator](https://github.com/NVIDIA/nv-ingest/blob/main/examples/building_vdb_operator.ipynb) -- [Try Enterprise RAG Blueprint](https://github.com/NVIDIA/nv-ingest/blob/main/deploy/pdf-blueprint.ipynb) -- [Evaluate bo767 retrieval recall accuracy with NV-Ingest and Milvus](https://github.com/NVIDIA/nv-ingest/blob/main/evaluation/bo767_recall.ipynb) -- [Multimodal RAG with LangChain](https://github.com/NVIDIA/nv-ingest/blob/main/examples/langchain_multimodal_rag.ipynb) -- [Multimodal RAG with LlamaIndex](https://github.com/NVIDIA/nv-ingest/blob/main/examples/llama_index_multimodal_rag.ipynb) +- [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/building_vdb_operator.ipynb) +- [Try Enterprise RAG Blueprint](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/deploy/pdf-blueprint.ipynb) +- [Evaluate bo767 retrieval recall accuracy with NV-Ingest and Milvus](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/evaluation/bo767_recall.ipynb) +- [Multimodal RAG with LangChain](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/langchain_multimodal_rag.ipynb) +- [Multimodal RAG with LlamaIndex](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/llama_index_multimodal_rag.ipynb) diff --git a/docs/docs/extraction/quickstart-guide.md b/docs/docs/extraction/quickstart-guide.md index 5b29256965..56ec379f12 100644 --- a/docs/docs/extraction/quickstart-guide.md +++ b/docs/docs/extraction/quickstart-guide.md @@ -5,7 +5,7 @@ This guide helps you get started using [NeMo Retriever Library](overview.md) in ## Step 1: Start Containers -Use the provided [docker-compose.yaml](https://github.com/NVIDIA/nv-ingest/blob/main/docker-compose.yaml) to start all needed services with a few commands. +Use the provided [docker-compose.yaml](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/docker-compose.yaml) to start all needed services with a few commands. !!! warning @@ -16,11 +16,11 @@ If you prefer, you can run on Kubernetes by using [our Helm chart](https://githu a. Git clone the repo: - `git clone https://github.com/nvidia/nv-ingest` + `git clone -b 26.03 https://github.com/NVIDIA/NeMo-Retriever` b. Change the directory to the cloned repo by running the following code. - `cd nv-ingest`. + `cd NeMo-Retriever`. c. [Generate API keys](ngc-api-key.md) and authenticate with NGC with the `docker login` command. @@ -58,7 +58,7 @@ f. Start core services. By default, the pipeline uses **LanceDB** as the vector !!! tip - By default, we have [configured log levels to be verbose](https://github.com/NVIDIA/nv-ingest/blob/main/docker-compose.yaml). It's possible to observe service startup proceeding. You will notice a lot of log messages. Disable verbose logging by configuring `NIM_TRITON_LOG_VERBOSE=0` for each NIM in [docker-compose.yaml](https://github.com/NVIDIA/nv-ingest/blob/main/docker-compose.yaml). + By default, we have [configured log levels to be verbose](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/docker-compose.yaml). It's possible to observe service startup proceeding. You will notice a lot of log messages. Disable verbose logging by configuring `NIM_TRITON_LOG_VERBOSE=0` for each NIM in [docker-compose.yaml](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/docker-compose.yaml). !!! tip @@ -431,7 +431,7 @@ python src/util/image_viewer.py --file_path ./processed_docs/image/multimodal_te !!! tip - Beyond inspecting the results, you can read them into things like [llama-index](https://github.com/NVIDIA/nv-ingest/blob/main/examples/llama_index_multimodal_rag.ipynb) or [langchain](https://github.com/NVIDIA/nv-ingest/blob/main/examples/langchain_multimodal_rag.ipynb) retrieval pipelines. Also, checkout our [Enterprise RAG Blueprint on build.nvidia.com](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag) to query over document content pre-extracted with the retriever pipeline. + Beyond inspecting the results, you can read them into things like [llama-index](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/llama_index_multimodal_rag.ipynb) or [langchain](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/examples/langchain_multimodal_rag.ipynb) retrieval pipelines. Also, checkout our [Enterprise RAG Blueprint on build.nvidia.com](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag) to query over document content pre-extracted with the retriever pipeline. @@ -474,7 +474,7 @@ Ensure the same image tags and `docker-compose.yaml` version are used in both en ## Docker Compose override files -The default [docker-compose.yaml](https://github.com/NVIDIA/nv-ingest/blob/main/docker-compose.yaml) might exceed VRAM on a single GPU for some hardware. Override files reduce per-service memory, batch sizes, or concurrency so the full pipeline can run on the available GPU. To use an override, pass a second `-f` file after the base compose file; Docker Compose merges them and the override takes precedence. +The default [docker-compose.yaml](https://github.com/NVIDIA/NeMo-Retriever/blob/26.03/docker-compose.yaml) might exceed VRAM on a single GPU for some hardware. Override files reduce per-service memory, batch sizes, or concurrency so the full pipeline can run on the available GPU. To use an override, pass a second `-f` file after the base compose file; Docker Compose merges them and the override takes precedence. | Override file | GPU target | |---------------|------------|