Model Library Explained In Depth ================================ The **Model Library** in **AI Factory** is your central interface for discovering, managing, and deploying AI models within EDB PG AI. It is powered by the core **Image and Model Library** of Hybrid Manager (HM). The Model Library presents a curated, AI-focused view of model images validated for production use. Model images deployed through the Model Library power AI Factory features including: - **Gen AI Builder** Assistants and pipelines - **Knowledge Base pipelines** (embeddings, RAG) - **Other AI Factory applications** using Model Serving ``---`` Before you start ---------------- Prerequisites for understanding Model Library: - Basic familiarity with **Model Serving** and KServe concepts - Understanding of **InferenceService** and runtime models - Awareness of **Image and Model Library** architecture in Hybrid Manager - Helpful: Awareness of **Sovereign AI principles** Suggested path: - :ref:`Fundamental AI Concepts ` - :ref:`Model Serving Concepts ` ``---`` What is the Model Library? -------------------------- The Model Library: - Presents a **curated AI view** of the broader Image and Model Library - Allows users to deploy validated model images to the **Model Serving (KServe)** layer - Provides lifecycle management for models powering AI Factory experiences **Key point:** Model Library is not a separate registry. It is a controlled *view* on top of the **governed Image and Model Library**. All model images: - Flow through the core Asset Library path - Are subject to platform-wide **governance** and **security policies** - Become available in Model Library only when validated for AI Factory use ``---`` How does it work? ----------------- Architecture flow: ^^^^^^^^^^^^^^^^^^ :: Container Registry → Image and Model Library → Model Library → Model Serving (KServe) → AI Factory Workloads .. csv-table:: :header: Layer,Role :widths: 10,30 :align: left :class: longtable Container Registry,"Stores model container images (e.g., NVIDIA NIM, open models)" Image and Model Library,Single source of truth for all container images Model Library,AI-focused console for validated model images Model Serving (KServe),Runs deployed model instances as InferenceServices AI Factory Workloads,"Gen AI Builder, Knowledge Base pipelines, Assistants, AI applications" Workflow: ^^^^^^^^^ 1. Images are ingested into **Asset Library** (private or public registries). 2. Images validated for AI model serving appear in **Model Library**. 3. Users browse, select, and deploy models to **Model Serving** via KServe. 4. Deployed models power: - **Knowledge Base ingestion and retrieval** - **Gen AI Builder Assistants and pipelines** - **Other AI Factory capabilities** ``---`` Why use the Model Library? -------------------------- - **Unified governance** — All images flow through the same governance process. - **Security and auditability** — Registry integration, tag selection, and image audit are unified. - **Sovereign AI** — You control which models are used, how they are deployed, and where they run. - **Consistency** — Database and AI models share the same container image management system. - **Flexibility** — You can add your own model images via Asset Library path and control their availability in AI Factory. ``---`` What does the Model Library provide? ------------------------------------ Users can: - Browse available **AI model images** - View supported **tags and versions** - Deploy models to **Model Serving** infrastructure - Manage model deployment lifecycle - Understand **which models power current AI Factory workloads**, including: - **Knowledge Bases** (via AIDB ingestion and retrieval) - **Gen AI Builder** (for Assistants and pipelines) ``---`` Supported model types --------------------- .. csv-table:: :header: Model Type,Example Image :widths: 10,30 :align: left :class: longtable Text Completion,llama-3.3-nemotron-super-49b Text Embedding,arctic-embed-l Image Embedding,nvclip OCR,paddleocr Text Reranker,llama-3.2-nv-rerankqa-1b-v2 - Support for additional model types is planned (vision, multi-modal, RAG-optimized). - **Custom models** can be added via private registry integration. ``---`` Patterns of use --------------- Knowledge Bases ^^^^^^^^^^^^^^^ - Embedding and retrieval models used in **Knowledge Base indexing** and **RAG** pipelines. - Example: Use **arctic-embed-l** or **nvclip** models. Gen AI Builder Assistants ^^^^^^^^^^^^^^^^^^^^^^^^^ - Power Assistants with models deployed through Model Library: - LLMs for text generation (e.g., llama-3) - OCR or Vision models (e.g., paddleocr) - Custom tools using Model Serving endpoints Hybrid + Sovereign AI alignment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - All models run on **your infrastructure** via Hybrid Manager KServe layer. - You control: - Which models are deployed - Resource allocations (CPU/GPU) - Deployment topology - Observability and auditing Custom model integration ^^^^^^^^^^^^^^^^^^^^^^^^ - Use **Integrate Private Registry** flow to add models from your own registry. - Define **Repository Rules** to control image availability. - Ensure images pass governance checks before appearing in Model Library. ``---`` Best practices -------------- - Maintain **clear governance** over model images. - Use **Repository Rules** to manage visibility and compliance. - Monitor **model usage** and resource consumption in Hybrid Manager dashboards. - Version and document models used in production. - Test custom models thoroughly before enabling for AI Factory workloads. ``---`` Model Serving integration ------------------------- Models deployed from Model Library run on the **Model Serving (KServe)** layer: - Guided workflow provided in Model Library console: 1. Select model image and tag. 2. Configure runtime (replicas, resources, GPUs, etc.). 3. Deploy → creates **InferenceService** in HM Kubernetes environment. - Deployed models power: - **AIDB Knowledge Base ingestion** - **Gen AI Builder Assistants and pipelines** - **Other AI Factory services** ``---`` Related topics -------------- - :ref:`Model Library Reference Manual ` - :ref:`How-To Deploy AI Models from Model Library ` - :ref:`How-To Integrate Private Registry with Model Library ` - :ref:`How-To Define Repository Rules for Model Library ` - :ref:`Model Serving Concepts ` - :ref:`Pipelines models reference ` ``---`` Next steps ---------- - Start by browsing available models in your **Model Library**. - Deploy your first model for **Knowledge Base** ingestion or **Gen AI Builder Assistants**. - Explore **Model Serving How-To Guides** to optimize deployment. - Define governance and compliance policies for model image flow. ``---``