AI Factory Models ================= .. _ai-factory-models-1: AI Factory Models ================= **AI Factory Models** provides comprehensive lifecycle management for AI models within EDB Postgres AI environments, delivering enterprise-grade governance and scalable inference capabilities through deep integration with Hybrid Manager infrastructure. System Overview --------------- AI Factory Models operates as a unified platform that bridges model governance and production deployment, ensuring organizational control over AI assets while enabling seamless integration across EDB PG AI capabilities. The system manages the complete model lifecycle from acquisition through retirement, providing consistent operational procedures and security controls. Core Components ^^^^^^^^^^^^^^^ **Model Library** Centralized governance system that manages model image repositories, approval workflows, and metadata tracking. The library ensures only validated models reach production environments while maintaining comprehensive audit trails for compliance requirements. **Model Serving Infrastructure** KServe-based deployment engine that transforms approved models into scalable inference endpoints within Kubernetes clusters managed by Hybrid Manager. The serving layer provides enterprise-grade operational characteristics including auto-scaling, health monitoring, and resource optimization. **Integration Framework** Standardized interfaces that connect model inference capabilities with AI Accelerator Pipelines, Knowledge Bases, and Gen AI Builder applications, ensuring consistent model access patterns across all EDB PG AI workloads. Strategic Benefits ------------------ Sovereign AI Implementation ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Organizations achieve complete control over their AI infrastructure by deploying models within their own Kubernetes environments rather than relying on external API services. This approach ensures sensitive data and proprietary models remain within organizational boundaries while maintaining enterprise-grade operational capabilities. Operational Consistency ^^^^^^^^^^^^^^^^^^^^^^^ Standardized deployment patterns and governance frameworks eliminate the complexity of managing diverse model types and frameworks. Teams benefit from unified operational procedures regardless of the underlying model architecture or intended use case. Scalable Resource Management ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Efficient GPU and compute resource allocation across multiple models optimizes infrastructure costs while maintaining performance guarantees. The system prevents resource underutilization through dynamic allocation strategies based on actual workload demands. Architecture Integration ------------------------ AI Factory Models integrates deeply with Hybrid Manager to provide comprehensive model lifecycle management within existing Kubernetes infrastructure. The system leverages Hybrid Manager’s cluster orchestration capabilities while extending them with AI-specific governance and deployment features. Hybrid Manager Dependencies ^^^^^^^^^^^^^^^^^^^^^^^^^^^ **Asset Library Integration** Model images, metadata, and version information are managed through Hybrid Manager’s Asset Library, providing centralized storage and governance capabilities that extend beyond traditional container image management. **Kubernetes Orchestration** Model serving workloads deploy as native Kubernetes resources through Hybrid Manager’s cluster management capabilities, ensuring consistent operational behavior and resource allocation across all deployed models. **Security Framework** Access control, network policies, and audit logging leverage Hybrid Manager’s security infrastructure while adding model-specific governance controls and compliance features. Operational Workflows --------------------- Model Lifecycle Management ^^^^^^^^^^^^^^^^^^^^^^^^^^ **Acquisition and Validation** Models enter the system through automated synchronization from private registries or manual registration processes. Each model undergoes security scanning, performance validation, and compliance verification before approval for production use. **Deployment and Scaling** Approved models deploy as InferenceServices that automatically scale based on demand patterns. The system optimizes resource allocation across multiple models while maintaining isolation and performance guarantees. **Monitoring and Maintenance** Comprehensive observability provides visibility into model performance, resource utilization, and operational health. Automated alerting and recovery procedures minimize service disruptions while maintaining detailed audit trails. Governance Framework ^^^^^^^^^^^^^^^^^^^^ **Approval Workflows** Configurable approval processes ensure models meet organizational standards before production deployment. The system supports complex multi-stakeholder workflows while automating routine approvals for models meeting predefined criteria. **Policy Enforcement** Security policies, resource constraints, and compliance requirements are enforced automatically across all model deployments. This ensures consistent application of organizational standards without manual intervention. **Audit and Compliance** Detailed logging and reporting capabilities support regulatory compliance requirements while providing comprehensive visibility into model usage patterns and operational activities. Learning Pathways ----------------- Foundation Understanding ^^^^^^^^^^^^^^^^^^^^^^^^ Begin with core concepts to establish architectural understanding before progressing to practical implementation: **Conceptual Framework** 1. :ref:`Model Serving Concepts ` - Understanding inference deployment patterns 2. :ref:`Gen AI Architecture (Entities and Flows) ` - System design and component relationships 3. :ref:`Model Library Explained In Depth ` - Governance and lifecycle management principles **System Design** 1. :ref:`Deployment Overview ` - Production deployment patterns and considerations 2. :ref:`Observability for Model Serving ` - Monitoring and operational visibility Practical Implementation ^^^^^^^^^^^^^^^^^^^^^^^^ Progress through hands-on implementation after establishing conceptual understanding: **Initial Setup** 1. :ref:`Quickstart — Use an Inference Endpoint with Python ` - Basic client integration 2. :ref:`How-To Setup GPU resources ` - Infrastructure preparation for model workloads 3. :ref:`How-To Configure a ClusterServingRuntime ` - Framework-specific deployment settings **Model Deployment** 1. :ref:`How-To Create an InferenceService for NVIDIA NIM Container ` - Basic model deployment procedures 2. :ref:`How-To Deploy an NVIDIA NIM Microservices available on build.nvidia.comcontainer with KServe ` - Specialized deployment patterns 3. :ref:`Accessing KServe endpoints (internal and external) ` - Internal and external connectivity **Advanced Operations** 1. :ref:`How-To Integrate Private Registry with Model Library ` - Custom model source configuration 2. :ref:`How-To Define Repository Rules for Model Library ` - Automated model discovery 3. :ref:`How-To Verify InferenceServices and GPU usage ` - Deployment validation and troubleshooting Reference Materials ^^^^^^^^^^^^^^^^^^^ **Technical References** - :ref:`Model Serving Reference Manual ` - Comprehensive deployment and configuration reference - :ref:`Model Library Reference Manual ` - Complete governance framework documentation - :ref:`Gen AI Builder FAQ ` - Common issues and implementation considerations **Integration Guides** - :ref:`AI Accelerator Pipelines ` - Pipeline model integration - :ref:`Knowledge Bases Reference Manual ` - Application development with served models - :ref:`Vector Engine (PGvector) ` - Embedding model integration Use Case Applications --------------------- Enterprise RAG Systems ^^^^^^^^^^^^^^^^^^^^^^ Deploy embedding models for semantic search and retrieval augmented generation workflows that require consistent model versions across development and production environments. Private LLM Deployment ^^^^^^^^^^^^^^^^^^^^^^ Implement internal language model services that process sensitive data within organizational boundaries while maintaining enterprise-grade operational capabilities. Multi-Modal AI Applications ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Integrate vision, language, and embedding models into unified applications that require coordinated inference capabilities across different model types. Hybrid Model Architectures ^^^^^^^^^^^^^^^^^^^^^^^^^^ Combine private and external model services within applications that require both proprietary models for sensitive operations and external services for general capabilities. Implementation Considerations ----------------------------- Infrastructure Requirements ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Model serving requires substantial compute resources, particularly GPU infrastructure for large language models and vision workloads. Organizations should plan capacity based on anticipated model sizes and throughput requirements. Network Architecture ^^^^^^^^^^^^^^^^^^^^ Hybrid Manager cluster connectivity affects both model deployment and runtime access patterns. Consider network latency, bandwidth requirements, and security boundaries when designing model serving architectures. Security Framework ^^^^^^^^^^^^^^^^^^ Model governance extends beyond traditional container security to include AI-specific concerns such as model provenance, data privacy, and inference audit trails. Implement comprehensive security controls aligned with organizational AI governance policies. Operational Complexity ^^^^^^^^^^^^^^^^^^^^^^ Managing multiple models across different frameworks requires substantial operational expertise. Establish clear procedures for deployment, monitoring, and troubleshooting before scaling to production workloads. -------------- AI Factory Models provides the foundation for enterprise AI operations through comprehensive model lifecycle management and scalable inference deployment within your controlled infrastructure environments. .. toctree:: :maxdepth: 3 Usage--model--Getting_Started--index Usage--model--Manuals--index Usage--model--Explainers--index Usage--model--How_to_set_up_serving--index Usage--model--How_to_deploy_model--index Usage--model--How_to_integrate_with_your_app--index Usage--model--How_to_operate_and_observe--index