Getting Started# Model Serving Concepts Before you start Why it matters Core concepts InferenceService (via KServe) Predictor ServingRuntime / ClusterServingRuntime How it works Lifecycle flow Key features Patterns of use Gen AI Builder Knowledge Bases Custom applications Hybrid + Sovereign AI alignment Best practices In AI Factory Related topics Next steps Gen AI Factory LLM Architecture (Library and Serving Overview) Purpose and Scope Architectural Principles Separation of Concerns Governance Integration Resource Optimization Control and Data Flow Flow Components Core Components Image & Model Library Model Library (Curated View) LLM Serving Infrastructure Access Management System Flow LLM Ingestion Pipeline Curation Process Deployment Workflow Runtime Operations Resource Management GPU Infrastructure for LLMs LLM Scaling Strategies Performance Optimization for LLMs Security and Governance Access Control Framework Compliance Integration Audit and Monitoring Integration Patterns Development Workflow API Compatibility Advanced Capabilities Multi-LLM Framework Support Custom Runtime Configuration Next Steps Deployment Overview Deployment flow overview Deployment components KServe InferenceService ClusterServingRuntime Where to start Hybrid Manager integration Best practices Next steps AI Factory Model Use Cases and Personas Persona-Based Implementation Scenarios Platform Engineers Infrastructure Management Scenario AI Engineers Model Deployment Optimization Scenario Application Developers RAG Application Development Scenario DevOps Engineers Automated Model Deployment Pipeline Scenario Security Engineers Secure Model Serving Architecture Scenario Cross-Functional Use Cases Enterprise Sovereign AI Implementation Hybrid Cloud Model Deployment Implementation Considerations Resource Planning Operational Complexity Performance Optimization Model Serving FAQ What models are currently supported? What Kubernetes resources are used for serving models? What hardware is required? Can I deploy custom models? How is scaling handled? How do I monitor running models? Where can I learn more about the model serving architecture?
