Knowledge Bases Reference Manual#
Knowledge Bases Reference Manual#
Knowledge Bases provide semantic search infrastructure that transforms organizational content into queryable knowledge repositories, enabling AI assistants to access and utilize specific organizational knowledge through retrieval‑augmented generation workflows. They serve as the operational interface between processed content and intelligent applications requiring contextual information.
Table of contents
Getting Started#
Create a KB: Create a Knowledge Base in Gen AI Builder
Prepare sources: Configure Data Sources in Gen AI Builder (Confluence, Google Drive, S3, Web Page, Data Lake, Custom)
Configure storage: Configure the Data Lake in Gen AI Builder if using object storage ingestion
Manage and update: Manage Knowledge Bases in Gen AI Builder
Connect to apps: use Retrievers Reference Manual and Assistants Reference Manual
Practical Scenarios#
Policy and procedures KB (enterprise internal): centralize HR/IT/Legal docs with strong citations. - Build steps: connect sources via Configure Data Sources in Gen AI Builder → Create a Knowledge Base in Gen AI Builder → validate retrieval with golden questions → attach via Create a Retriever in Gen AI Builder .
Product documentation KB (support): ground agent answers on manuals and FAQs. - Build steps: ingest docs (S3/Drive/Confluence) → Create a Knowledge Base in Gen AI Builder → tune chunking/metadata → link to a support Create an Assistant in Gen AI Builder .
Hybrid catalog KB (structured + unstructured): combine product attributes (structured) with descriptions/reviews (unstructured). - Build steps: prepare structured columns and unstructured text → create a Hybrid KB in Create a Knowledge Base in Gen AI Builder → expose filters in your Create a Retriever in Gen AI Builder .
Analytics RAG KB: enable semantic exploration of reports and wiki knowledge. - Build steps: prepare embeddings via Pipelines and data preparation or Data Lake + loaders → Create a Knowledge Base in Gen AI Builder → test relevance and latency; integrate with an Create an Assistant in Gen AI Builder .
Architectural Purpose#
Knowledge Bases function as the semantic search layer that converts static content collections into dynamic, queryable resources supporting AI applications with contextual understanding capabilities. They implement vector-based similarity search combined with metadata filtering to provide relevant information for language model context enhancement.
System Integration Framework#
Gen AI Builder Pipeline Position Knowledge Bases occupy the critical position between content processing and application consumption within the AI Factory ecosystem:
Libraries → Knowledge Bases → Retrievers → Assistants → Applications
↓ ↓ ↓ ↓ ↓
Content Semantic Search Context User
Processing Indexing Operations Integration Experience
Infrastructure Integration
Vector Engine: Leverages PostgreSQL-based vector storage for high-performance semantic search operations
Libraries: Consumes processed content with embedded vectors and enriched metadata
AI Accelerator Pipelines: Utilizes embedding models for semantic indexing and content transformation
Model Serving: Accesses language models for query understanding and response generation
Technical Architecture#
Vector Storage Framework#
Semantic Indexing Knowledge Bases implement sophisticated vector storage that captures semantic meaning through high-dimensional embeddings generated by specialized language models optimized for organizational content characteristics.
Storage Optimization Vector storage utilizes PostgreSQL’s native vector capabilities through the Vector Engine, providing enterprise-grade performance, scalability, and reliability while maintaining comprehensive query capabilities and operational management features.
Search Capabilities#
Similarity Algorithms Knowledge Bases implement multiple similarity measurement approaches including cosine similarity, euclidean distance, and hybrid scoring mechanisms that combine semantic understanding with organizational priority factors.
Query Processing Pipeline Search operations follow systematic processing that optimizes result relevance while maintaining computational efficiency:
Query Vectorization: User queries transformed into semantic embeddings using organizational embedding models
Similarity Computation: Vector similarity calculations across indexed content collections
Metadata Filtering: Result refinement based on access controls, content classification, and organizational policies
Relevance Ranking: Multi-factor scoring incorporating semantic similarity, content recency, and source authority
Context Assembly: Result selection and formatting optimized for language model context integration
Content Organization Framework#
Indexing Strategies#
Content Segmentation Knowledge Bases implement intelligent content chunking that preserves semantic coherence while optimizing for retrieval granularity and language model context windows.
Hierarchical Organization Content structure preservation maintains document relationships, section hierarchies, and cross-reference connections enabling comprehensive information discovery and accurate source attribution.
Metadata Integration#
Classification Systems Comprehensive metadata frameworks support organizational classification schemes including content types, access levels, subject classifications, and temporal relevance indicators.
Access Control Integration Metadata-based access control ensures appropriate information exposure aligned with organizational security policies and user permission frameworks.
Search Operation Types#
Semantic Search#
Vector Similarity Pure semantic search operations identify content based on conceptual similarity regardless of keyword matching, enabling discovery of relevant information through meaning rather than literal text matching.
Contextual Understanding Advanced semantic search incorporates conversation context, user intent, and organizational domain knowledge for enhanced relevance and accuracy in information discovery.
Hybrid Search Operations#
Combined Approaches Hybrid search strategies integrate vector similarity with traditional keyword matching, metadata filtering, and organizational priority factors for optimal balance between semantic understanding and precise information retrieval.
Performance Optimization Intelligent query routing selects optimal search strategies based on query characteristics, content types, and performance requirements ensuring consistent response times while maximizing result relevance.
Filtered Search#
Metadata-Based Filtering Advanced filtering capabilities enable precise content selection based on organizational classification, access permissions, content recency, and source credibility indicators.
Dynamic Filtering Context-aware filtering adjusts search scope based on user roles, conversation context, and organizational policies ensuring appropriate information exposure while maintaining comprehensive coverage.
Performance Characteristics#
Scalability Framework#
Horizontal Scaling Knowledge Base architecture supports horizontal scaling across multiple PostgreSQL instances while maintaining search consistency and performance characteristics through intelligent load distribution.
Query Optimization Advanced query planning optimizes search operations for response time consistency across growing content volumes through intelligent indexing strategies and caching mechanisms.
Resource Management#
Memory Optimization Vector storage optimization balances comprehensive content coverage with memory efficiency through intelligent indexing strategies and storage compression techniques.
Computational Efficiency Search operations optimize computational resource utilization through intelligent query planning, result caching, and parallel processing capabilities.
Quality Assurance#
Content Quality Management#
Ingestion Validation Systematic content validation during knowledge base creation ensures information accuracy, completeness, and appropriate formatting for optimal search effectiveness.
Ongoing Quality Assessment Continuous monitoring of search effectiveness and content quality supports systematic improvement through performance analytics and user feedback integration.
Search Effectiveness#
Relevance Monitoring Comprehensive assessment of search result relevance supports continuous optimization through systematic analysis of user interactions, feedback patterns, and task completion rates.
Performance Tracking Detailed performance monitoring tracks search response times, result accuracy, and resource utilization patterns supporting capacity planning and optimization decisions.
Implementation Patterns#
Enterprise Knowledge Systems#
Organizations implement comprehensive knowledge bases that unify diverse content sources while maintaining appropriate access controls and organizational governance requirements.
Unified Access Architecture
Cross-departmental content integration through centralized knowledge base infrastructure
Role-based access control ensuring appropriate information exposure
Comprehensive audit trails supporting compliance and operational oversight
Performance optimization supporting concurrent access and organizational scalability
Specialized Domain Applications#
Domain-specific implementations optimize knowledge bases for particular industries or use cases requiring specialized content organization and search characteristics.
Domain Optimization Features
Industry-specific embedding models optimized for domain terminology and concepts
Specialized metadata schemas supporting domain-specific classification and organization
Regulatory compliance integration ensuring industry-specific requirement adherence
Performance optimization for domain-specific query patterns and content characteristics
Multi-Modal Knowledge Integration#
Advanced implementations coordinate knowledge across diverse content types including documents, structured data, and multimedia resources while maintaining unified search experiences.
Cross-Modal Capabilities
Content type optimization ensuring appropriate handling across diverse media formats
Unified search interfaces providing consistent access across different information types
Performance optimization supporting diverse content processing and retrieval requirements
Quality assurance procedures ensuring consistent results across content modalities
Operational Considerations#
Deployment Management#
Configuration Management Knowledge Base deployment requires systematic configuration management ensuring consistent search behavior across development, staging, and production environments while supporting organizational governance requirements.
Update Procedures Content updates require coordinated index maintenance ensuring search effectiveness remains optimal while minimizing service disruption during content refresh operations.
Maintenance Framework#
Index Optimization Regular maintenance procedures ensure optimal search performance through index rebuilding, vector optimization, and storage consolidation supporting system efficiency and user experience.
Content Synchronization Systematic procedures coordinate content updates from Libraries with knowledge base refresh operations ensuring information currency while maintaining search performance characteristics.
Development Resources#
Configuration Procedures#
Knowledge Base Setup
Create a Knowledge Base in Gen AI Builder : Comprehensive knowledge base creation and configuration procedures
Manage Knowledge Bases in Gen AI Builder : Ongoing management and optimization techniques
Best practices for Hybrid Knowledge Bases : Advanced configuration for optimal search effectiveness
Integration Documentation#
System Connectivity
LibrariesΩ Manual : Content source configuration and processing optimization
Retrievers Reference Manual : Search interface configuration and optimization
Assistants Reference Manual : Knowledge base utilization within conversational systems
Technical Reference
Vector Store Drivers : Technical implementation details
Embedding Integration : Semantic indexing configuration
RAG Engine Reference : Advanced retrieval-augmented generation implementation
Knowledge Bases transform static organizational content into dynamic, intelligent resources that enable AI applications to provide contextually relevant, accurate responses grounded in specific organizational knowledge while maintaining appropriate access controls and performance characteristics.