Gen AI Builder FAQ

This FAQ focuses on Gen AI Builder inside Hybrid Manager’s AI Factory. It covers common questions about models, knowledge bases, assistants, tools, SDK usage, security, performance, and operations.

Table of contents

• Typical use cases and patterns

• Which models/drivers are supported?

• When to use RAG vs. fine‑tuning?

• How do I create and update Knowledge Bases?

• What are Retrievers and how do I tune them?

• How do Assistants call Tools and Structures?

• Where does data live and how is access governed?

• How do I improve latency and control cost?

• How do I test, observe, and troubleshoot?

• Where to find SDK references and examples?

Which models/drivers are supported?

Gen AI Builder is driver‑based. You can use private/endpoints or hosted APIs via Prompt Drivers, Embedding Drivers, and Vector Store Drivers.

• Prompt Drivers: OpenAI, Anthropic, Google, and more (see reference: reference/sdk/drivers/index.mdx and per‑provider pages)

• Embedding Drivers: OpenAI, Google, NVIDIA NIM, Hugging Face, etc. (reference/sdk/drivers/embedding-drivers.mdx )

• Vector Store Drivers: pgvector (Postgres), local, Pinecone, Qdrant, Redis, and others (reference/sdk/drivers/vector-store-drivers.mdx )

• NVIDIA NIM: deploy as private endpoints via Model serving and inference

See also: AI Factory Models and How-To Deploy an NVIDIA NIM Microservices available on build.nvidia.comcontainer with KServe .

When to use RAG vs fine‑tuning?

Prefer RAG for living knowledge and governed answers; use fine‑tuning for tone/format/narrow tasks. Many production assistants combine both: retrieve context from Knowledge Bases Reference Manual and guide responses with Rulesets Manual . For modular pipelines, explore RAG Engines .

How do I create and update Knowledge Bases?

• Create: Create a Knowledge Base in Gen AI Builder and configure Configure Data Sources in Gen AI Builder (Confluence, Google Drive, S3, Web Page, Data Lake, Custom)

• Update: Manage Knowledge Bases in Gen AI Builder ; re‑sync when source Libraries change

• Storage: embeddings live in Postgres (pgvector) or a configured vector store

• Tuning: choose chunking and metadata that match your retrieval; validate with golden questions

SDK references: Data Loaders , Embedding Drivers , Vector Store Drivers .

What are Retrievers and how do I tune them?

Retrievers control how assistants fetch context: target KBs, max tokens, filters, re‑ranking.

• Create: Create a Retriever in Gen AI Builder

• Tune: similarity thresholds, top‑K, metadata filters, rerankers

• Advanced: modular RAG Engines with retrieval/rerank/response stages

SDK references: RAG Engines , Rerank Drivers .

How do Assistants call Tools and Structures?

Assistants orchestrate retrieval + generation + actions. Use Tools for external systems, or promote Structures (pipelines/workflows/agents) as callable Tools.

• Build assistants: Create an Assistant in Gen AI Builder with Create a Ruleset in Gen AI Builder , Create a Retriever in Gen AI Builder , and Create a Tool in Gen AI Builder

• Structures: package logic as a ZIP and deploy Create a Structure in Gen AI Builder ; expose as Tool or run standalone

• Memory and threads: see Threads Reference Manual and conversation memory drivers

SDK references: Structures Reference Manual , Tools Reference Manual , Assistant Drivers , Conversation Memory Drivers .

Where does data live and how is access governed?

• Documents: ingest via Configure Data Sources in Gen AI Builder ; store in governed Data Lake or Postgres

• Embeddings/vectors: Postgres pgvector (recommended) or another configured vector store

• Governance: enforce permissions in source systems; restrict tool usage per project; audit retrieval and tool calls with threads/logs

See: Configure the Data Lake in Gen AI Builder , Vector Engine (PGvector) .

How do I improve latency and control cost?

• Retrieval: reduce top‑K, improve chunking/metadata, use re‑rank selectively

• Generation: choose right model per route; stream responses; batch where safe

• Caching: memoize embeddings, hot retrievals, and tool outputs when possible

• Infra: colocate models and KBs; scale with Model Serving autoscaling

See SDK: Engines and Drivers. See Models: Model serving and inference .

How do I test, observe, and troubleshoot?

• Testing: golden sets, conversation playbooks, SDK unit tests

• Observability: thread logs, assistant/structure run events; optionally export to your observability stack

• Debugging: verify retrieval set first; inspect ruleset changes; re‑run the same structure/assistant with stored inputs

Docs: Threads Reference Manual , SDK Observability for Model Serving .

Typical use cases and patterns

• Enterprise knowledge assistants: KB + Retriever + Assistant + Tooling (tickets/CRM)

• Customer support copilots: policy/FAQ KBs + routing + guardrails via Rulesets

• Workflow bots: Structures + Tools for approvals, data enrichment, and reporting

• RAG for analytics: pgvector + Pipelines + Assistant for guided exploration

Explore: Best practices for Hybrid Knowledge Bases , Customer Service Assistant Implementation Guide .

Where to find SDK references and examples?

• SDK overview: Gen AI Builder SDK

• Data: Artifacts , Loaders , Chunkers

• Drivers: Prompt/Assistant , Embeddings and vector representations , Vector Stores

• Structures: Tasks , Pipelines and data preparation , Structures (workflows and building blocks)

• Tools: Tools Reference Manual

See also the product guides: Assistants Reference Manual , Knowledge Bases Reference Manual , Retrievers Reference Manual , Rulesets Manual , Structures Reference Manual , Tools Reference Manual , Threads Reference Manual .