External model connections ========================== External models are API-based models hosted outside your Postgres instance. AIDB connects to them by registering a model with ``aidb.create_model()`` , supplying the provider name, a configuration object, and credentials such as an API key, if required. !!!note The examples on this page pass credentials inline as ``credentials => '{"api_key": "..."}'::JSONB`` for brevity. ``config`` itself can never contain an ``api_key`` or ``basic_auth`` key at any nesting depth — ``aidb.create_model()`` rejects it — so always pass credentials through the separate ``credentials`` argument. To avoid storing the secret in the database at all, pass ``credentials_env`` instead, naming an environment variable to read it from at connection time. See :ref:`Credentials from environment variables ` . Once registered, an external model is used exactly like a default local model — by name in any AIDB SQL function or pipeline step. Validating a model at creation ------------------------------ By default, ``aidb.create_model()`` validates an external model as part of registration by sending a small probe request to the provider. This confirms the endpoint is reachable and the configuration and credentials are correct before the registration commits, surfacing problems like an invalid API key or wrong URL immediately instead of on first use. If the probe fails, the error is reported and no model is registered. .. code:: sql - - Validated by default: sends a test request to the provider now. SELECT aidb.create_model( my_openai_embedder, openai_embeddings, config => aidb.embeddings_config(model => text-embedding-3-small), credentials => {"api_key": "sk-..."}::JSONB ); Pass ``validate => false`` to skip the probe — for example, when the endpoint or credentials aren’t available yet, or to avoid the request during scripted setup: .. code:: sql SELECT aidb.create_model( my_openai_embedder, openai_embeddings, config => aidb.embeddings_config(model => text-embedding-3-small), credentials => {"api_key": "sk-..."}::JSONB, validate => false ); !!!note Validation sends a real request to the provider, so it requires network access from the Postgres host at creation time and may incur a small usage cost. You can validate a model later — or re-check an existing one — with :ref:`aidb.validate_model() ` . OpenAI-compatible endpoints --------------------------- AIDB supports any embedding or chat/completions service that implements the OpenAI API. This includes OpenAI itself, self-hosted servers like `Ollama `_ , and any other compliant endpoint. Embeddings ^^^^^^^^^^ Use the ``openai_embeddings`` provider with ``aidb.embeddings_config()`` . Override the ``url`` parameter to redirect requests to any OpenAI-compatible server: .. code:: sql - - OpenAI SELECT aidb.create_model( my_openai_embedder, openai_embeddings, config => aidb.embeddings_config( model => text-embedding-3-small, url => https://api.openai.com/v1 ), credentials => {"api_key": "sk-..."}::JSONB ); - - Local Ollama server SELECT aidb.create_model( my_ollama_embedder, openai_embeddings, config => aidb.embeddings_config( model => llama3.2, url => http://llama.local:11434/v1/embeddings ) ); .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `model`,TEXT,Required,Model identifier as expected by the API. `api_key`,TEXT,NULL,API key for authentication. `url`,TEXT,NULL,API endpoint URL. Defaults to OpenAI's endpoint. `basic_auth`,TEXT,NULL,Basic auth credentials (`user:password`). `max_concurrent_requests`,INTEGER,NULL,Maximum concurrent requests to the endpoint. `max_batch_size`,INTEGER,NULL,Maximum number of inputs per batch request. `input_type`,TEXT,NULL,Input type hint for encoding (provider-specific). `input_type_query`,TEXT,NULL,Input type hint for query encoding (provider-specific). `is_hcp_model`,BOOLEAN,NULL,Set to `true` if referencing a model running on HCP. Chat completions ^^^^^^^^^^^^^^^^ Use the ``openai_completions`` provider with ``aidb.completions_config()`` . The ``url`` parameter works the same way — omit it for OpenAI, or override it for any other compliant server: .. code:: sql SELECT aidb.create_model( my_openai_llm, openai_completions, config => aidb.completions_config( model => gpt-4o, temperature => 0.2 ), credentials => {"api_key": "sk-..."}::JSONB ); .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `model`,TEXT,Required,Model identifier. `api_key`,TEXT,NULL,API key for authentication. `url`,TEXT,NULL,API endpoint URL. Defaults to OpenAI's endpoint. `basic_auth`,TEXT,NULL,Basic auth credentials (`user:password`). `temperature`,DOUBLE PRECISION,NULL,Sampling temperature. `top_p`,DOUBLE PRECISION,NULL,Nucleus sampling threshold. `seed`,BIGINT,NULL,Random seed for reproducible outputs. `system_prompt`,TEXT,NULL,Default system prompt prepended to every request. `max_tokens`,JSONB,NULL,Max tokens config (from `aidb.max_tokens_config()`). `thinking`,BOOLEAN,NULL,Enable extended reasoning (supported models only). `max_concurrent_requests`,INTEGER,NULL,Maximum concurrent requests to the endpoint. `extra_args`,JSONB,NULL,Additional provider-specific arguments. `is_hcp_model`,BOOLEAN,NULL,Set to `true` if referencing a model running on HCP. !!!note When using pgvector indexing, be aware that `pgvector limits indexed vectors to 2000 dimensions `_ . If the model you choose produces more dimensions, use ``aidb.vector_index_disabled_config()`` in your pipeline step and manage the index manually. OpenAI Responses API -------------------- AIDB has two providers for OpenAI models — both work with the same model identifiers (``gpt-4o`` , ``gpt-5.x`` , and so on), so pick between them based on what you’re doing, not which models you need: .. csv-table:: :header: Provider,API,Best for,Tool calling :widths: 15,12,12,20 :align: left :class: longtable `openai_completions`,Chat Completions (`/v1/chat/completions`),"Simple, one-off text generation — `aidb.generate_text()`, summarization, pipeline steps.",Rejects `tools`/`tool_choice`/`response_format` in `aidb.generate_text()`'s `inference_config`. `openai_responses`,Responses API (`/v1/responses`),"Agentic workloads — multi-step reasoning, reliable tool calling, structured output.",Native — emits the API's real `tools`/`tool_choice` request fields and parses real tool-call responses back. Both providers can also back an :ref:`Managing agents ` agent — AIDB always gives the agent tool-calling ability, but *how* differs by provider. Backed by ``openai_responses`` , tool calls go over the wire as genuine Responses API tool calls. Backed by ``openai_completions`` (or almost any other provider), AIDB describes the available tools in the prompt text and parses a tool call back out of the model’s plain-text reply — a simulation, not the model’s own tool-calling behavior. ``openai_responses`` is the more reliable choice for agents; use ``openai_completions`` when you just need text out, not tool calling. .. code:: sql SELECT aidb.create_model( my_gpt, openai_responses, config => aidb.openai_responses_config(model => gpt-5.1), credentials => {"api_key": "sk-..."}::JSONB ); Azure AI Foundry ^^^^^^^^^^^^^^^^ Use ``openai_responses_azure`` for models hosted on Azure AI Foundry’s unified Responses API surface. It’s the same wire shape as direct OpenAI, but ``url`` is required since there’s no universal default — point it at your resource’s endpoint: .. code:: sql SELECT aidb.create_model( my_gpt_azure, openai_responses_azure, config => aidb.openai_responses_config( model => gpt-5.1, url => https://.openai.azure.com/openai/v1/responses ), credentials => {"api_key": "..."}::JSONB ); See :ref:`aidb.openai_responses_config ` for the full parameter list, and :ref:`Tool calling and structured output ` for using ``tools`` , ``tool_choice`` , and ``response_format`` with either provider. Anthropic Messages API ---------------------- The ``anthropic_messages`` provider connects to Anthropic’s native Messages API (``/v1/messages`` ), with the same native tool-calling support as ``openai_responses`` : real ``tools`` /``tool_choice`` request fields, real tool-call responses, and structured output via ``response_format`` . .. code:: sql SELECT aidb.create_model( my_claude, anthropic_messages, config => aidb.anthropic_messages_config(model => claude-opus-4-6), credentials => {"api_key": "sk-ant-..."}::JSONB ); Azure AI Foundry and AWS Bedrock ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Two more variants share the same config helper and wire body, differing only in auth and endpoint: - ``anthropic_messages_azure`` — Azure AI Foundry’s native Anthropic Messages API surface. ``url`` is required (your resource’s Messages endpoint). - ``anthropic_messages_bedrock`` — AWS Bedrock’s InvokeModel API, authenticated with a Bedrock API key (bearer token, not AWS SigV4). ``url`` is required and should be the regional ``bedrock-runtime`` base endpoint — AIDB appends the model ID to the path automatically. .. code:: sql - - Azure AI Foundry SELECT aidb.create_model( my_claude_azure, anthropic_messages_azure, config => aidb.anthropic_messages_config( model => claude-opus-4-6, url => https://.services.ai.azure.com/anthropic/v1/messages ), credentials => {"api_key": "..."}::JSONB ); - - AWS Bedrock SELECT aidb.create_model( my_claude_bedrock, anthropic_messages_bedrock, config => aidb.anthropic_messages_config( model => anthropic.claude-opus-4-6-v1:0, url => https://bedrock-runtime.us-east-1.amazonaws.com ), credentials => {"api_key": "..."}::JSONB ); !!!note Anthropic’s Messages API has no native structured-output field. When you use ``response_format`` , AIDB translates it into a forced call to a synthetic tool internally — functionally equivalent, but it means ``response_format`` can’t be combined with ``tools`` /``tool_choice`` in the same call. ``anthropic_messages`` is AIDB’s only provider built specifically for Claude — there’s no older ``anthropic_completions`` equivalent to ``openai_completions`` . If you’d rather reach Claude (or OpenAI, or other vendors) through a single unified gateway instead of a per-vendor provider, see :ref:`OpenRouter ` below; the tradeoff is the same as ``openai_completions`` above — OpenRouter models get simulated (prompt-injected), not native, tool calling when backing an agent. See :ref:`aidb.anthropic_messages_config ` for the full parameter list. NVIDIA NIM ---------- AIDB supports NVIDIA NIM microservices hosted on `build.nvidia.com `_ as well as NIM instances running in your own environment. Text generation (NIM completions) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Use the ``nim_completions`` provider with ``aidb.completions_config()`` — the same config helper ``openai_completions`` uses, since NIM’s completions endpoint is OpenAI-compatible. See :ref:`Chat completions ` above for the full parameter list. .. code:: sql SELECT aidb.create_model( my_nim_llm, nim_completions, config => aidb.completions_config( model => meta/llama-3.3-70b-instruct ), credentials => {"api_key": "nvapi-..."}::JSONB ); To use a NIM instance running in your own environment, pass ``url`` to ``aidb.completions_config()`` pointing at your NIM endpoint. After registering, run the model with ``aidb.generate_text()`` : .. code:: sql SELECT aidb.generate_text(my_nim_llm, Tell me a short, one sentence story); __OUTPUT__ generate_text - --------------------------------------------------------------------------------- As the clock struck midnight, a single tear fell from the porcelain dolls eye. (1 row) Multimodal embeddings (NIM CLIP) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Use the ``nim_clip`` provider with ``aidb.nim_clip_config()`` for joint text and image embeddings: .. code:: sql SELECT aidb.create_model( my_nim_clip, nim_clip, config => aidb.nim_clip_config( model => nvidia/nvclip ), credentials => {"api_key": "nvapi-..."}::JSONB ); ``aidb.nim_clip_config()`` parameters: .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `api_key`,TEXT,NULL,API key for NIM authentication. `model`,TEXT,NULL,NIM CLIP model identifier. `url`,TEXT,NULL,NIM endpoint URL override. `basic_auth`,TEXT,NULL,Basic auth credentials. `is_hcp_model`,BOOLEAN,NULL,Set to `true` if the model is running on HCP. OCR (NIM OCR) ^^^^^^^^^^^^^ Use the ``nim_paddle_ocr`` provider with ``aidb.nim_ocr_config()`` to extract text from images: .. code:: sql SELECT aidb.create_model( my_nim_ocr, nim_paddle_ocr, config => aidb.nim_ocr_config(), credentials => {"api_key": "nvapi-..."}::JSONB ); ``aidb.nim_ocr_config()`` parameters: .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `api_key`,TEXT,NULL,API key for NIM authentication. `model`,TEXT,NULL,NIM OCR model identifier. `url`,TEXT,NULL,NIM endpoint URL override. `basic_auth`,TEXT,NULL,Basic auth credentials. `is_hcp_model`,BOOLEAN,NULL,Set to `true` if the model is running on HCP. Reranking (NIM reranking) ^^^^^^^^^^^^^^^^^^^^^^^^^ Use the ``nim_reranking`` provider with ``aidb.nim_reranking_config()`` to re-score search results by relevance. See :ref:`Text reranking ` for usage with ``aidb.rerank_text()`` : .. code:: sql SELECT aidb.create_model( my_reranker, nim_reranking, config => aidb.nim_reranking_config( model => nvidia/nv-rerankqa-mistral-4b-v3 ), credentials => {"api_key": "nvapi-..."}::JSONB ); ``aidb.nim_reranking_config()`` parameters: .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `api_key`,TEXT,NULL,API key for NIM authentication. `model`,TEXT,NULL,NIM reranking model identifier. `url`,TEXT,NULL,NIM endpoint URL override. `basic_auth`,TEXT,NULL,Basic auth credentials. `is_hcp_model`,BOOLEAN,NULL,Set to `true` if the model is running on HCP. To get a NIM API key, create an account at `build.nvidia.com `_ , select a model, and generate a key from the model’s page. Google Gemini ------------- Use the ``gemini`` provider with ``aidb.gemini_config()`` . ``api_key`` is ``aidb.gemini_config()`` ’s first parameter and has no default, but pass it as ``NULL`` and supply the real key through ``create_model()`` ’s ``credentials`` argument instead — ``config`` can never contain an ``api_key`` field: .. code:: sql SELECT aidb.create_model( my_gemini, gemini, config => aidb.gemini_config( api_key => NULL, model => gemini-2.0-flash ), credentials => {"api_key": "AIza..."}::JSONB ); ``aidb.gemini_config()`` parameters: .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `api_key`,TEXT,Required,"Has no default, but pass `NULL` and supply the real key via `create_model()`'s `credentials` argument instead — `config` can never contain `api_key`." `model`,TEXT,NULL,"Gemini model identifier (for example, `gemini-2.0-flash`)." `url`,TEXT,NULL,API endpoint URL override. `max_concurrent_requests`,INTEGER,NULL,Maximum concurrent requests to the API. `thinking_budget`,INTEGER,NULL,Extended thinking token budget (Gemini 2.x models only). OpenRouter ---------- `OpenRouter `_ provides a unified API gateway to 200+ models from multiple providers. AIDB supports both chat completions and embeddings via OpenRouter. .. _chat-completions-1: Chat completions ^^^^^^^^^^^^^^^^ .. code:: sql SELECT aidb.create_model( my_or_chat, openrouter_chat, config => aidb.openrouter_chat_config(anthropic/claude-3-5-haiku), credentials => {"api_key": "sk-or-..."}::JSONB ); ``aidb.openrouter_chat_config()`` parameters: .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `model`,TEXT,Required,OpenRouter model identifier. `api_key`,TEXT,NULL,OpenRouter API key. `url`,TEXT,NULL,API endpoint URL override. `max_concurrent_requests`,INTEGER,NULL,Maximum concurrent requests. `max_tokens`,JSONB,NULL,Max tokens config (from `aidb.max_tokens_config()`). .. _embeddings-1: Embeddings ^^^^^^^^^^ .. code:: sql SELECT aidb.create_model( my_or_embedder, openrouter_embeddings, config => aidb.openrouter_embeddings_config(mistral/mistral-embed), credentials => {"api_key": "sk-or-..."}::JSONB ); ``aidb.openrouter_embeddings_config()`` parameters: .. csv-table:: :header: Parameter,Type,Default,Description :widths: 15,12,12,20 :align: left :class: longtable `model`,TEXT,Required,OpenRouter embeddings model identifier. `api_key`,TEXT,NULL,OpenRouter API key. `url`,TEXT,NULL,API endpoint URL override. `max_concurrent_requests`,INTEGER,NULL,Maximum concurrent requests. `max_batch_size`,INTEGER,NULL,Maximum inputs per batch request. See the :ref:`Models reference ` for full parameter details on all config helper functions.