Local models
============

AIDB supports two kinds of local models — both run directly on your
Postgres host without requiring an external API:

- **Built-in models** — AIDB bundles a default variant of each supported
  model type and registers them automatically. No configuration is
  needed to start using them.

- **Custom local models** — If you need a different model variant, a
  fine-tuned model, or a specific local path, you can register a custom
  version using ``aidb.create_model()`` .

Built-in models
---------------

The following models are available in every AIDB installation without
any configuration:

.. csv-table::
  :header: Model name,Provider,Use
  :widths: 12,25,15
  :align: left
  :class: longtable

  `bert`,`bert_local`,Text embeddings
  `clip`,`clip_local`,Multimodal embeddings (text and image)
  `t5`,`t5_local`,Text-to-text generation
  `llama`,`llama_instruct_local`,Text-to-text generation
  `dummy`,`dummy`,Testing and development (returns zeros)

Use them directly by name in any AIDB SQL function or pipeline step
configuration:

.. code:: sql

   - - Text embedding with the built-in BERT model
   SELECT aidb.encode_text(bert, The quick brown fox);

   - - Image embedding with the built-in CLIP model
   SELECT aidb.encode_image(clip, pg_read_binary_file(/tmp/photo.jpg)::BYTEA);

   - - Text generation with the built-in T5 model
   SELECT aidb.decode_text(t5, Translate to French: Hello, world.);

Custom local models
-------------------

Use ``aidb.create_model()`` to register a custom variant when you need a
different model version, a fine-tuned model, or a specific local path:

.. code:: sql

   SELECT aidb.create_model(
       name     => my_model,
       provider => <provider>,
       config   => <config_helper>(...)
   );

.. csv-table::
  :header: Parameter,Type,Description
  :widths: 12,15,25
  :align: left
  :class: longtable

  `name`,TEXT,A unique name for this model. Used to reference it in pipelines and SQL functions.
  `provider`,TEXT,The model provider. Determines which inference engine AIDB uses.
  `config`,JSONB,"Provider-specific configuration, built with a config helper function."

BERT
^^^^

Use ``aidb.bert_config()`` to specify the HuggingFace model identifier,
an optional revision, and an optional local cache directory:

.. code:: sql

   SELECT aidb.create_model(
       my_bert,
       bert_local,
       config => aidb.bert_config(sentence-transformers/all-MiniLM-L6-v2)
   );

.. csv-table::
  :header: Parameter,Type,Default,Description
  :widths: 15,12,12,20
  :align: left
  :class: longtable

  `model`,TEXT,Required,Model name or path (HuggingFace identifier).
  `revision`,TEXT,NULL,Model revision or branch.
  `cache_dir`,TEXT,NULL,Local directory for caching model files.

CLIP
^^^^

Use ``aidb.clip_config()`` to configure a CLIP-based multimodal model
for joint text and image embeddings:

.. code:: sql

   SELECT aidb.create_model(
       my_clip,
       clip_local,
       config => aidb.clip_config(openai/clip-vit-base-patch32)
   );

.. csv-table::
  :header: Parameter,Type,Default,Description
  :widths: 15,12,12,20
  :align: left
  :class: longtable

  `model`,TEXT,Required,Model name or path (HuggingFace identifier).
  `revision`,TEXT,NULL,Model revision or branch.
  `cache_dir`,TEXT,NULL,Local directory for caching model files.

T5
^^

Use ``aidb.t5_config()`` to configure a T5 text-to-text model. T5 models
support tasks like translation, summarization, and question answering
depending on the variant:

.. code:: sql

   SELECT aidb.create_model(
       my_t5,
       t5_local,
       config => aidb.t5_config(google/flan-t5-base)
   );

.. csv-table::
  :header: Parameter,Type,Default,Description
  :widths: 15,12,12,20
  :align: left
  :class: longtable

  `model`,TEXT,Required,Model name or path.
  `model_path`,TEXT,NULL,Explicit local path to model weights (overrides `model`).
  `cache_dir`,TEXT,NULL,Local directory for caching model files.

Llama
^^^^^

Use ``aidb.llama_config()`` to configure a Llama-family language model
for text generation. Llama models support temperature, top-p sampling,
context length, and other generation parameters:

.. code:: sql

   SELECT aidb.create_model(
       my_llama,
       llama_instruct_local.,
       config => aidb.llama_config(
           meta-llama/Llama-3.2-3B-Instruct,
           temperature => 0.5
       )
   );

.. csv-table::
  :header: Parameter,Type,Default,Description
  :widths: 15,12,12,20
  :align: left
  :class: longtable

  `model`,TEXT,Required,Model name or path.
  `cache_dir`,TEXT,NULL,Local directory for caching model files.
  `model_path`,TEXT,NULL,Explicit local path to model weights (overrides `model`).
  `temperature`,DOUBLE PRECISION,NULL,Sampling temperature.
  `top_p`,DOUBLE PRECISION,NULL,Nucleus sampling threshold.
  `seed`,BIGINT,NULL,Random seed for reproducible outputs.

!!!Note Local model files are downloaded from HuggingFace and cached on
first use. The Postgres process must have network access and write
access to the cache directory. For air-gapped environments, pre-download
the model files and use ``model_path`` or ``cache_dir`` to point to the
local copy.

Local model downloads
---------------------

When a local model is used for the first time (or any time its cache is
missing or corrupt), AIDB downloads the model files from HuggingFace
into the configured ``cache_dir`` . The downloader emits progress
messages and automatically retries transient network failures.

Progress messages
^^^^^^^^^^^^^^^^^

For each file, AIDB emits a ``downloading`` line when the request starts
and a ``complete`` line when it finishes. For multi-gigabyte files, a
heartbeat line reports cumulative bytes and percent complete
approximately every 30 seconds, so the session doesn’t appear frozen
during large downloads.

.. code:: sql

   SELECT aidb.decode_text(t5, test);
   __OUTPUT__
   NOTICE:  [t5_local] loading t5-small (rev refs/pr/15) -> /var/.../snapshots/refs/pr/15
   NOTICE:  [hf] downloading https://huggingface.co/t5-small/resolve/refs%2Fpr%2F15/config.json (expected: 1.2 KB)
   NOTICE:  [hf] complete: https://huggingface.co/t5-small/resolve/refs%2Fpr%2F15/config.json (1.2 KB) in 0.9s
   NOTICE:  [hf] downloading https://huggingface.co/t5-small/resolve/refs%2Fpr%2F15/model.safetensors (size unknown)
   NOTICE:  [hf] downloading t5-small/model.safetensors: 115.4 MB / 230.8 MB (50.0%)
   NOTICE:  [hf] downloading t5-small/model.safetensors: 216.4 MB / 230.8 MB (93.7%)
   NOTICE:  [hf] complete: https://huggingface.co/t5-small/resolve/refs%2Fpr%2F15/model.safetensors (230.8 MB) in 83.5s

By default, these lines are sent to the client at ``NOTICE`` level. Use
the :ref:`aidb.download_log_level <aidb.download_log_level>`  parameter to route them to the server log instead, or
to suppress them entirely.

Retries and resume
^^^^^^^^^^^^^^^^^^

A CDN that drops the connection partway through a multi-gigabyte
download will not fail the request outright:

- Each retry sends a ``Range`` header from wherever the previous attempt
  stopped, so progress accumulates across attempts instead of restarting
  from zero.

- AIDB distinguishes “still receiving new bytes” from “zero progress”
  and only counts stalled attempts against :ref:`aidb.download_max_attempts <aidb.download_max_attempts>`  , with
  exponential backoff between attempts.

- A ``Ctrl-C`` from ``psql`` lands between retries instead of waiting
  for the current request to time out.

The total number of attempts per file is capped by :ref:`aidb.download_max_attempts <aidb.download_max_attempts>` 
(default ``30`` ).

Verification and self-heal
^^^^^^^^^^^^^^^^^^^^^^^^^^

AIDB verifies downloaded files using the size and (when available) the
SHA-256 hash advertised by HuggingFace, and writes a small sidecar file
next to each completed download recording those values. On every cache
hit AIDB rechecks the file size against the sidecar, so a file truncated
after the original download is caught before it’s used.

If a cached file is detected as corrupt at model-load time, AIDB purges
the affected snapshot and re-downloads it once automatically.

Using a registered model
------------------------

Once registered, a model is referenced by its name in any AIDB function.
Custom and built-in models are interchangeable:

.. code:: sql

   - - Use a custom BERT model for embedding
   SELECT aidb.encode_text(my_bert, Hello world);

   - - Use a custom BERT model in a pipeline step
   SELECT aidb.create_pipeline(
       name => my_pipeline,
       source => my_source_table,
       source_key_column => id,
       source_data_column => content,
       step_1 => KnowledgeBase,
       step_1_options => aidb.knowledge_base_config(model => my_bert, data_format => Text)
   );

See the :ref:`Models reference <Models reference>`  for full parameter details on all model config
helper functions.

For a list of HuggingFace model variants that have been tested and
verified to work with AIDB’s local providers, see :ref:`Support matrix <Support matrix>`  .