Observability
=============

See detailed status and progress output for all auto-processing modes.

A good place to get an overview is the statistics table. Use the view
:ref:`aidb.knowledge_base_stats <aidb.knowledge_base_stats>`  , or use its short alias ``aidb.kbstat`` . The view shows
all configured knowledge base pipelines, which processing mode is set,
and statistics about the processed records:

.. code:: sql

   SELECT * from aidb.kbstat;
   __OUTPUT__
        knowledge base     | auto processing | table: unprocessed rows | volume: scans completed | count(source records) | count(embeddings)
   - -----------------------+-----------------+-------------------------+-------------------------+-----------------------+-------------------
    kb_table_text_bg       | Background      |                       0 |                         |                    15 |                15
    kb_table_text_manual   | Disabled        |                       0 |                         |                    15 |                15
    kb_table_image_manual  | Disabled        |                       0 |                         |                     3 |                 3
    kb_table_text_live     | Live            |                       0 |                         |                    15 |                15
    kb_table_image_bg      | Background      |                       0 |                         |                     3 |                 3
    kb_volume_text_bg      | Background      |                         |                       6 |                     7 |                 7
    kb_volume_text_manual  | Disabled        |                         |                       0 |                     0 |                 0
    kb_volume_image_bg     | Background      |                         |                       4 |                   177 |                 6
    kb_volume_image_manual | Disabled        |                         |                       1 |                   177 |                 6
   (9 rows)

The :ref:`Change detection <Change detection>`  mechanism is central to how auto-processing works.
It’s different for volume and table sources. For this reason, the stats
table has different columns for these two source types.

- ``table: unprocessed rows`` : How many unique rows are listed in the
  backlog of change events. - If auto-processing is disabled, no new
  change events are captured.

- ``volume: scans completed`` : How many full listings of the source
  have been completed so far.

- ``count(source records)`` : How many records exist in the source for
  this pipeline.

  - For table sources, this number is always accurate.

  - For volume sources, this number is updated only after a full scan
    has completed.

- ``count(embeddings)`` : How many embeddings exist in the vector
  destination table for this pipeline.