Observability
=============

See detailed status and progress output for all auto-processing modes.

Two views cover most observability needs:

- :ref:`aidb.pipeline_metrics <aidb.pipeline_metrics>`  (alias ``aidb.pipem`` ) — one row per pipeline with
  backlog, source/destination counts, status, and error counts. This is
  the primary view for tracking processing progress.

- :ref:`aidb.knowledge_base_metrics <aidb.knowledge_base_metrics>`  (alias ``aidb.kbm`` ) — one row per knowledge base with
  pipeline count, embedding count, and aggregated status across all
  attached pipelines.

Per-pipeline progress
^^^^^^^^^^^^^^^^^^^^^

.. code:: sql

   SELECT * FROM aidb.pipem;
   __OUTPUT__
         pipeline       | auto processing | table: unprocessed rows | volume: scans completed | count(source records) | count(destination records) |  Status  | count(record errors) | count(blocking errors)
   - --------------------+-----------------+-------------------------+-------------------------+-----------------------+----------------------------+----------+----------------------+------------------------
    feedback_na_pipe    | Live            |                       0 |                         |                   412 |                        412 | UpToDate |                    0 |                      0
    product_docs_pipe   | Background      |                         |                       6 |                   1842 |                       1842 | UpToDate |                    0 |                      0
   (2 rows)

The :ref:`변화 감지 <변화 감지>`  mechanism is different for volume and table sources,
so the view exposes these metrics:

- ``table: unprocessed rows`` : How many unique rows are listed in the
  backlog of change events. Auto-processing captures no new change
  events when ``Disabled`` .

- ``volume: scans completed`` : How many full listings of the source
  have been completed.

- ``count(source records)`` : How many records exist in the source.
  Accurate for table sources at all times; updated after each full scan
  for volume sources.

- ``count(destination records)`` : How many rows the pipeline has
  written to its destination (embeddings, chunks, and so on).

Knowledge base overview
^^^^^^^^^^^^^^^^^^^^^^^

.. code:: sql

   SELECT * FROM aidb.kbm;
   __OUTPUT__
                name             | pipelines | embeddings |  status
   - -----------------------------+-----------+------------+----------
    public.product_catalog_kb    |         1 |       1842 | UpToDate
    public.customer_feedback_kb  |         2 |        915 | UpToDate
   (2 rows)

The ``status`` column rolls up the worst pipeline status across all
pipelines attached to that KB — drill into ``aidb.pipeline_metrics`` to
find which pipeline is responsible.