Observability#
See detailed status and progress output for all auto-processing modes.
A good place to get an overview is the statistics table. Use the view
aidb.knowledge_base_stats , or use its short alias aidb.kbstat . The view shows
all configured knowledge base pipelines, which processing mode is set,
and statistics about the processed records:
SELECT * from aidb.kbstat;
__OUTPUT__
knowledge base | auto processing | table: unprocessed rows | volume: scans completed | count(source records) | count(embeddings)
- -----------------------+-----------------+-------------------------+-------------------------+-----------------------+-------------------
kb_table_text_bg | Background | 0 | | 15 | 15
kb_table_text_manual | Disabled | 0 | | 15 | 15
kb_table_image_manual | Disabled | 0 | | 3 | 3
kb_table_text_live | Live | 0 | | 15 | 15
kb_table_image_bg | Background | 0 | | 3 | 3
kb_volume_text_bg | Background | | 6 | 7 | 7
kb_volume_text_manual | Disabled | | 0 | 0 | 0
kb_volume_image_bg | Background | | 4 | 177 | 6
kb_volume_image_manual | Disabled | | 1 | 177 | 6
(9 rows)
The Change detection mechanism is central to how auto-processing works. It’s different for volume and table sources. For this reason, the stats table has different columns for these two source types.
table: unprocessed rows: How many unique rows are listed in the backlog of change events. - If auto-processing is disabled, no new change events are captured.volume: scans completed: How many full listings of the source have been completed so far.count(source records): How many records exist in the source for this pipeline.For table sources, this number is always accurate.
For volume sources, this number is updated only after a full scan has completed.
count(embeddings): How many embeddings exist in the vector destination table for this pipeline.