Embedded offline context store for AI agents (OpenViking-inspired).
A URI-addressed tree with layered content (L0 abstract, L1 overview, L2 full content), keyword and vector search over it, append-only sessions that can be committed into the tree, and typed durable memories with their provenance.
Every function here is a stable entry point. The work behind them belongs to the application workflows, which reach the store and the models through replaceable ports rather than naming either directly -- so what a caller depends on stays the same when what serves it changes.
Starting
{:ok, _} = Application.ensure_all_started(:agent_db)
:ok = AgentDb.write("viking://resources/readme.md", "# Title")
{:ok, content} = AgentDb.read("viking://resources/readme.md")Layered content
A document is stored with its full content and, optionally, a caller's own abstract and overview. A layer that was not supplied is generated in the background, and a read resolves to the first thing available: the stored layer, then the generated one, then a deterministic fallback that needs no model at all.
Summary
Functions
Reads a document's L0 abstract, falling back to frontmatter identity or the first non-empty body line.
Appends a message to a session (role: :user | :assistant | :system).
Enqueues :embed jobs only for URIs missing in the active dim table.
Applies set_enabled to every URI in uris, attempting each one.
Answers per-URI outcomes plus summary counts.
Applies set_group to every URI in uris, attempting each one.
The size of the store's disposable read caches: entries and bytes per table.
Commits a session into the tree at destination_uri as one document.
Creates a session and returns its id.
The bounds one data transfer accepts, for a caller that has to bound its own input.
The confidence recorded when a caller supplies none.
Exports store data to a tar archive at path.
Why a data export or import was refused, as a sentence an operator can act on.
Discovers files and directories whose URI path contains query.
Removes a memory and its provenance. See AgentDb.Application.Memories.forget/1.
Returns every message of a session, in order.
Searches full document content (L2) for a literal substring.
Whether the store and its models are usable, checked individually.
Imports a transfer archive created by export_data/1 into the running store.
Imports Agent Skills into viking://user/{user_id}/skills.
How much of the store's content each index covers.
Reads a document's three LLM-facing layers together, each with its source.
Reads the L0/L1/L2 layers for several URIs in one store operation.
Lists the names of a URI's direct children.
Recursive document URIs under scope in deterministic order, paged.
Installed skill roots in URI order, paged. Each entry carries name,
owner, uri, files, enabled, and group_tag.
Possible conflicts between active memories. See AgentDb.Application.Memories.conflicts/1.
Alias for memory_statistics/0.
Memory statistics: counts per type and status, pending, never surfaced. See AgentDb.Application.Memories.statistics/0.
Alias for memory_statistics/0.
The memory types, for callers that need to enumerate the taxonomy.
How the store's models are doing, including a load in progress.
Operation, job, and model counts by outcome since this process started.
Reads a document's L1 overview, falling back to the first 280 characters of the body after frontmatter.
Candidates waiting for promotion review. See AgentDb.Application.Memories.pending_candidates/1.
Promotes the waiting candidate at uri to the active memory.
Drops a non-active vector dim table (vec_nodes_<dim>).
How far behind the queue is, and which jobs are failing.
How much background work is outstanding, by status.
Reads a document's full content (L2).
Reads memories back. See AgentDb.Application.Memories.recall/1.
Recent operational failures, newest first, with the operation they happened in and their classified reason. Bounded and in memory only.
Rejects the waiting candidate at uri, leaving no trace of it.
Records a durable fact as a memory. See AgentDb.Application.Memories.remember/3.
Removes the subtree at uri, from every store keyed by URI.
A bounded, read-only view of the BEAM runtime: uptime, applications, supervisors, process counts, ETS tables, memory, and scheduler figures.
Searches documents by :keyword, :vector, or :hybrid (default
:keyword).
Enables or disables the subtree at uri.
Assigns the operator group tag for the subtree at uri. Empty clears.
Why a skill import was refused, as a sentence an operator can act on.
The bounds one skill import accepts, for a caller that has to bound its own input.
What the store holds and how much room it takes on disk.
Reads a document's stored L0/L1 layers without fallback.
Subscribes the caller to changes beneath a viking:// URI.
A depth-limited projection of the tree at uri (default depth 2).
Unsubscribes the caller from a viking:// URI scope.
Writes a document at uri, creating missing parent directories.
Types
Functions
Reads a document's L0 abstract, falling back to frontmatter identity or the first non-empty body line.
Appends a message to a session (role: :user | :assistant | :system).
@spec backfill_vector_index() :: {:ok, non_neg_integer()} | {:error, term()}
Enqueues :embed jobs only for URIs missing in the active dim table.
Switching providers creates the new table if missing and backfills the diff; URIs already covered are never re-enqueued.
@spec bulk_set_enabled([uri()], boolean()) :: {:ok, %{results: [map()], updated: non_neg_integer(), failed: non_neg_integer()}}
Applies set_enabled to every URI in uris, attempting each one.
Answers per-URI outcomes plus summary counts.
@spec bulk_set_group([uri()], String.t()) :: {:ok, %{results: [map()], updated: non_neg_integer(), failed: non_neg_integer()}}
Applies set_group to every URI in uris, attempting each one.
@spec cache_stats() :: map()
The size of the store's disposable read caches: entries and bytes per table.
@spec commit_session(String.t(), String.t(), keyword()) :: {:ok, String.t() | :unchanged} | {:error, term()}
Commits a session into the tree at destination_uri as one document.
Idempotent per (session, destination): re-committing an unchanged session
returns {:ok, :unchanged}. If the destination has since been removed, the
commit rebuilds it rather than reporting it unchanged.
Creates a session and returns its id.
@spec data_transfer_limits() :: %{ max_entries: pos_integer(), max_bytes: pos_integer() }
The bounds one data transfer accepts, for a caller that has to bound its own input.
@spec default_confidence() :: float()
The confidence recorded when a caller supplies none.
Exports store data to a tar archive at path.
Snapshots documents (full content plus caller-supplied abstract/overview),
memory provenance, and sessions into a single .tar or .tar.gz file that
another store can import with import_data/1. With scope: the export is
limited to one subtree (sessions are included only for a full export).
Returns {:ok, %{path:, scope:, documents:, memories:, sessions:, messages:}}.
Why a data export or import was refused, as a sentence an operator can act on.
Discovers files and directories whose URI path contains query.
A literal, case-insensitive substring match over the URI path (excluding
the viking:// scheme). See AgentDb.Application.Documents.find/2 for
scoping, limits, ordering, and errors.
Removes a memory and its provenance. See AgentDb.Application.Memories.forget/1.
Returns every message of a session, in order.
Searches full document content (L2) for a literal substring.
A literal, case-insensitive match that never looks at abstracts or overviews. See AgentDb.Application.Search.grep/2 for scoping, limits, ordering, and errors.
Whether the store and its models are usable, checked individually.
Imports a transfer archive created by export_data/1 into the running store.
Accepts a filesystem path or {:archive, binary}. Validates the whole
archive before writing anything; a refused archive leaves the store exactly
as it was. Import merges by URI and never deletes content outside the
archive. Re-importing an unchanged archive converges without duplication.
@spec import_skills(String.t(), AgentDb.Skills.Source.source()) :: {:ok, %{ skills: [ %{ name: String.t(), status: :imported | :replaced | :failed, files: non_neg_integer(), reason: term() | nil } ] }} | {:error, term()}
Imports Agent Skills into viking://user/{user_id}/skills.
A source is a folder, a tar archive (gzip-compressed or not), or the files of a browser directory selection:
AgentDb.import_skills("alice", {:path, "./my-skills"})
AgentDb.import_skills("alice", {:path, "./my-skills.tar.gz"})
AgentDb.import_skills("alice", {:uploads, [%{path: "alpha/SKILL.md", content: body}]})Each skill is a directory with a SKILL.md at its root, and a source may hold
one skill or a collection of them; a skill's name is its directory's name. See
AgentDb.Skills.Source for what is accepted and what is refused.
A skill stored under that name already is replaced whole -- files the new source does not have go with it -- and a replacement that fails leaves the stored skill as it was, so a valid skill in the same source still lands.
Answers the outcome of every skill, or one error for the source as a whole.
Nothing is written until the whole source has been accepted, so an error leaves
the store as it was. AgentDb.skill_import_error_message/1 turns the error
into a sentence.
@spec index_coverage() :: map()
How much of the store's content each index covers.
Vector index availability and row count against the document count, plus the
documents under each index's own roots. An index that cannot be queried
reports unavailable rather than reporting zero. The vector leg additionally
reports active_dim and needs_backfill when the provider serves them.
@spec layers(uri()) :: %{ abstract: %{ text: content(), source: :stored | :fallback | :unavailable, chars: non_neg_integer() }, overview: %{ text: content(), source: :stored | :fallback | :unavailable, chars: non_neg_integer() }, content: %{ text: content(), source: :stored | :unavailable, chars: non_neg_integer() } } | {:error, term()}
Reads a document's three LLM-facing layers together, each with its source.
Answers %{abstract: layer, overview: layer, content: layer}, where each layer
carries text, source (:stored, :fallback, or :unavailable), and
chars. A :fallback layer was derived from L2 at read time because nothing
was stored; :unavailable means that layer's read failed. A missing URI is an
error, as with any other read.
One call instead of four, so a caller deciding whether to read a whole document pays one round trip rather than one per layer.
Reads the L0/L1/L2 layers for several URIs in one store operation.
Lists the names of a URI's direct children.
Recursive document URIs under scope in deterministic order, paged.
Options: :page (default 1), :per_page (default 50, max 200),
:substring, :include_disabled (default false), :group.
Installed skill roots in URI order, paged. Each entry carries name,
owner, uri, files, enabled, and group_tag.
Options: :page, :per_page, :substring, :owner, :include_disabled,
:group.
Possible conflicts between active memories. See AgentDb.Application.Memories.conflicts/1.
Alias for memory_statistics/0.
Memory statistics: counts per type and status, pending, never surfaced. See AgentDb.Application.Memories.statistics/0.
Alias for memory_statistics/0.
@spec memory_types() :: [String.t()]
The memory types, for callers that need to enumerate the taxonomy.
@spec model_status() :: map()
How the store's models are doing, including a load in progress.
@spec operation_stats() :: map()
Operation, job, and model counts by outcome since this process started.
Reads a document's L1 overview, falling back to the first 280 characters of the body after frontmatter.
Candidates waiting for promotion review. See AgentDb.Application.Memories.pending_candidates/1.
Promotes the waiting candidate at uri to the active memory.
@spec prune_vector_index(pos_integer()) :: :ok | {:error, term()}
Drops a non-active vector dim table (vec_nodes_<dim>).
Refuses the active dim; boot never wipes. An operator action for reclaiming disk after a provider switch, not part of any write path.
@spec queue_detail(pos_integer()) :: map()
How far behind the queue is, and which jobs are failing.
Beyond queue_stats/0's counts: the age of the longest-waiting pending job,
and the failures that spent their retries, each with the classified reason.
@spec queue_stats() :: map()
How much background work is outstanding, by status.
Reads a document's full content (L2).
Reads memories back. See AgentDb.Application.Memories.recall/1.
@spec recent_errors(pos_integer()) :: [map()]
Recent operational failures, newest first, with the operation they happened in and their classified reason. Bounded and in memory only.
Rejects the waiting candidate at uri, leaving no trace of it.
Records a durable fact as a memory. See AgentDb.Application.Memories.remember/3.
Removes the subtree at uri, from every store keyed by URI.
A bounded, read-only view of the BEAM runtime: uptime, applications, supervisors, process counts, ETS tables, memory, and scheduler figures.
Read-only by construction: nothing is started, stopped, or messaged to take one. Never writes, so a snapshot is safe to take at any time.
Searches documents by :keyword, :vector, or :hybrid (default
:keyword).
Options:
:mode-:keyword,:vector, or:hybrid:scope- a URI prefix to limit the search to one subtree:top_k- maximum results (default 10):hybrid_weights-{keyword_weight, vector_weight}or[keyword: kw, vector: vw](default{0.5, 0.5}):fields- which per-result fields to return: any of:uri,:content,:abstract,:overview,:score.:urimust be named, since a result without its URI cannot be fetched. Omitted returns all of them.
Enables or disables the subtree at uri.
Disabled is blocked-from-use: excluded from search and default listings, but still readable and editable. Notifies subscribers so the console reloads without restart.
Assigns the operator group tag for the subtree at uri. Empty clears.
Why a skill import was refused, as a sentence an operator can act on.
@spec skill_import_limits() :: %{max_entries: pos_integer(), max_bytes: pos_integer()}
The bounds one skill import accepts, for a caller that has to bound its own input.
@spec storage_stats() :: map()
What the store holds and how much room it takes on disk.
Document and directory counts, documents per top-level subtree, and the database and write-ahead-log sizes.
@spec stored_layers(uri()) :: {:ok, %{abstract: content() | nil, overview: content() | nil}} | {:error, term()}
Reads a document's stored L0/L1 layers without fallback.
Subscribes the caller to changes beneath a viking:// URI.
@spec tree(uri(), pos_integer()) :: {:ok, map()} | {:error, term()}
A depth-limited projection of the tree at uri (default depth 2).
Unsubscribes the caller from a viking:// URI scope.
Writes a document at uri, creating missing parent directories.
Persists and acknowledges before any embedding or summarization completes, so
a read returns content immediately and searchability arrives progressively.
With async: false the write waits instead, and reports whether the work
completed, failed, or is still outstanding.
Options:
:async- overrideasync_writes:sync_timeout_ms- how long a synchronous write waits (default 30_000):abstract/:overview- the caller's own L0 / L1 layers