AgentDb.ML.ModelManager.Backend behaviour (agent_db v0.1.0)

Runtime that loads embedding and summarization models and runs inference.

Backend.Exla is the current Bumblebee + EXLA path. Backend.Emlx is the Apple MLX path (EMLX + EMLXAxon). ModelManager selects one at startup and falls back to EXLA when EMLX is unavailable.

Summary

Types

config()

model_ref()

Callbacks

embed(model_ref, list)

load_embedding(config)

load_llm(config)

summarize(model_ref, t, keyword)

Functions

load(config, role)

Loads a role's model through the configured backend, falling back to EXLA when the EMLX backend fails.

module_for(arg1)

Backend module for a resolved backend name.

resolve(arg1)

Resolves :auto to :emlx on Apple Silicon macOS, else :exla.

Types

config()

@type config() :: map()

model_ref()

@type model_ref() :: map()

Callbacks

embed(model_ref, list)

@callback embed(model_ref(), [String.t()]) :: {:ok, [Nx.Tensor.t()]} | {:error, term()}

load_embedding(config)

@callback load_embedding(config()) :: {:ok, model_ref()} | {:error, term()}

load_llm(config)

@callback load_llm(config()) :: {:ok, model_ref()} | {:error, term()}

summarize(model_ref, t, keyword)

@callback summarize(model_ref(), String.t(), keyword()) ::
  {:ok, String.t()} | {:error, term()}

Functions

load(config, role)

@spec load(config(), :embedding | :llm) :: {:ok, model_ref()} | {:error, term()}

Loads a role's model through the configured backend, falling back to EXLA when the EMLX backend fails.

The fallback lives here rather than in the manager because backend selection is this module's decision: the manager asks for a model to be loaded, not how the backend that loads it is chosen. A failure is reported, never raised, so the caller keeps its load-state machine intact.

module_for(arg1)

@spec module_for(:exla | :emlx) :: module()

Backend module for a resolved backend name.

resolve(arg1)

@spec resolve(:auto | :exla | :emlx) :: :exla | :emlx

Resolves :auto to :emlx on Apple Silicon macOS, else :exla.