@spec backend_spec(:cpu | :cuda | :rocm | :emlx | {module(), keyword()}) ::
  :none | {module(), keyword()}

Translates an AgentDb backend name into Bumblebee's :backend value.

Returns :none to mean "do not pass a :backend option", which is how Bumblebee is told to use the default backend.

@spec build_serving(role(), model_info(), term(), keyword()) ::
  {:ok, Nx.Serving.t()} | {:error, term()}

Builds the inference serving for a loaded model.

Bumblebee builds an Nx.Serving from the model info and the tokenizer, and runs inference through that serving rather than through a tokenize/generate pair on the serving module. Loading therefore produces the serving itself, once, and run/2 is the only inference entry point.

opts carries the role's compile options (:batch_size, :sequence_length) and, for the LLM, the generation settings to configure (:max_new_tokens, :temperature, :top_p).

@spec load_model(repository(), keyword(), module()) ::
  {:ok, term()} | {:error, term()}

Loads the model for a repository onto the backend named by opts[:backend].

opts[:backend] is AgentDb's own vocabulary — :cpu, :cuda or :rocm, the values AgentDb.Config.exla_backend/0 returns. Bumblebee wants a backend module or {module, opts}, so it is translated here rather than passed through: an untranslated :cpu is read as a module named :cpu and raises.

:cpu passes no :backend at all, which is Nx's default CPU backend. :cuda/:rocm require an EXLA client configured under config :exla, clients; without one, EXLA.Client.fetch!/1 raises and the caller sees a load failure rather than a silent fall back to CPU.

@spec load_tokenizer(repository(), module()) :: {:ok, term()} | {:error, term()}

Loads the tokenizer for a repository.

@spec run(Nx.Serving.t(), term()) :: {:ok, term()} | {:error, term()}

Runs a built serving over input and returns its result.

Nx.Serving.run/2 answers with the serving's result directly rather than wrapping it, so an error is recognised by its own shape and anything else is a successful result. Inference goes through this one function so the Bumblebee API is reached from a single place, and a loader without Bumblebee installed can answer build_serving/4 and run/2 instead.