Skip to content

Latest commit

 

History

History
63 lines (53 loc) · 12.6 KB

AGENTS.md

File metadata and controls

63 lines (53 loc) · 12.6 KB

LocalAI Agent Instructions

This file is the entry point for AI coding assistants (Claude Code, Cursor, Copilot, Codex, Aider, etc.) working on LocalAI. It is an index to detailed topic guides in the .agents/ directory. Read the relevant file(s) for the task at hand — you don't need to load all of them.

Human contributors: see CONTRIBUTING.md for the development workflow.

Policy for AI-Assisted Contributions

LocalAI follows the Linux kernel project's guidelines for AI coding assistants. Before submitting AI-assisted code, read .agents/ai-coding-assistants.md. Key rules:

  • No Signed-off-by from AI. Only the human submitter may sign off on the Developer Certificate of Origin. One exception: automation a maintainer operates signs off with that maintainer's identity, since no other human submitter exists to certify it. See .agents/ai-coding-assistants.md.
  • No Co-Authored-By: <AI> trailers. The human contributor owns the change.
  • Use an Assisted-by: trailer to attribute AI involvement. Format: Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2].
  • The human submitter is responsible for reviewing, testing, and understanding every line of generated code.

Topics

File When to read
.agents/ai-coding-assistants.md Policy for AI-assisted contributions — licensing, DCO, attribution
.agents/building-and-testing.md Building the project, running tests, Docker builds for specific platforms
.agents/ci-caching.md CI build cache layout (registry-backed BuildKit cache on quay.io/go-skynet/ci-cache, per-arch keys), DEPS_REFRESH weekly cache-buster for unpinned Python deps, prebuilt base-grpc-* images for llama.cpp variants, per-arch native + manifest-merge pattern, setup-build-disk /mnt relocation, path filter on master push, manual eviction
.agents/ci-bot-pull-requests.md Scheduled bot workflows that open PRs (bumps, swagger, gallery agent): GitHub App token setup (UPDATE_BOT_APP_ID, UPDATE_BOT_PRIVATE_KEY), permissions, branch conventions
.agents/adding-backends.md Adding a new backend (Python, Go, or C++) — full step-by-step checklist, including importer integration (the /import-model dropdown is server-driven from GET /backends/known)
.agents/coding-style.md Code style, editorconfig, logging, documentation conventions, React UI styling and the UI kit controls rules
.agents/llama-cpp-backend.md Working on the llama.cpp backend — architecture, updating, tool call parsing
.agents/vllm-backend.md Working on the vLLM / vLLM-omni backends — native parsers, ChatDelta, CPU build, libnuma packaging, backend hooks
.agents/sglang-backend.md Working on the SGLang backend — engine_args validation against ServerArgs, speculative-decoding (EAGLE/EAGLE3/DFLASH/MTP) recipes, parser handling
.agents/ds4-backend.md Working on the ds4 backend - DSML state machine, thinking modes, KV cache, Metal+CUDA matrix
.agents/testing-mcp-apps.md Testing MCP Apps (interactive tool UIs) in the React UI
.agents/api-endpoints-and-auth.md Adding API endpoints, auth middleware, feature permissions, user access control
.agents/debugging-backends.md Debugging runtime backend failures, dependency conflicts, rebuilding backends
.agents/adding-gallery-models.md Adding GGUF models from HuggingFace to the model gallery
.agents/localai-assistant-mcp.md LocalAI Assistant chat modality — adding admin tools to the in-process MCP server, editing skill prompts, keeping REST + MCP + skills in sync
.agents/sandbox-profiles-and-media.md Sandbox profiles of the Sandbox add-on (manifest, guest image targets, digests, CI, delivery), media client tools and arguments, built-in skills with a sandbox: hint, and the checks to run before a push
.agents/backend-signing.md Backend OCI image signing (keyless cosign + sigstore-go) — producer-side CI setup, consumer-side gallery verification: block, strict mode (LOCALAI_REQUIRE_BACKEND_INTEGRITY), revocation via not_before
.agents/preparing-a-release.md Cutting a release: PR labels, RELEASE_NOTES_vX.Y.Z.md, the blog post under website/content/blog/, and the demo clips under website/static/media/
.agents/distributed-state.md Features that keep runtime state — how they must behave with several frontends (syncstate, advisory-lock leaders, fakebus tests)
.agents/distributed-seams.md Distributed mode transports: the fan-out, queue, control verb, agent RPC and dial seams, subject rules, the no-route contract, conformance suites, open items for a second carrier
.impeccable.md Design context for UI/UX work — users, brand personality, aesthetic direction, and design principles

Quick Reference

  • Coverage gates: Never lower a coverage baseline or widen a gate's tolerance to turn a red gate green — the coverage ratchet only moves up. If a change drops coverage, add tests to raise it (e.g. render-smoke specs). See .agents/building-and-testing.md.

  • Logging: Use github.com/mudler/xlog (same API as slog)

  • Go style: Prefer any over interface{}

  • Comments: Explain why, not what

  • Docs (docs-with-code rule): When you change user-facing behavior (API endpoints, CLI flags, config keys, or features), update the corresponding page under docs/content/ in the SAME change, not as a follow-up. A user-facing change without a matching docs update is incomplete. See also the documentation conventions in .agents/coding-style.md.

  • New API endpoints: LocalAI advertises its capability surface in several independent places — swagger @Tags, /api/instructions registry, auth RouteFeatureRegistry, React UI capabilities.js, docs. Read .agents/api-endpoints-and-auth.md and follow its checklist — missing any surface means clients, admins, and the UI won't know the endpoint exists.

  • Admin endpoints → MCP tool: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in core/services/localaitools/. The LocalAI Assistant chat modality and the standalone local-ai mcp-server consume that package; drift between REST and MCP is a real risk. Read .agents/localai-assistant-mcp.md — the TestToolHTTPRouteMappingComplete test fails until you wire the new tool and update the route map.

  • API changes → LocalAI client: every change to the API that a user could use through an agent (a new inference endpoint, or a changed or removed parameter or response of an existing one) MUST be reflected in the same change in the LocalAI client in core/services/localaitools/: the LocalAIClient interface, both implementations (inproc and httpapi), the tool schema and its DTOs, the ClientTools route and permission table, the built-in localai skill (core/services/studio/builtinskills/tree/localai/SKILL.md) and the docs. A route that must not be a client tool needs an entry with a reason in clientToolExemptions (clientcoverage_test.go). Read .agents/localai-assistant-mcp.md — the client coverage test fails until every inference route has a tool or an exemption.

  • Model tool surface: every tool of an always-sent group costs tokens on every turn of every chat, so a new model tool must justify its place. Merge operations into one action-parameter tool only when they share state in one domain (one flat schema, no oneOf/anyOf, per-action checks in the handler); media and other unrelated tools stay single-purpose. A group that is large must be deferrable behind the tool search (tool_search, tool_describe, tool_call, setting chats_tool_search). The client-group budget test (clientBudgetBytes in core/services/localaitools/clienttools_merge_test.go) is raised only with a reason in the commit. See .agents/localai-assistant-mcp.md.

  • Sandbox profiles: one table (addons/sandbox/internal/profiles/profiles.go) is the source of the manifest, the model note, the image check and the docs table; never hand-write a program list. Each profile needs a Dockerfile target, a check-tools.sh check for every tool, a digest variable in package.sh, and a step in sandbox-images.yml. A missing image falls back to base with a notice; the policy is open by default. See .agents/sandbox-profiles-and-media.md.

  • Media tools and skills: a tool argument exists only if the HTTP API takes it. Files are references resolved by the chat file store, results are written to the workspace and returned as references, and params is an allow-list of validated scalars, never free-form. A skill with a sandbox: hint only hints; run every command in it in the profile's image first. See .agents/sandbox-profiles-and-media.md.

  • Releases ship with a post and clips: a release is not done at the tag. It needs labelled PRs, RELEASE_NOTES_vX.Y.Z.md, a blog post under website/content/blog/, and a short demo clip in website/static/media/ for each notable feature. See .agents/preparing-a-release.md.

  • Build: Inspect Makefile and .github/workflows/ — ask the user before running long builds

  • Backend OS coverage: a new backend must target every OS it can build for, not just Linux. .github/backend-matrix.yml has two matrices — include: (Linux) and includeDarwin: (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the includeDarwin entry + the metal: entries of the backend library (gallery/backends/index.yaml, added with backendcatalog --add; backend/index.yaml is generated, never edited), or say in the PR why an OS is unsupported. See the darwin checklist in .agents/adding-backends.md.

  • Gallery variant ranking: a gallery entry can declare variants (alternative builds of the same weights), and LocalAI ranks the ones a host can run by engine preference first, size second. A new backend that should be preferred on some hardware must be listed in engineNamePreferenceRules in pkg/system/capabilities.go; the sibling backendBuildTagPreferenceRules speaks build tags rather than engine names, and using the wrong table matches nothing without erroring. See .agents/adding-backends.md.

  • UI: The active UI is the React app in core/http/react-ui/. The only Alpine.js/HTML page left is the standalone Explorer server page (core/http/views/explorer.html, assets in core/http/static/), pending a move to the React UI — all new UI work goes in the React UI

  • UI controls: use the kit in core/http/react-ui/src/components/ui/ for buttons, switches, pills, callouts, panes and sheets; never write a new one. Most of it lives in the React-only package core/http/react-ui/packages/ui-kit/ (no router, no i18n, no import from src/). Run npm run lint:ui-kit before finishing. See "React UI controls" in .agents/coding-style.md.

  • Distributed-aware state: any feature that keeps runtime state (maps, caches, pins, schedulers, background loops) must choose shared (syncstate), single-runner (advisorylock), stateless, or documented per-instance behaviour for multi-frontend clusters. See .agents/distributed-state.md.

  • Add-on settings authority: before adding settings or host-rendered controls, follow the existing checklist in .agents/distributed-state.md and the owner/two-frontend matrix in addon testing. UI placement does not establish authority. Declare owner-wide, cluster-wide, or node-local scope; owner-wide settings must use authenticated owner dispatch in both combined and distributed modes, not frontend-local /api/settings.