Best AI Agent Skills for Local and Self-Hosted AI (2026)

Running an agent against a local model stopped being a hack in January 2026, when Ollama v0.14.0 added an Anthropic-compatible /v1/messages endpoint. Before that, connecting Claude Code to a self-hosted model meant a translation proxy and a lot of broken tool calls. Now it's two environment variables.

That changed which skills matter. The runtime question is mostly solved. What's left is everything the agent reaches for besides the model: web search, memory, documentation lookup, the skill registry itself. Each one of those is a place where "local" quietly turns back into "someone else's server."

Star counts and update dates below come from our skill database, refreshed August 1, 2026.

The stack

Start with the connection, not a skill

Before installing anything, get the base case working. With Ollama running:

ANTHROPIC_AUTH_TOKEN=ollama ANTHROPIC_BASE_URL=http://localhost:11434 claude --model qwen3-coder

The auth token is required and ignored. Ollama also ships ollama launch claude as a shortcut.

Read the unsupported list before you commit, because Ollama publishes it plainly and it explains most of the weirdness people hit: no token-counting endpoint, no tool forcing via tool_choice, no prompt caching, no Batches API, no citations or PDF documents, and images must be base64 rather than URLs. Token counts are approximations from the underlying model's tokenizer.

The prompt-caching gap is the one that will surprise you. Long agent sessions on a hosted model lean on cached context; without it, every turn re-processes the full conversation, and on consumer hardware you feel that immediately.

Model servers

LocalAI is the general answer, one engine for text, vision, voice, image, and video models, running on CPU if that's what you have. 48,066 stars, MIT, active as of July 31, 2026.

claude mcp add localai -- npx -y mudler/LocalAI

The "no GPU required" claim is true and worth reading carefully: it will run, and a 14B model on CPU is slow enough that you'll use it for batch work rather than interactive coding.

Lemonade goes the other direction and targets accelerated hardware, serving models optimized for consumer GPUs and NPUs.

claude mcp add lemonade -- npx -y lemonade-sdk/lemonade

If your laptop shipped with an NPU that has done nothing since you bought it, this is what finally uses it. Apache-2.0, 5,181 stars, updated July 31, 2026.

On Apple Silicon, vllm-mlx is the interesting one. It serves Llama, Qwen-VL, and LLaVA through the MLX framework with continuous batching, and it exposes both OpenAI- and Anthropic-compatible APIs.

claude skill add waybarrios/vllm-mlx

Anthropic compatibility from a second project matters more than it sounds. It means the Ollama endpoint isn't a single point of failure for anyone building on this pattern. Community tier, unreviewed, 1,473 stars. Read the source before you point production at it.

Keeping the rest of the loop local

Here's where most "private AI" setups leak. The model runs locally, then the agent calls a hosted search API, stores memory in a managed vector database, and pulls documentation over the network. Three network calls with your context attached.

AnythingLLM is the most-starred item on this page at 64,163, and it handles document ingestion and retrieval on-device.

claude mcp add anything-llm -- npx -y Mintplex-Labs/anything-llm

MIT, updated July 31, 2026. For "answer questions about this folder of PDFs without any of it leaving the machine," this is the default choice and has been for a while.

Wax is smaller and more specific: a single-file memory layer with sub-millisecond retrieval, Metal-optimized for Apple Silicon, no server and no API.

claude mcp add wax -- npx -y christopherkarani/Wax

776 stars, Apache-2.0, community tier. The single-file design is the point: nothing to run, nothing listening on a port, and the whole memory store is one artifact you can back up or delete.

Two projects in this category look useful and aren't. ollama-mcp-bridge hasn't been updated since April 2025, and web-search-mcp since August 2025. Both sit in a part of the stack that has been rewritten twice in the past year. Their stars are real; their maintenance isn't.

For a current local-model client, MCP Client for Ollama is a terminal UI with agent mode, multi-server support, and model switching, updated July 27, 2026.

claude mcp add mcp-client-for-ollama -- npx -y jonigl/mcp-client-for-ollama

Sandboxing, gateways, and registries

Local doesn't mean safe. A model running on your laptop with shell access is a model running on your laptop with shell access.

Moltis is a personal agent server written in Rust that ships as one binary with sandboxed execution, its own memory, and multi-provider model routing.

claude mcp add moltis -- npx -y moltis-org/moltis

MIT, 2,800 stars, updated July 31, 2026. The sandbox is the reason to pick it over a looser setup.

Osaurus is the macOS counterpart: a native harness written in Swift that runs any model with persistent memory, autonomous execution, and cryptographic identity for the agent.

claude mcp add osaurus -- npx -y osaurus-ai/osaurus

MIT, 7,446 stars, updated July 31, 2026. The cryptographic identity piece is unusual and useful if you ever need to prove which agent did what.

MCPJungle puts a self-hosted gateway in front of your MCP servers, so agents talk to one endpoint and you control what's behind it.

claude mcp add mcpjungle -- npx -y mcpjungle/MCPJungle

MPL-2.0, community tier, and its last update was May 20, 2026, so not stale but not fast-moving either.

SkillHub is the enterprise piece: a private skill registry with versioning, RBAC, audit logs, and Docker or Kubernetes deployment.

claude skill add iflytek/skillhub

If you self-host models for compliance reasons, installing skills straight from public GitHub undoes a good part of that. Our compliance and audit readiness guide covers why that gap shows up in audits.

Mobile and edge

Google AI Edge Gallery demonstrates on-device generative models on phones. 24,316 stars, Apache-2.0, updated July 31, 2026.

claude skill add google-ai-edge/gallery

It's a showcase rather than infrastructure. Useful if you're deciding whether an on-device model is viable for something you're shipping, less useful as part of a daily setup.

What to install

For one machine, working offline:

• claude mcp add localai -- npx -y mudler/LocalAI
• claude mcp add anything-llm -- npx -y Mintplex-Labs/anything-llm

Plus Ollama pointed at Claude Code through the environment variables above. On Apple Silicon, swap LocalAI for vllm-mlx and add Wax for memory.

For a team with a policy reason to self-host, add MCPJungle in front of your MCP servers and SkillHub as the registry. Those two are what turn "we run models locally" into something you can describe to a security reviewer.

The trade you're making

Local models in mid-2026 are good. Qwen-class coding models handle single-file work and refactors well. They are not equal to frontier hosted models on long multi-file reasoning, and the missing prompt caching makes extended agent sessions expensive in wall-clock time even when they cost nothing in dollars.

The setups that work best route by task: local for anything touching sensitive code, offline work, and high-volume drafting; hosted for the hard reasoning passes. That's less satisfying than full independence, and it's what people who use this daily settle into.

Related: self-hosted open source AI developer tools, running LLMs privately, and the developer tools list.

FAQ

Can Claude Code run fully offline against a local model? Yes. Set ANTHROPIC_BASE_URL to your Ollama instance and ANTHROPIC_AUTH_TOKEN to any value, and the agent loop runs against a local model. Skills that make network calls, such as web search and hosted docs, will still reach out unless you replace them with local equivalents.

What's the minimum hardware? LocalAI runs on CPU with no GPU, so the floor is low, but a 7B model on CPU responds in tens of seconds and that makes interactive coding painful. A GPU with 16GB, or Apple Silicon with 32GB unified memory, is where it starts feeling usable for agent work.

Which local model should I pair with these? Qwen-family coding models are the common choice at every size tier and are what Ollama uses in its own Claude Code example. Match the parameter count to your memory, not to benchmark tables.

Does self-hosting satisfy a data-residency requirement? For the model inference, yes. Auditors will also ask about the rest of the loop: where skills come from, where retrieval indexes live, whether MCP servers call out. That's the case for a gateway and a private registry rather than direct installs.

Are community-tier skills here safe to install? Community tier means we haven't reviewed them, not that they're dangerous. vllm-mlx, Wax, MCPJungle, and MCP Client for Ollama are all in that bucket. Read the source, and consider scanning them first with the tools in our security audit skills guide.

Why not just use a cheaper hosted model instead? If cost is the only driver, a small hosted model usually wins on effort. Self-hosting earns its setup time when the reason is privacy, offline capability, or a contractual restriction on where data goes.