Atomic Chat runs an OpenAI-compatible server at http://localhost:1337/v1 — a drop-in replacement for the OpenAI SDK. Load a model in the app, then point any client at it:
from openai import OpenAI
# Atomic Chat is OpenAI API-compatible — only the base_url changes.
client = OpenAI(base_url="http://localhost:1337/v1", api_key="not-needed")
resp = client.chat.completions.create(
model="<model-id-loaded-in-atomic-chat>",
messages=[{"role": "user", "content": "Say hello in one word"}],
)
print(resp.choices[0].message.content)
Bound to 127.0.0.1 by default; set host: 0.0.0.0 to expose it on your LAN. Works with any agent, CLI, or IDE plugin that speaks the OpenAI API — see Launch With below.
✨ Features
Local models
Run open-weight LLMs locally from HuggingFace — Llama, Gemma, Qwen, Mistral, Phi, and others
Multi-Token Prediction (MTP) speculative decoding — 30–70% throughput boost on supported models, up to 3× on Gemma 4
DFlash block-diffusion decoding — up to 6× faster on Qwen 3.6, Gemma 4, Kimi K2.5
Flash Attention toggle (on / off / auto)
Automatic reasoning-context tracking for chain-of-thought models
Auto context-window expansion with overflow notifications
EAGLE-3 speculative decoding for Gemma 4 on Apple Silicon (MLX)
MTP on MLX for Qwen 3.5 / 3.6 and DeepSeek V4
TurboQuant KV cache (turbo3 / turbo4) on llama.cpp — now on Windows & Linux too, not just macOS: up to ~4.3× smaller KV cache footprint, CPU and GPU (CUDA / Vulkan)
TurboQuant KV cache on MLX-VLM — smaller memory footprint via RHT-correct fast paths
Bring your own key, switch model per chat, mix local and cloud freely
Tools & integrations
One-click agent launch — launch agents like Atomic Agent, Claude Code, Codex CLI, Cline, OpenCode, Droid, Goose, OpenHands, Copilot CLI, Kilo Code, Zed and ZCode in one click from the Integrations tab
Artifacts — live preview panel for HTML/CSS/JS code with copy, download and print
Connect multiple MCP servers — bring your own tools, file access, web search
Custom assistants with per-assistant system prompts
Projects with conversation tree view in the sidebar
Local API
OpenAI-compatible server at http://localhost:1337/v1 — drop-in replacement for the OpenAI SDK
Works with any agent, CLI, or IDE plugin that speaks the OpenAI API
Bound to 127.0.0.1 by default; set host: 0.0.0.0 to expose on LAN
Privacy
Everything runs locally when you want it to — local server is loopback-only by default
Your conversations and keys stay on your machine
⚙️ Inference Engines
Three engines under the hood, all exposed through one OpenAI-compatible API at http://localhost:1337/v1:
atomic-llama-cpp-turboquant — our llama.cpp fork with TurboQuant KV-cache optimizations (turbo3 / turbo4) for faster, lower-memory quantized inference. Now a selectable second provider ("Atomic Llama.cpp Turboquant") on all three desktops — macOS, Windows, and Linux — CPU and GPU (CUDA / Vulkan).
Upstream llama.cpp — official ggml-org build, the default engine on Windows and Linux for the widest hardware coverage and MTP support.
MLX-VLM — Apple Silicon-native engine for vision-language models, running on the Neural Engine and unified memory. Faster than llama.cpp on M-series chips for supported models.
Speculative-decoding features available across backends:
MTP (Multi-Token Prediction) — a draft model predicts ahead, the full model verifies in one pass. Available on macOS and Windows.
DFlash — block-diffusion speculative decoding for Qwen 3.6, Gemma 4, Kimi K2.5 and others. Apple Silicon only; can't be enabled together with MTP.
Flash Attention — Settings → on / off / auto.
Tools talking to http://localhost:1337/v1 don't need to know which backend is running underneath — switch engines without reconfiguring clients.
The engines are driven by atomic-chat-core — the core of Atomic Chat, in its own repository. It doesn't do inference itself; it manages the engines: downloads and launches backends, manages models, routes requests between local engines and cloud providers, and serves the OpenAI-compatible API on :1337. The desktop app ships a pinned core release; the core also runs on its own as a CLI (atomic-chat-core serve …) or as a library.
🚀 Launch With
Atomic Chat runs an OpenAI-compatible server at http://localhost:1337/v1, so any agent, CLI, IDE plugin, or app that speaks the OpenAI API can run on top of your local models — no extra glue needed. Just point its base URL at Atomic Chat and you're done.
A few projects already ship first-class support with their own setup docs:
If prompted about FUSE on first launch: sudo apt install fuse libfuse2 (Debian/Ubuntu) or sudo dnf install fuse fuse-libs (Fedora). GPU acceleration (Vulkan) is auto-detected on first launch; only GGUF models run on Linux.
Atomic Chat is built by a small core team and 140+ contributors — including everyone who shaped the project from its earliest days. Pull requests are welcome in both repositories:
Atomic-Chat (this repo) — the desktop and mobile app: UI, chats, assistants, projects, MCP, integrations, settings. See CONTRIBUTING.md to get started.
atomic-chat-core — the core that manages the engines (llama.cpp, MLX, Foundation Models): backend and model management, cloud providers, the router and the OpenAI-compatible server. See its AGENTS.md to get started.
Not sure where a change belongs? Open an issue in either repo and we'll point you the right way.
Atomic Chat began as a fork of Jan by Menlo Research — an excellent open-source local-AI app. We're grateful to the Jan team and its contributors for the foundation they built. Atomic Chat has since grown its own direction, engines, and roadmap, but we tip our hat to where it started. 🙏
