TurboLLM

TurboLLM lets you run open‑source large language models directly on your own computer, without needing a cloud account or an internet connection. The service provides a browser‑based UI and an API that mimics OpenAI and Anthropic endpoints, so existing tools that expect those APIs can point at a local model instead of a remote service.

The platform supports any GGUF model that works with a llama.cpp‑compatible engine. You can choose from eight engine types, including llama.cpp, ik_llama.cpp, TurboQuant, KoboldCpp, llamafile, MLX, vLLM and SGLang, as well as community forks you may have compiled yourself. After installing Node 22+ (or using the provided one‑command installer), you launch TurboLLM with `npx turbollm`. The command detects your GPU, runs benchmarks, and shows a VRAM‑fit verdict and measured tokens‑per‑second for each model before you load it. This eliminates the need to guess flags or rebuild binaries.

TurboLLM's UI runs in any browser on the same LAN, so phones, tablets or laptops can access a model hosted on a desktop or server. The API layer listens on a single port and accepts OpenAI‑compatible or Anthropic‑compatible requests, allowing tools like Claude Code to operate locally with zero token cost. A built‑in coding agent can read, edit, and execute commands in a real repository, using the same SDK that powers the UI. You can also define agents with custom system prompts, skills files, or tool access, and schedule routine tasks that run chats or code jobs on a timer, with manual approval required before any destructive action.

The system is designed for developers who want full control over model execution, privacy, and hardware utilization. It works on Windows, macOS, Linux and Android as a standalone app; iOS is not supported. The package size is about 7 MB, and the tool does not require any external accounts or backend services.

Key features

• Runs any GGUF model on any llama.cpp‑compatible engine, including community forks
• Auto‑tunes engine defaults and displays live tokens‑per‑second measured on your hardware
• Provides OpenAI and Anthropic‑compatible endpoints for smooth integration with existing tools
• Browser‑based UI accessible from any LAN device, with no Electron dependency
• Local coding agent that can edit files, run commands and report diffs in a real repository
• Routine scheduler that can execute chats or code tasks on a configurable timer