best 9 tools for running llms locally

gui, local openai-compatible server and the cortex inference engine underneath are all apache 2.0 in the same repository. of the nine tools here that is a claim only jan, llama.cpp, gpt4all and localai can make, and jan is the only one of those four that is also a finished desktop application.

practically it does what you want: cuda, metal and vulkan backends, gguf models, an openai-compatible endpoint so existing code points at it unchanged, and installers for macos, windows and linux.

the honest limitation is maturity. the model library is smaller than ollama's, the interface is less polished than lm studio's, and it has less momentum than either. it gained ground through 2026 largely because ollama's licensing drift pushed people to look, which is a thin reason to lead a category and a real one to exist.

mit, no conditions, and the widest hardware support here: cuda, metal, rocm/hip, vulkan, sycl and cpu, on desktop, mobile and embedded builds. it is also first to support new model architectures and quantisation formats, usually within days, which is why every wrapper on this list depends on it.

it ships a cli, a server with an openai-compatible api, and a deliberately minimal built-in web ui. that is enough to run anything, and it is not an experience most people want as their daily driver.

the reason to know this entry exists even if you never install it directly: when you compare 'ollama versus lm studio versus koboldcpp' on speed, you are mostly comparing the same c++ code with different paint. the differences that matter between those tools are licence, interface and model management, not inference.

`ollama pull` and `ollama run` are why local models became approachable to people who don't build software, and the model library remains the largest and best-curated here. cuda, rocm, metal and partial vulkan, with an openai-compatible server that most local tooling assumes by default.

the licensing has drifted. the cli, engine and server are mit and remain so. the desktop gui merged into the main distribution around november 2025 without an open licence, so the interface most casual users interact with is closed. the company has also added hosted cloud models on a usage basis, which is a reasonable business and a departure from the project's origins.

none of that makes it a bad tool — it is still the fastest path from nothing to a running model. it makes 'ollama is the open one' a claim that needs the qualifier this page exists to supply.

the product case is strong. it bundles both llama.cpp and apple's mlx runtime, so on apple silicon you get native mlx rather than a port, and it configures hardware-aware settings automatically instead of making you guess at layer offloads. gguf and mlx formats, cuda, metal, vulkan and rocm, and an openai-compatible server.

the licence is a flat contradiction of its reputation. element labs' terms grant free use 'solely for your personal and/or internal business purposes', prohibit resale, redistribution and offering it as a service to third parties, and forbid integrating anything that would oblige the app's own code to be opened. it is closed software that wraps open engines.

that matters unevenly. for an individual running models on a laptop it changes nothing. for anyone building a product, embedding it in a workflow they sell, or needing to audit what runs on their machine, it is the whole answer. a paid pay-as-you-go tier for hosted third-party models has also been added, moving it beyond purely local inference.

localai exposes an openai-compatible api spanning chat, image generation and audio, so a codebase written against openai can be pointed at it wholesale rather than per-model. gguf plus onnx and diffusers-family formats, cuda, rocm, vulkan and limited metal.

it is mit with no conditions, and docker-first, which tells you the intended audience: this is something you deploy, not something you install and click.

that framing is also the limitation. there is no native desktop app, setup is meaningfully more work than ollama or jan, and a casual user wanting to try a model on their laptop will find it heavier than the job requires.

localdocs is the reason to pick it: point the app at a folder and ask questions about its contents, entirely locally, with no vector database to stand up and no configuration. rivals either omit this or treat it as an advanced feature.

the licence is mit with no conditions, the gui is qt-based and available on macos, windows and linux, and there are python bindings and a local server alongside cuda, vulkan, metal and cpu backends.

our reservation is about momentum rather than terms. we could not confirm nomic's current release cadence from a primary source, and the feature and model-support pace visibly trails ollama, lm studio and jan. for a tool you rely on, that's worth checking before committing.

it ships as a single dependency-free executable bundling a server and the kobold lite web ui. no installer, no runtime, no package manager — download and run. for a category where setup friction is the main barrier, that is a real design achievement.

the feature set leans towards long-form fiction and roleplay: memory, world info, instruct and chat modes, all built around keeping a long narrative coherent rather than answering one-off questions. cuda, rocm, vulkan and cpu via its own llama.cpp fork.

the licence is agpl-3.0, and the direction of its restriction is worth understanding. it is unambiguously open source — but if you modify it and run it as a network service, you must publish your modified source. for an individual that obligation never triggers. for a company building on it quietly, it is the strictest term on this page.

as a product it is excellent: the most full-featured browser front-end for local models, actively developed, and it pairs with ollama, llama.cpp, localai or any openai-compatible backend. for most people running models locally with a nice interface, this is the interface.

the licence is a modified bsd-3-clause with an added branding clause. you may use it commercially only if open webui's branding stays intact — unless your deployment stays under 50 users in any rolling 30-day period, you obtain written permission, or you sign a paid enterprise licence. removing the branding without one is defined in the licence as a material breach.

that is a source-available licence with a genuine commercial restriction, not an osi-approved open-source licence, and it is the one most likely to catch a company out — because the code is on github, modifiable, and described as open source almost everywhere including by people deploying it at work.

the distinctive feature is genuine: a split-chat interface that runs the same prompt against multiple models side by side, plus local rag over your own documents, in one polished window. nothing else here does the comparison view as cleanly.

the licensing is a blank. we found no licence file, no terms naming a licence, and no public source repository — only a pricing page from cloudstack, llc listing a free studio tier and an aurum add-on at $149 a year or $349 for a lifetime seat. older community posts from 2024 describe it as open source, and there is no current evidence supporting that.

we also could not confirm its gpu backends from msty's own documentation; third-party sources describe a llama.cpp-based engine, which is plausible and unverified. for a tool asking to run on your machine and read your documents, publishing neither a licence nor an architecture is a lot to ask on trust.