verifier.org

best 8 coding llms

the models themselves rather than the assistants built on them, ranked on what the licence file permits and what fits on hardware you own.

last reviewed 29 jul 2026 · 8 tools tested ·list curated by Onur Ozcanxin

the short version
best overallQwen3-Coderrepository-scale agentic work where the context window and the licence both need to be unrestricted.90/100runner-upDeepSeek Coderfrontier-tier coding where the score matters more than running it yourself.88/100

'open' means four different things in this category and only one of them is what you'd assume. mistral's codestral ships under the mistral ai non-production licence, which bars commercial and production use outright — not a revenue cap, a flat prohibition unless you negotiate a paid licence. its own launch page invites you to download it from hugging face without leading with that, and the current 25.08 iteration appears to be api-only, so the open-weight framing may now apply only to the legacy 22b checkpoint.

the second shape is a revenue cliff. devstral 2's 123b model carries a modified mit licence that revokes all rights if your company's global consolidated monthly revenue exceeded $20 million in the preceding month — and mistral, to its credit, says so plainly rather than burying it, while keeping the smaller 24b devstral on clean apache 2.0. same company, same family, two entirely different deals.

the third is openrail. starcoder2 permits commercial use and attaches binding conditions you must carry into your own terms of service: no malware, no verifiably false or harmful disinformation, no impersonation for deception, no undisclosed automated decisions affecting people's legal rights, and public-facing generated content must intelligibly disclose that it is machine-generated. that is commercial permission with strings that follow you downstream, and the model card doesn't spell them out — they live in the attachment.

the fourth is genuinely clean, and it's the good news. qwen3-coder is plain apache 2.0, confirmed in the licence text rather than the badge. seed-coder is plain mit. granite is apache 2.0. and deepseek moved the other way from everyone else: the original coder generation paired mit code with a restrictive custom weights licence, while the current v4 weights are plain mit — a loosening, which almost nothing else in this field has done.

advertisement
  1. 1

    Qwen3-Coder

    apache 2.0 confirmed in the licence text, a million tokens of context, and now a version small enough to actually run.

    90/100

    verdictthe cleanest licence and the longest context in the category, with a february variant that finally makes it deployable outside a datacentre.

    best for
    repository-scale agentic work where the context window and the licence both need to be unrestricted.
    price
    free (apache 2.0)
    pricing note
    480b total with 35b active on the flagship; qwen3-coder-next arrived february 2026 at 80b total and 3b active for local and agent deployment
    free tier
    yes
    licence
    apache 2.0
    params
    480b/35b active; next 80b/3b
    context
    262k, 1m via yarn
    commercial use
    unrestricted
    runs on
    ~240gb, or ~40gb for next

    apache 2.0, and we read the licence file rather than trusting the model-card badge — no revenue cap, no use restrictions, no naming mandate. after codestral's non-production bar and devstral's revenue cliff, that plainness is the argument.

    262,144 tokens natively, extendable to a million through yarn, which is what makes repository-scale agentic coding practical rather than aspirational. the flagship is 480b total with 35b active.

    the flagship also needs about 240gb at 4-bit, which is multi-gpu server territory. qwen3-coder-next, added in february 2026 at 80b total and 3b active, is the answer to that — alibaba claims it matches models with ten to twenty times the active parameters, and at roughly 40gb it lands within reach of serious local hardware. one caveat: secondary sources report swe-bench verified anywhere between 50 and 66.5% and we found no authoritative primary figure, so we're treating that metric as unresolved.

    pros
    • +plain apache 2.0, verified in the licence file
    • +262k native context extending to 1m via yarn
    • +qwen3-coder-next brings it to ~40gb for local use
    • +38.7 on swe-bench pro from the model's own repo
    cons
    • flagship needs ~240gb at 4-bit — server only
    • swe-bench verified figures conflict across sources
    • fill-in-the-middle support unconfirmed for this generation
    • two very different models under one name
  2. 2

    DeepSeek Coder

    the brand is now an api alias — and the models behind it went from a restrictive weights licence to plain mit.

    88/100

    verdictthe strongest swe-bench numbers here under a licence that got more permissive rather than less — with no small variant left for anyone who wanted the original.

    best for
    frontier-tier coding where the score matters more than running it yourself.
    price
    free (mit)
    pricing note
    'deepseek-coder' resolves to deepseek-v4-flash at 284b total and 13b active; v4-pro is 1.6t total and 49b active
    free tier
    yes
    licence
    mit
    params
    284b/13b active (flash)
    context
    not confirmed per variant
    commercial use
    unrestricted
    runs on
    ~140gb at 4-bit

    deepseek moved in the opposite direction to everyone else in this ranking. the original 2023 coder generation paired mit code with a separate, more restrictive custom weights licence; the current v4 weights are plain mit for both. in a field where minimax relicensed to non-commercial after launch and mistral gates its best coder behind a revenue cliff, a vendor loosening its terms deserves saying out loud.

    the performance is the top of this page: 80.6% swe-bench verified for v4-pro and 79.0% for v4-flash, with 55.2% on livecodebench in flash's non-thinking mode. these are aggregated figures rather than ones we confirmed on deepseek's own benchmark page.

    what's gone is the thing that made deepseek coder useful to individuals. the standalone line no longer exists — 'deepseek-coder' is an api alias pointing at the general v4 family — and the smallest current option, v4-flash, is 284b total and needs about 140gb at 4-bit. the original shipped at 1.3b to 33b. anyone who chose it for local editor completion has no successor here.

    pros
    • +plain mit on both code and weights, a genuine loosening
    • +80.6% swe-bench verified on v4-pro
    • +frontier-tier coding without a licence negotiation
    • +one-million-token-scale context in the v4 family
    cons
    • no standalone coder models remain — it's an api alias
    • smallest option needs ~140gb at 4-bit
    • nothing runs locally, unlike the original 1.3b-33b line
    • fill-in-the-middle support unconfirmed for v4
  3. 3

    Seed-Coder-8B

    plain mit, native fill-in-the-middle, and it fits in five gigabytes — the practical local pick.

    84/100

    verdictthe best thing here that actually runs on a laptop, with the cleanest licence and the fill-in-the-middle training the frontier models skipped.

    best for
    editor completion on your own machine, where latency beats leaderboard position.
    price
    free (mit)
    pricing note
    8b parameters across base, instruct and reasoning variants; roughly 5gb at 4-bit
    free tier
    yes
    licence
    mit
    params
    8b
    context
    32k tokens
    commercial use
    unrestricted
    runs on
    ~5gb at 4-bit

    bytedance trained it natively on fill-in-the-middle and suffix-prefix-middle objectives, which is what editor integration actually needs and what most of the large agentic models in this ranking don't offer. at 8b and roughly 5gb quantised it runs on hardware you already own, and completion latency is local rather than a round trip.

    the licence is plain mit, confirmed from the repository. no thresholds, no use restrictions, no attribution mandate — the same standing as qwen3-coder and deepseek, at a fraction of the size.

    it reports humaneval 84.8, mbpp 85.2 and 24.7 on livecodebench for the october 2024 to february 2025 window, self-reported on its own model card. the ceiling is real: 8b is the only size, there's been no update since june 2025, and it cannot compete with 480b agentic models on repository-scale work. within the small-model tier it is the strongest thing here.

    pros
    • +plain mit with no conditions
    • +native fill-in-the-middle and spm training
    • +~5gb at 4-bit — genuinely local
    • +humaneval 84.8 and mbpp 85.2 self-reported
    cons
    • 8b is the only size — no larger variant
    • no update since june 2025
    • 32k context, shortest among the current models here
    • cannot match frontier agentic coders on large repos
    advertisement
  4. 4

    Granite (IBM)

    apache 2.0, native fim, half a million tokens of context — and the dedicated code line is officially deprecated.

    80/100

    verdictthe cleanest licensing story in this ranking and native fim at 5gb — on a product line ibm has folded into its general models.

    best for
    enterprises wanting unambiguous licensing and very long context at sizes that run locally.
    price
    free (apache 2.0)
    pricing note
    granite 4.1 shipped 29 april 2026 at 3b, 8b and 30b dense; the standalone granite code family is marked not recommended for new projects
    free tier
    yes
    licence
    apache 2.0
    params
    3b, 8b, 30b dense
    context
    512k in 4.1
    commercial use
    unrestricted
    runs on
    ~5gb at 4-bit (8b)

    apache 2.0, verified in the repository's raw licence file, with ibm's documentation and the licence in complete agreement — which after codestral and starcoder2 is worth noting as a category in itself. granite 4.1 also signs its weights cryptographically, which nothing else here does.

    the practical specs are strong for local work: 8b at roughly 5gb quantised, native fill-in-the-middle documented by ibm, and context extended to 512k in 4.1 through staged extension with model merging. for an editor integration inside a regulated environment that combination is hard to beat.

    the caveat is what happened to the specialist. ibm's own model card for granite code says 'not recommended for use in new projects' — the dedicated line is deprecated and coding is now one capability of the general granite models. that means cleaner licensing and less code-specific tuning, so it trails dedicated coders like qwen3-coder and seed-coder on pure code benchmarks.

    pros
    • +apache 2.0 with licence and documentation in agreement
    • +cryptographically signed weights
    • +native fill-in-the-middle support
    • +512k context in granite 4.1 at ~5gb for the 8b
    cons
    • dedicated granite code line explicitly deprecated
    • coding is now a general-model capability, not a specialism
    • trails dedicated coders on pure code benchmarks
    • legacy code models capped at 4k context
  5. 5

    Devstral

    the small one is apache 2.0 and the big one stops working for you the month you pass $20m in revenue.

    77/100

    verdicttwo models, two licences, and mistral says so plainly — which after codestral from the same company is a meaningful improvement in candour.

    best for
    agentic coding scaffolds, at 24b for anyone, at 123b for companies under the revenue line.
    price
    free below the threshold
    pricing note
    devstral 2 at 123b under a modified mit licence with a $20m monthly revenue cutoff; devstral small at 24b under plain apache 2.0
    free tier
    yes
    licence
    modified mit (123b), apache (24b)
    params
    123b and 24b
    context
    128k on small
    commercial use
    capped at $20m/month on 123b
    runs on
    ~14gb (24b), ~70gb (123b)

    devstral 2's 123b model carries a modified mit licence that revokes all rights if your company's global consolidated monthly revenue exceeded $20 million in the preceding month. that is a hard cutoff rather than a graduated fee, pushing larger firms onto mistral's paid api or a negotiated deal. devstral small at 24b is plain apache 2.0 with nothing attached.

    credit where it's due: mistral's own launch material states the split explicitly rather than letting people discover it in the licence file, which is the failure mode we found at fish speech and marker. same company shipped codestral under a flat non-commercial bar while framing it as a download, so the contrast within one vendor is instructive.

    on capability, devstral is built for agentic coding scaffolds rather than editor completion — it was developed with all hands ai for openhands-style tooling — and devstral small 2507 reported 53.6% on swe-bench verified, up 6.8 points on its predecessor. the 24b runs in about 14gb; the 123b needs 65 to 70gb.

    pros
    • +devstral small 24b is plain apache 2.0
    • +mistral states the licence split explicitly
    • +purpose-built for agentic coding scaffolds
    • +24b runs in about 14gb at 4-bit
    cons
    • 123b rights revoked above $20m monthly revenue
    • hard cutoff rather than a graduated licence fee
    • fill-in-the-middle support unconfirmed
    • devstral 2's context length not confirmed
  6. 6

    GLM-4.6

    near-frontier coding under what its badge says is mit — and the licence file itself returns 404.

    72/100

    verdictreportedly close to claude sonnet 4 on real-world coding, with the one thing this ranking insists on — a readable licence — missing.

    best for
    hosted agentic coding through claude code or cline-style tooling, where you won't be self-hosting anyway.
    price
    free (mit, unverified)
    pricing note
    roughly 355b parameters; the hugging face licence tag reads mit, but the raw licence file returned 404 on both path variants we tried
    free tier
    yes
    licence
    mit per tag, file unreadable
    params
    ~355b
    context
    200k tokens
    commercial use
    unrestricted, unverified
    runs on
    ~180gb at 4-bit

    glm-4.6 was zhipu's coding-focused release and is reported at near-parity with claude sonnet 4 on cc-bench, a human-evaluated real-world coding benchmark. it's positioned for claude code and cline-style integrations, with 200k context, up from 128k in glm-4.5.

    the licence is the problem for a page built on reading licence files. hugging face tags it mit and zhipu's messaging says fully open and self-hostable with no lock-in, but the raw licence file returned 404 on both paths we tried, so we could not check the text for an added clause. given that kimi k3's licence carried two commercial gates invisible from its badge, a metadata tag is not proof.

    at roughly 355b it needs about 180gb at 4-bit — datacentre only. and the coding specialism has moved on: glm-5.1 and 5.2 have shipped as general flagships claiming stronger coding, neither branded as a coder variant, so 4.6 is the last one carrying the label.

    pros
    • +reported near-parity with claude sonnet 4 on cc-bench
    • +200k context, up from 128k in glm-4.5
    • +built for claude code and cline-style integration
    • +mit per the hugging face licence tag
    cons
    • raw licence file returned 404 — text unverified
    • ~180gb at 4-bit, datacentre only
    • superseded by glm-5.1 and 5.2 as general flagships
    • no primary benchmark figure retrievable from zhipu
  7. 7

    StarCoder2

    600 languages at two gigabytes, under a licence whose conditions you have to pass on to your own users.

    66/100

    verdictstill the best small multi-language coder and two and a half years old — with commercial permission that comes with strings attached to your own terms of service.

    best for
    broad language coverage on small hardware, for teams willing to carry the licence terms downstream.
    price
    free (openrail-m)
    pricing note
    3b, 7b and 15b sizes at roughly 2gb, 5gb and 9gb at 4-bit; commercial use permitted subject to binding use restrictions
    free tier
    yes
    licence
    bigcode openrail-m v1
    params
    3b, 7b, 15b
    context
    16k, 4k sliding window
    commercial use
    permitted with use restrictions
    runs on
    ~2-9gb at 4-bit

    trained on the stack v2 across more than 600 programming languages, with native fill-in-the-middle, at sizes from 3b to 15b — roughly 2gb to 9gb quantised. bigcode's paper reports the 15b matching or beating codellama-34b, a model more than twice its size. for breadth of language support at local scale nothing here competes.

    the licence is openrail-m and it is not what people assume. commercial use is permitted, and attachment a attaches binding use restrictions you must pass into your own terms of service for your end users: no malware, no verifiably false or harmful disinformation, no impersonation for deception, no undisclosed fully-automated decisions affecting people's legal rights, and public-facing generated content must intelligibly disclose that it is machine-generated. the model card doesn't spell those out — the attachment does.

    the age is the other issue. released february 2024 with no successor identified, it now sits well behind current-generation models on agentic and swe-bench-style tasks, and its 16k context with 4k sliding-window attention is short for repository work.

    pros
    • +600+ programming languages via the stack v2
    • +sizes from ~2gb, the smallest here
    • +native fill-in-the-middle training
    • +15b reported matching codellama-34b
    cons
    • openrail-m restrictions must be passed to your end users
    • disclosure obligation on public-facing generated content
    • no update since february 2024
    • 16k context with 4k sliding-window attention
  8. 8

    Codestral

    downloadable, and forbidden in production — the licence is a flat non-commercial bar, not a threshold.

    58/100

    verdictthe most restrictive licence in this ranking by a distance, on a model whose current iteration may not have open weights at all.

    best for
    research, evaluation and prototypes that will never ship.
    price
    research use only
    pricing note
    the open-weight 22b checkpoint is under the mistral ai non-production licence — research and testing only, with commercial use requiring a separately negotiated paid licence
    free tier
    yes
    licence
    mistral non-production licence
    params
    22b (open checkpoint)
    context
    128k tokens
    commercial use
    prohibited without a paid licence
    runs on
    ~13gb at 4-bit

    the mistral ai non-production licence bars commercial and production use outright for the open-weight release — research and testing purposes only. that is stricter than every threshold-based licence here: devstral's $20m revenue cliff, kimi k3's scale gates and marker's funding cap all permit shipping until you get large. this permits none.

    the framing compounds it. mistral's launch page emphasises downloading from hugging face without leading with the prohibition, and the same company ships devstral small under plain apache 2.0 and states devstral 2's revenue cap explicitly. the candour is inconsistent within one vendor's own product line.

    there's a second problem for anyone assessing it today. the current codestral 25.08 appears to be offered only through mistral's paid api, with no open-weight release we could find, so the open-weight framing may now apply only to the legacy 22b checkpoint from may 2024. the fill-in-the-middle support remains genuinely good for editor work — if you're paying for the api, which is the point.

    pros
    • +long-established fill-in-the-middle tuned for editors
    • +128k context on the documented model card
    • +22b runs in about 13gb for research use
    • +available as a supported paid api
    cons
    • non-production licence bars commercial use entirely
    • commercial terms require a private negotiation
    • current 25.08 iteration appears to be api-only
    • launch framing understates the prohibition

how this ranking was made

the licence in each entry comes from the licence file in the model's own repository, read as text rather than taken from the hugging face badge. where we could not retrieve the raw file — glm-4.6's returned 404 on two path variants — the entry says the licence is reported from metadata and unverified against the document.

we report what it takes to run each model at 4-bit, because for a coding model that is often the deciding fact. seed-coder and granite fit in about 5gb; starcoder2's largest is under 9gb; qwen3-coder's flagship needs 240gb and deepseek v4-pro around 800gb. a frontier score is worth less than local latency if you're completing code in an editor.

fill-in-the-middle support is treated as a spec because editor integration depends on it. starcoder2, seed-coder and granite are explicitly trained for it; the agentic models mostly aren't, which tells you what they were designed to do.

benchmark figures are attributed and dated, with the dataset named, and we distinguish a vendor's self-reported model-card figure from a leaderboard position. where secondary sources conflict — qwen3-coder's swe-bench verified number is reported anywhere from 50 to 66.5% — we say the metric is unresolved rather than picking one.

two entries are here to record what happened to them. the standalone deepseek coder line no longer exists as separate models — 'deepseek-coder' is now an api alias pointing at the general v4 family — and ibm's granite code family is marked 'not recommended for use in new projects' on its own model card, with coding folded into general-purpose granite.

our general methodology and disclosures →
was this useful?