Run the open models on your own hardware.

Widely regarded as the sharpest mind in the 30 billion class, and the easiest download on this page. Because only about 3B parameters fire per token, it is fast even on modest hardware: a measured 52 tokens per second on a DGX Spark, and comfortable on a single 24 GB GPU.

Hardware verdict: a single RTX 3090 or 4090, or a 24 to 32 GB Mac, runs this at full speed. Two things to know: pull a current copy of the GGUFs (early January uploads had a routing bug, since fixed and re-uploaded), and the community steers away from Ollama for this one due to chat template issues. Use llama.cpp, LM Studio, or MLX.

Google's dense 31B that punches far above its size, and the best licensed model on this page: Gemma 4 moved from the old Gemma Terms to genuine Apache 2.0, patent grant included. Dense means no expert tricks, but at 4-bit that is still only ~18 GB, and the quant ecosystem around it is the best studied of any model here.

Hardware verdict: a single 24 GB GPU or a 24 GB Mac runs 4-bit comfortably; independent KL-divergence testing found the unsloth dynamic quants and bartowski's ladder lead the field. One flag: vision needs the separate mmproj file with GGUF builds, otherwise your download is text-only.

The compact sibling of the Qwen flagship, and the best consumer-hardware story of 2026 so far. Independent benchmarks show the 4-bit file matching full precision, and a speculative-decode head ships inside every GGUF: one llama.cpp flag buys a 33 to 120 percent speedup, measured across five different GPUs.

Hardware verdict: a 12 GB card runs it at about 10 tokens per second, a 24 GB card flies (31 to 76 tokens per second measured on RTX 3090 and 4090, 135 on a 5090 at full context), and on Macs use MLX rather than llama.cpp for this one: the gap is about 4x today. Turn on the built-in speculative flag (--spec-type draft-mtp) for the free speedup. Skip 1-bit entirely; quality collapses to coin-flip territory there.

The nimble tier of the V4 family, and the most local-friendly of the big MoE models. DeepSeek trained the experts natively in 4-bit, so the near-original file is only ~156 GB and, unusually, the "lossless" 8-bit is just 7 GB bigger than the 4-bit. If you can hold ~165 GB, you are running the real thing.

Hardware verdict: a 128 GB Mac or DGX Spark runs it at 2-bit (reported 13 to 35 tokens per second, engine depending); a 256 GB workstation runs it at effectively original quality. Use a current llama.cpp build and the fixed GGUF uploads: early conversions broke tool calling and multi-turn chat, both since repaired. There is even a dedicated community engine for this family (antirez's DwarfStar) tuned for Macs.

DeepSeek's 7/31 checkpoint of V4 Flash, a startling quality jump at the same size, and now the best-measured quant ladder in the family: three independent quality curves agree on exactly where the knee is. Near-native is loss-free, ~110 GB costs about 16 percent agreement, ~85 GB costs about 25.

Hardware verdict: the best-documented home-run model in this guide. Around 17 to 20 tokens per second on a 128 GB DGX Spark, 27 to 39 on a 128 GB M5 Max MacBook, and one well-liked report of the smallest 82 GB build writing complete working applications at 30 tokens per second. Use a current llama.cpp build, and avoid the two 4-bit IQ4 files for now: there are unresolved garbled-output reports on exactly those, while 3-bit and Q4_K_XL run clean.

MiniMax's swift planner, and the best price-of-entry giant on this page: a single 16 GB GPU plus 96 GB of ordinary system RAM reportedly clears 25 tokens per second at 4-bit. If you want to taste a 200B-class model without exotic hardware, start here.

Hardware verdict: reported 15+ tokens per second on a 128 GB Mac, 25+ on the GPU-plus-RAM hybrid. One sharp caution from the quantizer: avoid CUDA 13.2 with this model, it can produce garbled output.

Xiaomi's lean 310B reasoner, MIT licensed. Upstream it is an omnimodal model; know before downloading that every GGUF today is text-only, because llama.cpp has not yet added its vision and audio path. The full experience lives on vLLM with the original FP8 weights.

Hardware verdict: 128 GB machines run 2 to 3-bit, 4-bit quality wants about 200 GB. One open question the community has flagged: some users report the GGUF behaving differently from Xiaomi's hosted FP8 on long reasoning chains, so temper expectations at low quants.

MiniMax's newest: natively multimodal with a million-token window built on sparse attention. The local catch is real, though: llama.cpp needs a special build for it, and sparse attention falls back to dense there, so keep context modest and treat the million-token story as a server-engine feature. (Built with MiniMax M3.)

Hardware verdict: 3-bit on a 192 to 256 GB machine is the realistic local entry, via the dedicated llama.cpp branch with modest context. Community members run 5-bit on a single 512 GB M3 Ultra and call it near-frontier at home. For the full multimodal, million-token experience, this one really wants a server engine.

Z.ai's brand-new 5-series Flash, days old as we write this: a fifth of Kimi K3's size with a million-token context, and the most exciting new candidate for serious home hardware. The quant ecosystem is hours old, so treat this entry as a snapshot that will improve quickly. One thing to know going in: the source weights are 8-bit, not 4-bit, so unlike the DeepSeeks, 4-bit here costs real quality and the lower rungs fall off hard.

Hardware verdict: the 4-bit builds fit a 256 GB Mac Studio or a 192 GB-plus DDR5 workstation with a 24 GB GPU holding attention and cache. But be honest about the moment: mainline llama.cpp support is still in review (two competing pull requests as we write), MLX needs a bleeding-edge runtime, and nobody anywhere has published a home tokens-per-second number yet. We would rather tell you that than invent one. Check back soon; this card will age quickly, in a good way.

The foremost US-developed open weight of 2026, and the most openly licensed model on this page: NVIDIA published the weights, the training data, and the recipes under the Linux Foundation's permissive OpenMDW license. The hybrid Mamba design is also why it is the most reachable of the giants: 3-bit fits a 256 GB machine.

Hardware verdict: 3-bit on a 256 GB box is real; about 300 GB for 4-bit. Two honest caveats: the multi-token prediction speedup does not exist in GGUF form, and hybrid Mamba support in llama.cpp is young, so expect rougher edges than the pure transformer models. The polished experience is the official NVFP4 on server GPUs.

The first of Z.ai's 745B-class flagships, MIT licensed like the whole family. The GLM 5.x giants share one local story: unsloth's dynamic GGUFs are the only realistic path, 2-bit is the accessible entry at roughly 82 percent of full accuracy, and coding work deserves the 4-bit step up.

Hardware verdict: a 256 GB Mac Studio or a 512 GB EPYC build, at roughly 3 to 9 tokens per second. Long context runs dense in llama.cpp (the sparse attention path is server-engine only).

The methodical engineer of the family, tuned for very long agentic runs. That tuning is exactly where low-bit quantization bites hardest, so of all the GLMs this is the one where the community most insists on 4-bit for real coding work.

Hardware verdict: same floor as GLM 5: 256 GB gets you in at 2-bit, coding-grade 4-bit wants a 512 GB Mac Studio or a 768 GB EPYC workstation.

Z.ai's current flagship and the highest scoring open weight ever released, with a million-token window. Yes, the top-three-on-Earth model is MIT licensed and downloadable: 1.51 TB at full precision, 239 GB at the dynamic 2-bit that people actually run.

Hardware verdict: a named community rig (512 GB RAM, two RTX 3090s, 32-core EPYC) reports about 6 tokens per second; Mac Studios land in the 3 to 9 range at 2-bit. Long context falls back to dense attention in llama.cpp, so the million-token story belongs to the server engines.

Moonshot's trillion-parameter flagship, trained with 4-bit experts from the start. That native INT4 is why the "lossless" download is 595 GB rather than two terabytes, and why the 8-bit file is only 10 GB bigger than the 4-bit: there is no hidden full-precision version you are missing.

Hardware verdict: needs about 350 GB of memory minimum. A used EPYC with big DDR5 runs 2-bit at 8 to 12 tokens per second; a documented Blackwell-pair-plus-EPYC rig runs the 3-bit quality tier at 18 to 21; a 512 GB Mac Studio holds Q4. Image input works in GGUF via the mmproj file; video needs the server engines.

The coding specialist built on K2.6, and the only K2.7 there is. Same trillion-parameter economics, one practical upgrade for agent work: it reaches answers in roughly a third fewer thinking tokens, which matters more at home than anywhere, because every token is your electricity.

Hardware verdict: identical floor to K2.6 (about 350 GB minimum, EPYC or 512 GB Mac territory). Unlike DeepSeek V4 it ships a proper chat template, so tool calling wires up cleanly in llama.cpp. Runs text-only as a code model.

Moonshot's 2.8 trillion parameter flagship and the largest open weight ever released. Honesty first: the hardware floor moved roughly an order of magnitude past K2, and even the 1-bit file is 594 GB. K3 also uses a new 4-bit-native format, so K2-era quants, sizes, and folk wisdom do not transfer.

Hardware verdict: big workstations technically load it and produce 0.7 to 3 tokens per second: running, not usable. Genuine K3 speed starts at an 8x B300-class node, where a single user sees 111 or more tokens per second, 370 with speculative decoding. This is the clearest case in the guide where hosted is not just the cheaper path, it is the only practical way to touch the full model.

Xiaomi's trillion-parameter flagship, prized for token efficiency on long engineering work, and MIT licensed, which is remarkable at this scale. Local running is real but firmly server class: the best documented rig pairs two 96 GB workstation GPUs with 755 GB of RAM.

Hardware verdict: about 300 GB of combined memory is the floor, and quality quants want 550 GB and up. Stick to mainline llama.cpp builds (a load bug is open against the ik_llama fork for this model). Xiaomi's own blazing FP4 serving path exists, but it assumes a multi-GPU server.

The open-weight twin of Qwen3.8 Max and one of the largest open models ever released. We list it plainly: the 397 GB 1-bit GGUF is a genuine marvel of compression whose quality nobody has measured yet, and the one person who ran this model on a home-class machine measured 0.17 tokens per second and called it unusable. This is a rack model.

Hardware verdict: not a local model, full stop. The cheapest usable configuration is an 8-GPU B300 or MI355X-class node at 4-bit, call it a quarter million dollars of hardware or a rental; even 64x H100 measures about 11 tokens per second single-stream. If you want Qwen on your own machine, the 27B in the first section is the one you actually want, and it is genuinely excellent.

The original preview checkpoint, still fully servable (the official 0813 release is just below). The summit: 1.6 trillion parameters, one of the most formidable open weights in existence, under plain MIT. The native FP4-expert download is 865 GB (not the 3.2 TB you might fear), and exactly one community quant currently fits a single machine.

Hardware verdict: barely local, and we say that with affection. A 512 GB Mac Studio runs 2-bit at about 9.5 tokens per second as a glorious stunt; genuinely usable serving means B200-class hardware. If any model on this page makes the case for letting someone else run it, it is this one.