There is a quiet architectural argument running through the local-AI world: dense models versus mixture-of-experts. One is a reliable workhorse, the other a specialist squad. Both have their place — but only one of them belongs on any given machine, and the reason is not the one most people expect.
The distinction
A dense model is a company where every employee attends every meeting. Global strategy or who finished the yogurt, the entire staff is mobilised. Every parameter is used for every token generated.
A mixture-of-experts model is a conglomerate with specialised departments. A router at the front desk reads the task and sends it to a few relevant departments; the rest stay idle. Far fewer active parameters per token — only a fraction of the network wakes up.
The consequence is speed. Less arithmetic per token means high throughput, and on the right hardware MoE feels dramatically faster.
The catch that trips everyone up
Here is the misconception that drives most bad hardware decisions:
The building does not get smaller because most of the staff are idle.
Even when only a few experts are active, the whole model still has to fit in memory. MoE saves compute cycles. It does not save storage or RAM. That entire model file must sit in your VRAM or system memory at all times — and if the space is not there, the specialists never show up for work.
So: MoE is about saving processing power, not memory footprint. Every other decision follows from keeping that distinction clear.
What that means for your machine
It makes dense models the safer choice on memory-limited laptops. They are simple and predictable, without the routing logic that can introduce overhead.
It makes MoE attractive on high-memory Macs and workstations — machines with abundant memory and a need for throughput. If you are running coding agents or work that leans on deep reasoning, MoE is frequently the better performer.
The pattern that makes MoE compelling at the high end is a model with a very large total parameter count that activates only a small fraction per token — Qwen3.8-2.4T-A95B is the example from the video: server-grade capability at desktop generation speeds, provided you have the memory to hold the whole thing. And some models are simply server-scale regardless of architecture — no quantisation makes them a desktop proposition, and forcing them is wasted effort.
As with the quantisation ladder, the specific model names in the video are a dated snapshot. The selection rule is not: there is no universal best model, only the best model for your specific hardware.
Settings worth changing (and the ones worth leaving alone)
Defaults in tools like LM Studio are safe starting points. They are not automatically optimal for every model, machine or task.
Leave alone: chat template on auto, GPU offload and flash attention on auto. These avoid compatibility problems, and overriding them usually costs more than it gains.
Choose deliberately:
- Instruct or chat builds, never base models — base models are for research, not conversation.
- Q4 as the default quantisation, Q5 if it fits. Avoid Q2 and Q3; they lose too much nuance.
- Context of 8–16K for daily use, 32K only for long documents. High context consumes VRAM quickly, and over-allocating it is the most common self-inflicted memory problem.
- 8-bit KV cache when memory gets tight during long conversations.
- Temperature 0.2–0.5 for factual work and code, 0.6–0.8 for natural conversation — but follow the model card first.
- Reasoning effort low for simple summaries, medium or high for difficult code, planning or mathematics. Do not spend compute on easy tasks.
- Output limits sized to the job, to stop the model rambling.
And one rule that matters more than any individual setting: change exactly one thing at a time, then test quality and speed. Otherwise you learn nothing about which change helped.
If you are running on Apple Silicon, load the model's own generation defaults — they exist for a reason — stick to one or two concurrent requests, and set a time-to-live on occasional models so they release memory.
Always require confirmation for risky tool actions. A model calling a function is still software making a choice.
The decision rule
It comes down to two questions, in order.
First: does the model fit comfortably, with room for context and room for your operating system? If not, stop. Choose a smaller model or a stronger quantisation. Nothing else you tune will rescue a model that does not fit.
Second: limited memory means dense. Abundant memory plus a need for throughput means try MoE — and benchmark it on your own hardware rather than trusting anyone's table, including this one.
The best local model is not the biggest. It is the smallest one that performs your task correctly, quickly and reliably. Efficiency is the flex.
Model recommendations in the video are a snapshot as of 29 August 2026. Local-AI tooling moves quickly — check the model card and benchmark on your own hardware.


