The model loaded. The fans settled. Then the first answer took long enough to make the cloud look tempting.

Verdict: check model size, quantization and context length before replacing the mini PC. Start with Q4_K_M, leave memory headroom, and test your own prompts. Then check speculative decoding support. On shared Wi-Fi, keep the API bound to your machine.

Leave memory for the rest of the workday

On an Apple silicon Mac or a Strix Halo box such as the GMKtec EVO-X2, the processor and graphics share one memory pool. Weights, context, runtime and open apps all draw from it.

Unified or shared memory Sensible starting point Main constraint
24 to 32GB 7B to 14B models A 27B model leaves little working room
48GB Qwen3.8-27B at Q4_K_M Long context and a second large model
64GB 27B model plus a smaller helper Dense 70B models need tighter compromises
128GB Large MoE models or dense 70B Architecture can matter more than fit

The Strix Halo guide identifies Ollama’s qwen3.8:27b build as Q4_K_M, about 17.7GB before context and runtime overhead. These are starting points, not fit guarantees. Your browser and video call still need memory.

Start at Q4_K_M and compare your actual work

Q4_K_M cuts memory use without the most aggressive compression. If the model only fits below Q4, compare a smaller model before accepting heavier compression. That is a practical starting preference, not proof the smaller model will answer better. Try Q5 or Q6 when code matters and memory allows, with prompt and sampling fixed, or you’ll learn more about randomness than quantization.

Context has its own bill. Keep a short preset for everyday work, a longer one for documents, and check what the runtime actually loaded.

Separate the first-token pause from the stream

A long prompt can make the opening pause feel broken even when the answer streams quickly afterward. Apple’s M5 MLX research separates prompt processing from generation; the latter remained limited by memory bandwidth. Update the runtime before comparing results, and note which backend it used.

On Strix Halo, speculative decoding is a setting worth checking. One independent tester on a 128GB Beelink box measured Qwen3.8-27B at about 13 tokens per second without drafting and about 23 with Ollama’s default MTP drafting, 9 warm runs each. The same model blob was used, but the generated text differed even at temperature zero. This is one tester’s result, not a speed or answer-quality promise.

Benchmark your own document task, not someone else’s model.

Keep the API on your side of the door

LM Studio and Ollama both expose OpenAI-compatible local endpoints. Ollama binds to your own machine by default; setting OLLAMA_HOST to 0.0.0.0 opens it to the network. In LM Studio, leave Serve on Local Network off on coworking, hotel or apartment Wi-Fi.

Keeping the API local reduces network exposure; it is not a complete security guarantee. Update before the trip, then check the setting again. In Ollama, disable cloud features if your work must stay entirely local.

Test the trip before taking it

  • Generate the same 500-token answer twice; record first-token delay and speed.
  • Save the exact model tag, quantization, context and a few test prompts. Tags get re-pointed.
  • Restore one real file from the backup drive, such as a Samsung T7 Shield.
  • Turn Wi-Fi off and confirm the runtime, model and documents still open.

Who should skip

Daily benchmarkers are past these defaults. If every task goes to the cloud, the local server is extra luggage.

Still choosing? Start with the packable local LLM workstation guide.

Cited sources


More field guides

More Remote Office guides

All Remote Office guides