AI

I wanted to try Qwen 3.8 Flash Next, a 177B MoE model, on the same RTX 3060 used in Part 1. Its quantized weights occupy 76.3 GiB, and my machine has 12 GB of VRAM and 32 GB of system RAM.

The first launch froze my desktop. Getting it working required changing how the server loaded the weights.

I used the UD-IQ3_XXS quantization and GenerelSchwerz’s llama-moe-cache branch to test GPU expert caching with weights backed by NVMe storage.

This is Part 2 of a three-part series on running, benchmarking, and scaling local coding agents. Part 1 covers the local Qwen setup; Part 3 explores running a larger MoE model on the same desktop hardware.

In Part 1, I got Qwen 3.6 35B A3B NVFP4 running locally on an RTX 3060 with a 64k context window and wired it into Crush, Pi, and Oh My Pi. I wanted to know how that setup compared with the terminal coding agents I already had on subscriptions. Could it finish the same tasks, and would I want to merge the patches?

This is Part 1 of a three-part series on running, benchmarking, and scaling local coding agents. In Part 2, we benchmark this setup across 640 public trials and 280 private trials on a production Go monorepo, including Claude Code, Codex, Antigravity, local Qwen, and two hosted models. In Part 3, we build and test the specialized llama-moe-cache fork to run 177B Qwen3.8-Flash-Next on desktop hardware.

I wanted a capable local reasoning model for coding agents on my workstation, but my GPU is a standard desktop NVIDIA GeForce RTX 3060 with 12GB VRAM. The weights would not all fit in VRAM, so I needed a setup that could offload them.

I use big LLMs to draft blog posts and LinkedIn posts. They help with research and structure, but I still spend time rewriting them to sound like me. Every draft comes out with the same polished, slightly corporate voice, and even after I edit it by hand, something of that voice stays in the text. There is actually research confirming this: a 2026 study on post-editing found that even when humans edit LLM drafts, the result stays measurably closer to LLM style than to the author’s own writing.