Why It Matters
That's $400 today, so system price would go up significantly.
Overview
This is a post asking for advice / user experience. The goal is simple: deploy a small private server for a developer to run a harness that rivals/beats Claude Opus (in perf/intelligence, not necessarily speed). I believe the model to target is a Q3 or so version of the Qwen 3.8 Next model, and to build a dual GPU, 64GB setup.
Why It Matters
That's $400 today, so system price would go up significantly.
Key Details
Streaming n-gram off RAM/SSD (~85GB for total model + KV, minus 27 n-gram = 63GB) would mean the 64GB might be enough.
Budget: as low as possible, up to $4000.
I am willing to get extremely scrappy though.
I am wondering: if I want to run this model with a full 256k context, and want about ~500 tok/sec PP, and +20 tok/sec decode, what kind of cards do I need?
I see my options as follows: Budget and weird Something super silly with 2 or 3 Tesla V100s, the 32GB version, which I assume is possible but have yet to see anyone actually deploy.
--- *Original report published by [Reddit LocalLLaMA (Model Leaks & Local AI)](https://www.reddit.com/r/LocalLLaMA/comments/1wipju7/qwen_next_38_claude_opus_level_local_model_what/).*