I have nearly the same setup, except OpenCode for the harness. I run 200k context for anything that doesn’t need vision, or 150k otherwise. Typically get about 30 t/s decode With MTP, starts at close to 800 t/s for prefill and drops to the 200s for a long context. I’d imagine higher decode speeds are possible with some tinkering, but I haven’t gotten around to it.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
replies: