This seems to be a wrapper over Kokoro which is an Apache-licenced TTS model and library which has been out for over a year. That model lists about 10% of its approx 1000-hour training data but also says that all of its training data was CC, synthetic (by way of closed source TTs), or otherwise permissively-licensed.
I noticed the same thing. I went and tried it just now and found that there’s a reasoning switch on the web ui (it looks like a light bulb in the chat box💡). It defaults to off
I find llama.cpp with Vulkan EXTREMELY reliable. I can have it running for days at once without a problem. As far as tokens/sec that’s that’s a complicated question because it depends on model, quant, sepculative, kv quant, context length, and card distribution. Generally:
Models’ typical speeds at deep context for agentic use. Simple chats will be faster
| Model | Quant | Prompt Processing (tok/s) | Token Generation (tok/s) | Hardware | Quality |
|---|---|---|---|---|---|
| Qwen 3.5 397B | Q2_K_M | 100-120 | 18-22 | 2 x 7900 + 4 x Mi50 | ★★★★★ |
| Gemma4 31B or Qwen3.5 27B | Q8_0 | 400-800 | 20-25 | 2 x 7900xtx | ★★★★ |
| Qwen 3.6 35B | Q5_K_M | 1000-2500 | 60-100 | 2 x 7900xtx | ★★★★ |
| Qwen 3.5 122B | Q4_0 | 200-300 | 30-35 | 4 x MI50 | ★★★★ |
| gpt-oss 120b | mxfp4 (native) | 500-800 | 50-60 | 3 x Mi50 | ★★ |
| Nemotron 3 Nano 30B | IQ3_K_XXS | 2500-3000 | 150-180 | 1 x 7900xtx | ★ |
I’ve used Gemma4-31B for agentic coding and its actually very good as far as local models go. Its less verbose than qwen3.5 so it ends up being faster too. Gemma4-26B can do agentic but its noticeably worse so you have to go slow with it. I haven’t had any coherence issues like other commenters mention but I’ve only been using higher quality quants from unsloth on llama.cpp
I wish I bought an epyc board last year instead of my rig. Would have been far fewer headaches and, with the price of RAM, I would have quintupled in value now!
This is something I learned the hard way.
Consumer hardware is limited by multiple factors when it comes to PCIe connectivity.
Your graphics card might be a 16 lane card (referred to as “x16”), but sometimes, not all of them are used. Aforementioned 5060ti - I believe only uses x8. Some devices like graphics cards can use a physically smaller slot with an adapter for a loss in performance (a few frames in game play performance)
Similarly, your motherboard might have a x16 slot and another x16 at the bottom. That second slot might only function as x8 or even x4. Does this matter? Sort of. Inta-card communication aka peer to peer communication can affect affect performance and that can compound with multiple cards.
Even worse, some motherboards may have all sorts of connectivity but may have limitations like only 2 out of the bottom 4 slots, PCIe and m.2, can work at a time. ASK ME HOW I KNOW.
Your CPU controls PCIe. It has a hard cap in how many PCIe devices it can handle and what speed. AMD tends to be better here.
Enterprise gear suffers from none of this bs. Enterprise CPUs have a ton of PCIe lanes and enterprise motherboards usually match the physical size of their PCIe slots to their capacity and support full bifurcation*
PCIe lanes are used up by and consumable by m.2, MCIO, and occulink to name a few. That means that you can connect a graphics card to either one is those of you can figure out the wires and power**
Bifurcation is a motherboard feature that lets you split PCIe capacity, so a 16x slot can support two x8 devices. My motherboard lets me do this on just the main slot and in a strange x8x4x4 configuration. I have an MCIO adapter (google it) which plugs into the PCIe and gives me 3 PCIe adapters with those corresponding speeds.
it also has 2 m.2 slots which connect to the CPU. One is them, I use for a nvme ssd like a normal person. The other is an m.2 to PCIe adapter which gives me an x4 PCIe slot. For those keeping track, that’s 24 PCIe lanes so far. That’s the maximum my processor Intel 265k can handle
But wait! The motherboard also has a kind of PCIe router and that thing can handle 8 more lanes! So I use the bottom 2 PCIe lanes on my motherboard for 2 cards at x4 each. The thing that kills me is that there are more m.2 ports. But the mobo will not be able to use any more than 2 devices at once. AND even though that bottom PCIe slot is sized at x16, electrically, its x4.
Do your research (level1techs is great) and read the manuals to really understand this stuff before you buy
My mobo for reference ASUS: TUF GAMING Z890-PRO WIFI
Vulkan helps with speed. Must benchmarks prove that out. Concurrency is a mixed bag. You can get some with llama.cpp bit vllm is concurrency king.
Just a couple of weeks ago llama.cpp released tensor parallelism which helps, but its still a experimental feature.
Unfortunately, I don’t know of any diffusion runners that work in vulkan. If someone has expertise, let me know!
I’m going to be brutal with you. I spent a few thousand dollars on 176GB of AMD vram because I was happy with getting vram for cheap and I hate Nvidia. It works and its nice to be able to run bigger models at usable performance, but if you need serious concurrency or good support for diffusion, you NEED Nvidia. AMD(and likewise Intel) just doesn’t have the environment support for non-server GPUs. Again, coming from someone who’s using this shit daily.
If you understand this limitation, then yes those B70s are cool as are AMD Pro 9700 which might have slightly better support rn. You may consider nvidia V100s which are old and cheap. I always recommend people start with 3090s (as a general powerhouse) or a pair of 5060tis (for really hood llm support) though. It will make your life easy if you can afford the vram limitation
Not exactly. Some people have used Claude’s thinking patterns to train other open models.
This one has been in the top 5 on huggingface for weeks https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled
I’ve done some testing with the two large models and my initial impression is that they seem very similar in quality to Qwen3.5 35B and 27B. Some notable exceptions:
Can’t wait for the inevitable Claude/Gemini distils.
My verdict is that even though these models benchmark slightly lower then their Qwen equivalents, their performance and support will likely drive me to pick them.
If my guy was out there to subvert my expectations, he got me. Still bad. But bad with a purpose?
Not to besmirch the quality of the content or the opinions of the author but highlighter yellow as a website background in 2026. Why?!?!?
An untested disaster recovery plan is wishful thinking
As always, this is incredible engineering. I’m so excited that M series macs have a supportability path beyond Apple’s proprietary support
surprised_pikachu.jpg
You can look at manufacturers info pages and see what they support. Intel integrated chips usually list the capabilities and you’ll want to double check with your mini PC or motherboard manufacturer to make sure they support it too. I think any i5+ from the past 5 years with integrated graphics should be able to play/decode 4k media (someone correct me if this sounds crazy). Fornsure my core ultra 265. As far as codec support, I’m not familiar with the compatibilities but I’m sure everything CAN be played on recentish hardware. Encoding is out of my weelhouse.
I’ve used HDMI 2.1 hdr 4k120 on Linux with Nvidia, AMD and Integrated Intel. AMD will be the best experience especially on cards from the past 5 years. Nvidia, with proprietary drivers, on 3000 series or newer should be good for a few more years. I heard 2000 series will be dropped from support soonish m. Intel HDMI 2.1 is a pain on linux and I’ve only been able to get HDR 4k120 using a special DP to HDMI cable.
Notepad++ works fine on Wine on Mac and Linux. After being away from it from awhile, I realized I don’t need it anymore. I would often use the column edit mode and recorded macros, but I just bash script those now. I guess I’m a different person now?!?
You should check out the LibRedirect Firefox addon. It does exactly what you’re describing you need. You can set up multiple redirect destinations for all kinds of sites and its easy to turn on/off
I use a lot of PCIe extenders to abuse my consumer motherboard with too many GPUs. You should be fine as long as your processor and chipset have enough lanes and power. Cheap extenders can be had for like $10.