you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 1 week ago (4 children)

Oh thank fuck. I thought I was going crazy... My first attempt at local models was a Qwen. A base model. I swear to god all I said was hello and the thing started and never stopped. I guess that's what a filibuster must be like? It was both amazing, yet terrifying. I've a bit more xp now, but yeah Qwen and really really thinky, dear gods ! My colleague got himself an abliterated one and just leave it to run overnight trying to pentest shit in his home network, but I guess I prefer something a bit more hands on, myself.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 1 week ago (3 children)

    Heh. I use Qwen3.8-27B as my main coding agent, and I sometimes feel really bad for it as I watch the stream of thought go by filled with "wait... am I doing this right? What if I'm completely wrong about this? Is this really what I should be doing??" I want to pet it and assure it that it should have more confidence in itself. :)

    But it's the tradeoff that Qwen made to cram that much smarts into such a small bundle of parameters, I guess.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 1 week ago (2 children)

    Yeah, I can see that but it's horrendously slow, I find. I tried a 3.8 online and I just dont' like the way it works, overall :( It goes on and on and on and never stops to ask despite that being one of my main rules.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 5 points 1 week ago (1 child)

    There's a fine-tuned version called Swift that's been trained to use 58.3% fewer reasoning tokens, apparently with almost no loss of quality. I've not been in a hurry with my personal usage (I set my coding agent loose on a task and then go have tea or whatever) so I haven't experimented with it much but it might be worth checking out if the verbosity of reasoning is a problem.

    Qwen4 will be coming out soon too, it has a 27B version, so perhaps they'll have incorporated more efficient reasoning into it from the start. I'm looking forward to testing it out.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 week ago

    Good tip! I'll definitely be checking this out, as someone who also uses Qwen3.8 27B. The capability leap over the model I used before is definitely worthwhile, but wow does it overthink really easily.

  • source
  • parent