Basically the title.

For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 day ago (1 child)

How's your experience with the 1070?

  • source
  • parent
  • hideshow 1 child comment
  • Mixed?

    It's the lower bound of where LLMs are helpful. Anything less and the models would be either too slow to work or too dumb. With the 1070 they are already painfully slow but at least they're still good enough to do some menial tasks.

    A good harness for web search etc does a lot though.

  • source
  • parent