trebach comments on "LLMs can do anything" except make positive cashflow.

Funny

16,293 readers

6,542 active

424 users here now

General rules:

Be kind.
All posts must make an attempt to be funny.
Obey the general sh.itjust.works instance rules.
No politics or political figures. There are plenty of other politics communities to choose from.
Don't post anything grotesque or potentially illegal. Examples include pornography, gore, animal cruelty, inappropriate jokes involving kids, etc.

Exceptions may be made at the discretion of the mods.

founded 3 years ago

moderators

[–] trebach@sh.itjust.works 2 points 4 days ago (1 child)

They're actually not because they have randomness built in. Since we are working with probabilities, it won't always pick the next token that has the highest probability and the randomness can be tuned via a "temperature" setting to make it more or less likely that it will choose the most probable token.

The weights for the model could be stored in a firmware chip but you still need RAM because it pulls all the weights into RAM in order to perform the calculations.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] Strawberry@lemmy.blahaj.zone 1 point 3 days ago* (last edited 3 days ago)

The LLM itself is deterministic. It outputs a vector that we interpret as a probability distribution over the set of tokens. It's the program using the output of the LLM, such as a chatbot program, that selects an individual token using (or not using) these vector elements as weights.

The weights do not change once the model is trained. This is why I am suggesting they could be incorporated directly into the structure of an ASIC for a specific model, rather than storing them in memory.

Edit: Of course another major factor could be that the models are just to big to be wholly implemented in a single IC by any currently existing manufacturer.

permalink

fedilink

source

parent