Bernie vs. Claude

[HesNotHere]

9. LLMs are non-deterministic

They do this by sampling and randomizing. For example, consider "The dog is ____".

It may determine that "running", "barking" and "brown" are the only likely possible answer that meet a threshold (top_p). The machine may be told to only to only select the two most likely (top_k), which would be "running" and "barking". Now lets assume the probability of "running" is .75 and "barking" is .25, it will randomly pick either based on the likelihood.

Then it goes again. Maybe it picked "running". Now perhaps the most likely token is a ".", "fast" or "away".....

Every time a token is added, the context changes, and the possible next token probabilities shift. The longer the response and more variability allowed, the more likely two responses will differ.

IOW, LLMs do not know anything. They predict numerical sequences based on their training, and they do it with a random component. Thats why you got such different answers. The number of possible responses to a complicated question is infinite.