What does a lower temperature setting in LLMs result in?

A lower temperature setting in large language models (LLMs) typically results in more predictable and structured responses. The temperature setting is a parameter used during the generation of text that controls the randomness of predictions made by the model. When the temperature is set low, the model's outputs are more focused on the highest probability responses, leading to answers that tend to be safer, more reliable, and consistent with common knowledge or patterns in the training data.

This predictability in responses occurs because the model leans towards generating outputs that are generally accepted or expected, minimizing the risk of generating unusual or unexpected answers. Consequently, the structure of the responses tends to be more coherent and aligned with the context of the prompt, hence giving a sense of authoritative and structured language.