On LLM Settings | Jason Siu

type

Post

Created date

Apr 21, 2024 05:51 AM

tags

Prompt Engineering

status

Published

Language

English

From

summary

slug

llm-param-settings

password

Author

Priority

Featured

Cover

Origin

Type

URL

Youtube

icon

Parameter	Purpose
Temperature	Controls the randomness of responses. Lower values yield more deterministic results; higher values promote diversity and creativity.
Top P	Controls the range of token probabilities considered (nucleus sampling). Lower values yields more confident and factual responses.
Max Length	Determines the maximum number of tokens in a response. Setting this helps you prevent long and/or irrelevant response and manage computational costs.
Stop Sequences	Controls specific strings that, when generated, will halt further token generation. Useful for controlling the structure of outputs. For example, you can tell the model to generate lists that have no more than 10 items by adding "11" as a stop sequence.
Frequency Penalty	Applies a penalty to tokens that have appeared frequently. The higher the frequency penalty, the less likely a word will appear again.
Presence Penalty	Penalises tokens based on their occurrence to avoid repeated phrases, aiding in generating more diverse content. - A token that appears twice and a token that appears 10 times are penalized the same. - If you want the model to generate diverse or creative text, you might want to use a higher presence penalty. Or, if you need the model to stay focused, try using a lower presence penalty.

Note: The general suggestion is to adjust temperature or Top P but not both.