Key generative AI terms with concise definitions

I can never remember all of these, so here’s a list of key generative AI terms with concise definitions:

Key Generative AI Terms

Inference – The process of an AI model generating output based on input.
Training – Teaching an AI model by adjusting its weights using large datasets.
LLM (Large Language Model) – A neural network trained on vast text data to generate human-like language responses.
Token – A unit of text (word, subword, or character) that a model processes.
Weights – Numerical values that determine how much influence different inputs have on a model’s output.
Open Weights – AI models whose weight parameters are publicly available for use, modification, and inspection.

Distillation – Compressing a large AI model into a smaller one while retaining most of its capabilities.
Fine-tuning – Retraining a model on a specific dataset to specialize it for certain tasks.
Pretraining – The initial training phase where a model learns general language patterns before fine-tuning.
RLHF (Reinforcement Learning from Human Feedback) – A technique where human reviewers rate model responses to refine behavior.

Transformer – The neural network architecture behind modern LLMs, enabling efficient text generation.
Self-Attention – A mechanism that helps models understand relationships between words in a sentence.
Prompt Engineering – Crafting input text to guide an AI model’s response effectively.
Hallucination – When an AI generates incorrect or nonsensical information that sounds plausible.
Bias – Systematic errors in an AI model’s outputs due to imbalances in its training data.

Zero-shot Learning – When an AI performs a task it wasn’t explicitly trained for by leveraging general knowledge.
Few-shot Learning – When an AI adapts to a task using just a few examples provided in the input.
Multimodal AI – AI models that process and generate multiple types of data (e.g., text, images, audio).

From ChatGPT.