GenAI Fundamentals
Understanding Large Language Models (LLMs) and their capabilities.
How it Works
At its core, a Generative Pre-trained Transformer (GPT) is a prediction engine. Given a sequence of text (tokens), it predicts the most statistically likely next token.
Tokens
The basic unit of text. 1,000 tokens is roughly 750 words. Models are billed and limited by token count.
Context Window
The "short-term memory" of the model. How much text it can consider at once (e.g., 8k, 32k, 128k).
Temperature
A parameter controlling randomness. Low temp (0.2) = focused/deterministic. High temp (0.8) = creative/random.
Embeddings
Converting text into numerical vectors so computers can understand semantic similarity.