LLM Fundamentals Terms

What we verified before publishing

Fact-check status: passed

The following claims were verified through Perplexity Sonar before the questions were finalised. The host can reference any of them during play:

• A token is a small chunk of text processed by an LLM, which can be a whole word, part of a word, or punctuation.
• The context window is the number of tokens an LLM can process at once, such as 4,096 tokens equaling about 3,000 words.
• The knowledge cutoff is the date when an LLM stopped learning, meaning it lacks knowledge beyond that date unless updated.
• Pre-training is the first stage of LLM training where the model learns patterns, grammar, and general knowledge from a massive dataset.
• Parameters are the internal settings or weights LLMs adjust to generate responses, with more parameters usually meaning better performance.
• Inference is the operation where a generative LLM produces text by taking a sequence of tokens as input and predicting the next token.
• Hallucination occurs when an LLM generates factually incorrect but plausible-sounding information by filling in gaps without real grounding.
• Few-shot learning is the ability to provide an LLM examples of a task to perform, which enhances its performance on that task.
• Retrieval-Augmented Generation (RAG) combines AI with external data sources so the model can pull in relevant information instead of relying only on training data.
• Chain-of-Thought prompting guides an LLM to reason step-by-step through a problem rather than jumping directly to an answer.
• Reinforcement Learning from Human Feedback (RLHF) aligns model behavior with human preferences by having humans rank the model's responses.
• Rective embeddings are vector representations of text used for similarity matching and stored in vector databases for fast retrieval.