LLM Fundamentals

What we verified before publishing

AI-generated content may contain mistakes. Check the sources below before relying on important information.

The following claims were verified through Perplexity Sonar before the questions were finalised. The host can reference any of them during play:

• A Large Language Model (LLM) is a deep learning neural network trained on immense amounts of text data to predict the next token in a sequence.
• LLMs are built on a transformer architecture, which uses self-attention to capture patterns and relationships between words in text sequences.
• A token is the basic unit of text processed by an LLM, and can be a full word, a subword, a single character, or punctuation.
• LLMs generate text one token at a time using an autoregressive process, calculating the probability of potential next tokens and selecting the most likely one.
• Zero-shot learning allows a base LLM to respond to broad requests using prompts without explicit training for that specific task.
• Fine-tuning is the process of training a base LLM with additional specific data to adjust its parameters for a particular application.
• Word embeddings are numerical representations of words that allow LLMs to understand context and relationships between terms with similar meanings.
• LLMs can perform diverse tasks such as summarizing articles, debugging code, translating languages, and drafting legal clauses.
• Pretraining is the phase where an LLM learns the bulk of its language structure and world knowledge by predicting the next token from massive internet text.
• Model weights are the billions of numerical parameters learned during training that encode the LLM's knowledge and capabilities.