LLM Fundamentals Common Mistakes

What we verified before publishing

AI-generated content may contain mistakes. Check the sources below before relying on important information.

The following claims were verified through Perplexity Sonar before the questions were finalised. The host can reference any of them during play:

• LLMs operate as advanced statistical engines that match input queries to learned textual patterns rather than possessing conceptual understanding.
• A hallucination occurs when an LLM generates text that sounds accurate but is entirely false due to missing information.
• Bias and hallucination issues cause models to reflect social biases or create confident but false details like made-up facts.
• Retrieval-Augmented Generation (RAG) requires chunking, embeddings, recall, re-ranking, and guardrails instead of simply attaching a PDF.
• Overfitting and underfitting are critical training errors that occur when models fail to generalize or learn training data effectively.
• Context windows represent a limited memory span for AI, creating computational costs and performance degradation with large sizes.
• Successful LLM applications require continuous evaluation systems to test accuracy, bias, safety, and edge case handling across diverse scenarios.
• LLMs are inherently probabilistic and introduce controlled randomness during generation rather than functioning as deterministic engines.
• Grounding responses in traceable data and including human oversight are essential to prevent reliance on inaccurate LLM outputs.
• Learning token economics and attention mechanisms is more fundamental than starting with prompt engineering for effective LLM development.