Tokens, Context, and Memory
Understand why AI forgets, truncates, or misses details.
What you will learn
- Explain tokens, context, and memory in clear language.
- Recognise where this concept appears in real work.
- Apply the idea and check the result using a practical activity.
The concept
Language models process text as tokens within a context window. They do not automatically remember everything forever. Product memory usually comes from saved state, retrieval, summaries, or external databases.
In plain English
AI has a backpack for information. If the backpack gets too full, some things get left out unless the app stores them somewhere else.
Work example
A support assistant should retrieve account policy and ticket history instead of relying on a long pasted chat.
Try it yourself
Activity
List what an AI assistant would need to remember for your role, then mark what should be stored permanently.
Knowledge check
What is the most accurate description of LLM memory in products?
- A.The model remembers every user forever by default
- B.Memory is usually built with product systems around the model
- C.Memory only works for code
- D.Context windows are unlimited
Show answer
B. Memory is usually built with product systems around the model
Persistent memory is an application design choice, not magic inside every model call.