The Bot Forgot Her Latte. Here Is the Clipboard Fix.

Every API call hires a new intern. messages[] is the clipboard you hand over. Twenty turns of cafe chat move 2,420 tokens, not 216.Nothing was forgotten. The second call never received the first.Four seconds after Amina told the bot her drink order, it had no idea who she was.I train GenAI cohorts…

Every API call hires a new intern. messages[] is the clipboard you hand over. Twenty turns of cafe chat move 2,420 tokens, not 216.Nothing was forgotten. The second call never received the first.Four seconds after Amina told the bot her drink order, it had no idea who she was.I train GenAI cohorts for a living, about 100 students at a time. This is the café role-play I run whenever someone tells me a model “has memory.”Turn 1. Amina, playing the guest, types: “I like iced lattes.” The bot replies: “Noted: iced latte.”Turn 2. Amina types: “What did I like?”The bot says it does not know.Someone in the back said the model was broken. It was not. The second request never contained the first.We had just watched this same model draw purple tickets out of a hat. It sampled beautifully. It could not hold a drink order for four seconds.The answer (read this first)Forget the word “memory” for ten seconds. Picture a café where a brand new intern walks in for every single question. All they know is what is written on the clipboard you hand them.The browser lab uses the same three lines as every table below. No API key.Three lines. That is the entire memory.The lab freezes one café exchange so you can add up every number by hand:Role | Text | Chars | ÷ 4-----------|---------------------------------------------------|-------|-----system | You are a café helper. Remember the guest name if | 63 | 16 | they give it. | |user | I like iced lattes. | 19 | 5assistant | Noted: iced latte. | 18 | 5-----------|---------------------------------------------------|-------|----- Total | 100 | 26Add the characters: 63 + 19 + 18 = 100. That is the whole conversation so far.English averages about four characters per token, so we divide by four. The clipboard costs about 26 tokens.Why 26 and not 25? Because we round each message up on its own (16 + 5 + 5), not the total. Chat APIs tokenize each message separately and add overhead around it, so per-message rounding lands closer. When the number has to be exact, use tiktoken.The whole memory is three lines you resend.63 + 19 + 18 = 100 characters. Round each message up: 16 + 5 + 5 = 26 tokens.Nothing was forgotten. Nothing was ever stored.When you call OpenAI, Gemini, or Anthropic, you are not opening a session. The request arrives, the model replies, and the server keeps nothing.There is no forgetting. There is only what you sent.That last row is where the confusion starts. The product does exactly what your code must do: keep a list, post all of it again.Turn 2, two payloads, two answersTurn 2, the way most beginners send it:messages = [ {"role": "user", "content": "What did I like?"},]Reply: “I don’t know what you like.” Which is the correct answer. You asked a stranger.Turn 2 with the clipboard attached:messages = [ {"role": "system", "content": "You are a café helper. Remember the guest name if they give it."}, {"role": "user", "content": "I like iced lattes."}, {"role": "assistant", "content": "Noted: iced latte."}, {"role": "user", "content": "What did I like?"},]Reply: “An iced latte.”Same model. Same temperature. Different payload. Different answer.Three roles, three jobs, three different failures when you drop one:Those role names are not a LangChain invention. They are the standard chat format open models are trained against too. LangChain’s ChatMessageHistory is just this list, stored between clicks.Turn 2 sent twice. Only one of them carries the latte.Left: one line, no clipboard, no answer. Right: system plus the pair plus the question, and the drink comes back.The bill is a staircase, not a lineYou resend the clipboard every call, so the clipboard is the invoice. Count one turn as one round trip: the question you send plus the reply that comes back.System prompt: 16 tokens, on the clipboard for every single turn.One turn (question + reply): 5 + 5 = 10 tokens, and it never leaves the clipboard.So turn n moves:tokens(turn n) = 16 + 10nturn 1 → 16 + 10 = 26turn 5 → 16 + 50 = 66turn 10 → 16 + 100 = 116turn 20 → 16 + 200 = 216Turn 20 moves more than eight times what turn 1 moved, and the guest is still talking about coffee.The real number is the total across the conversation, because you paid for every step on the way up:20 turns × 16 (system) = 32010 × (1 + 2 + ... + 20) = 2,100 ------total tokens moved = 2,420That middle line is the triangle: 1 + 2 + … + 20 = 210, times 10 tokens per turn, equals 2,100.A twenty-turn café chat moves 2,420 tokens, not 216. You paid for the transcript about eleven times over. Forty turns reach 8,840, roughly 3.7 times the twenty-turn bill, because the history term grows with the square of the turn count.(Token volume, not dollars. Providers price what you send and what comes back at different rates.)Every turn resends everything before it.Turn 20 moves 216 tokens. The twenty turns together moved 2,420, because 1 + 2 + … + 20 = 210 turns of recent history.Sliding window: pin the system prompt, drop the middleYou cannot resend forever. Context windows end, and the bill grows faster than the conversation.The first fix students build is a sliding window: keep the system prompt plus the last N pairs, throw the rest away.def trim(messages, max_pairs=6): # 👇 The system prompt is pinned. It is the one line you almost never drop. system = [m for m in messages if m["role"] == "system"] # 👇 Keep the last N pairs. Each pair is one user message + one assistant reply. rest = [m for m in messages if m["role"] != "system"] return system + rest[-max_pairs * 2:]With a six-pair window, the café bill stops climbing at turn 6 and goes flat at 16 + 60 = 76 tokens per turn. Over the same twenty turns, that is 1,370 tokens instead of 2,420, a saving of about 43%.Here is the trade: at turn 7, the iced latte falls off the clipboard. Ask “what did I like?” at turn 12, and the bot has no idea again. You did not buy memory. You bought a budget.Summarization is the next step up. One extra model call compresses the old turns into a paragraph you keep on the clipboard. Same principle: context is finite, and you are the librarian.The window flattens the bill at 76 tokens and costs you the latte.System pinned. Last six pairs kept. 1,370 tokens instead of 2,420, and the drink ages out at turn 7.The clipboard is not your handbookFix the message list, and Amina gets her latte back. Now ship that bot to a gym, where a member asks: “How long do I have to refund a class pack?”The clipboard holds her drink. It does not hold your refund policy, because nobody typed it into the chat.Chat memory and document retrieval solve different problems. Confusing them is how a team ships a bot that remembers your coffee order and invents your HR policy. That is the failure in Tiny RAG on a gym handbook, where the model offered a 30-day refund window that does not exist.And do not paste the handbook into the system prompt “just in case.” Re-read the staircase. You would resend all of it, every turn, forever.The Python version (same lab math)import mathSYSTEM = "You are a café helper. Remember the guest name if they give it."DRINK_USER = "I like iced lattes."DRINK_ASST = "Noted: iced latte."def tokens(text: str) -> int: # 👇 Classroom estimate. Round up per message, the way chat APIs count them. return math.ceil(len(text) / 4)messages = [ {"role": "system", "content": SYSTEM}, {"role": "user", "content": DRINK_USER}, {"role": "assistant", "content": DRINK_ASST},]print("chars on the clipboard:", sum(len(m["content"]) for m in messages))print("turn 1 clipboard:", sum(tokens(m["content"]) for m in messages))S = tokens(SYSTEM) # 16, on every clipboardP = tokens(DRINK_USER) + tokens(DRINK_ASST) # 10 per turn, never leavesmoved = 0for n in range(1, 21): # 👇 Every turn moves the whole clipboard again: what you send plus the reply. moved += S + n * Pprint("turn 20 moves:", S + 20 * P)print("moved over 20 turns:", moved)windowed = 0for n in range(1, 21): # 👇 Sliding window: the clipboard stops growing after 6 pairs. windowed += S + min(n, 6) * Pprint("moved with a 6-pair window:", windowed)Output, and every number matches the tables above:chars on the clipboard: 100turn 1 clipboard: 26turn 20 moves: 216moved over 20 turns: 2420moved with a 6-pair window: 1370The OpenAI text generation guide says it in one sentence: the model has no memory of previous requests, so you include the history yourself.Try it live (no API key)Chatbots Forget: Message List MemorySame three lines. Append turns. Watch the token meter. Trim on purpose and break the recall yourself.What this means before you ship anythingEvery API call is a new intern. Resend the clipboard or the model knows nothing.messages[] is the memory. Your app owns it. Session state, Redis, a JSON file: all just places to keep the list alive.The bill is a staircase. 16 + 10n per turn, 2,420 tokens across twenty café turns. The history term grows with the square of the turn count.A window buys budget, not memory. Six pairs cut the bill 43% and lost the latte at turn 7. Pin the system prompt.Chat history is not your document library. The clipboard only holds what somebody typed.If you came from the sampling article, you know the model draws one ticket at a time. It also reads one clipboard at a time.Go deeperSession 07 and Session 10 NB11: chatbot memoryOpenAI: managing conversation stateTiny RAG: the 30-day lieWhat comes nextYou can now explain it in one breath. Each call hires a new intern; the clipboard ismessages[], and you resend all of it every time. Trim the middle, pin the system prompt, accept what falls off.Still missing: the clipboard cannot hold a 200-page policy PDF, so we retrieve instead. But real PDFs are not ten neat sentences. Cut one wrong and “fourteen days” lands on a boundary nobody retrieves. That is the chunking article next.Which one bit you harder: a bot that forgot the drink, or a bill that quietly grew eleven times larger than you expected? I read every comment.I’m Mohamed Noordeen, a GenAI trainer building the zero-to-genai-engineer curriculum: TF-IDF search to production RAG and LangGraph agents. No GPU required.Series: TF-IDF · Self-attention · Tiny RAG · BPEThis story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories.Subscribe to our newsletter and YouTube channel to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!The Bot Forgot Her Latte. Here Is the Clipboard Fix. was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: Generative AI Pub — Published August 28, 2026 at 14:14 UTC — Category: Image AI