What Are Tokens? The Hidden Meter Behind Every AI Bill
Why did our AI bill triple last month?This is the question almost every team using AI runs into eventually. Nothing big launched. No spike in users. No new features. The numbers just crept up week after week, and then finance came knocking with a printout and a frown.When you go look at the usage…
Why did our AI bill triple last month?This is the question almost every team using AI runs into eventually. Nothing big launched. No spike in users. No new features. The numbers just crept up week after week, and then finance came knocking with a printout and a frown.When you go look at the usage dashboard, the first column does not say dollars. It says tokens. And most people staring at that column have no idea what a token actually is, or why they are paying for so many of them.So let me write it down in plain words, with the math you can actually use.What a token actually isWhen you send a message to an AI, the AI does not read your sentence the way you wrote it. Before anything happens, your text gets chopped into small pieces. Each piece is called a token.A token is sometimes a whole word. Sometimes it is part of a word. Sometimes it is just a punctuation mark. The exact split depends on how common the word is and how the model was trained.Here is the rule of thumb that matters in real life:Or to flip it the other way, 100 words is roughly 130 tokens. A page of writing, around 500 words, is roughly 650 to 700 tokens. A million tokens is around 750,000 words, which is about five normal books.So when you type one sentence, you might think you are sending five words. The AI sees ten or twelve tokens, and that is what gets billed.Input tokens and output tokens are not the same priceThis is the part most people miss until they get a surprise bill.Every request has two parts:Input tokens. Everything you send into the model. Your question, your instructions, any documents you paste in, the conversation history, the system prompt the application has hidden in the background.Output tokens. Everything the model writes back. The answer, the analysis, the code, the long essay you asked for.Here is the kicker. Output tokens are usually five times more expensive than input tokens. This holds across every current Claude model. So a long answer costs you a lot more than a long question, for the same number of words.This is why asking an AI to write a 30-page report is much more expensive than asking it to read a 30-page report. The reading happens at the cheap rate. The writing happens at the expensive rate.What it actually costs todayPrices change, but here is what the current generation of Claude models cost as of May 2026. The number on the left is the input rate. The number on the right is the output rate. Both are per million tokens.Two things to notice. First, the output rate is always five times the input rate. Second, the gap between the cheapest model and the most expensive is about 5x. So if you pick the wrong model for the wrong job, you can quietly pay five times more than you needed to.Three real examples with real numbersConcept is fine, but numbers stick. Here are three workloads people actually run, with rough costs using Sonnet 4.6 prices ($3 in, $15 out per million tokens). All numbers are approximations to help you reason about the order of magnitude, not exact quotes.Example 1: Summarizing a 50-page contract50 pages of contract is roughly 25,000 words, which is around 33,000 tokens of input.The summary you want back is maybe 500 words, around 700 tokens of output.Input cost: 33,000 ÷ 1,000,000 × $3 = about 10 centsOutput cost: 700 ÷ 1,000,000 × $15 = about 1 centTotal: about 11 cents per contract.Run that on 10,000 contracts in a quarter, and you are at $1,100. Still cheap. But notice that 90% of the cost is the reading, not the writing, because the input is so much larger than the output.Example 2: A customer support chatbot conversationA typical support chat is around 10 back-and-forth turns.Each turn, the system sends in roughly 2,000 tokens (the system prompt, instructions, the conversation history so far, the new user message). By turn 10, the input is bigger because the history is bigger.Total input across the conversation: roughly 20,000 tokens.The bot writes maybe 100 words per reply, so 10 replies is around 1,300 tokens of output.Input cost: 20,000 ÷ 1,000,000 × $3 = about 6 centsOutput cost: 1,300 ÷ 1,000,000 × $15 = about 2 centsTotal: about 8 cents per conversation.That sounds tiny. Multiply by 100,000 conversations a month, and you are at $8,000. This is the kind of math that creeps up on a finance team.Example 3: A coding agent doing real workA coding agent on a real task might run for an hour, making 50 to 100 tool calls along the way.Each tool call sends back the same growing conversation plus the new tool result. Input can easily reach 500,000 tokens or more across the session.Output across the whole session, including all the thinking and the code it writes, might be 30,000 tokens.Input cost: 500,000 ÷ 1,000,000 × $3 = about $1.50Output cost: 30,000 ÷ 1,000,000 × $15 = about 45 centsTotal: roughly $2 per coding session.Run a coding agent on a team of 20 engineers, 5 sessions a day, 20 days a month, and you are at $4,000 a month. That is real money, and it is also why teams obsess over context.The pattern across all three: input volume tends to be much larger than output volume, especially in long conversations and agent workflows. So even though output is 5x more expensive per token, your bill is often driven by input, because there is so much more of it.Where the input gets so bigMost people think of input as just their question. In reality, every request to an AI carries a lot of invisible passengers along with the question.This is also why long conversations get expensive even when you only type short messages at the end. Every new message sends the whole history back in. Turn 20 costs more than turn 2, because by turn 20 the input includes the previous 19 turns of conversation.The five levers that actually move the billOnce you understand input and output, you can pull these levers in order.Pick the right model for the job. Do not use the most expensive model for tasks the cheapest model can handle. Sorting, tagging, simple replies, and basic extraction all work fine on the cheap tier. Save the expensive tier for hard reasoning.Cut the input. Trim long system prompts, stop pasting entire documents when only one section is needed, and prune conversation history. Most teams discover their input is two to three times bigger than it needs to be.Cap the output. Tell the model how long the answer should be. A request for a 200-word summary will cost far less than letting the model write whatever length it wants, which often ends up being three pages.Use prompt caching. If the same chunk of input shows up over and over, like a long system prompt or a reference document, modern AI APIs let you cache it. Cached input can be up to 90% cheaper than fresh input. This is one of the biggest cost cuts in production.Use batch processing for non-urgent work. If you do not need an answer in the next second, batching cuts the price in half. Good for nightly jobs, bulk analysis, document processing pipelines.A team that pulls all five of these levers can often cut their AI bill by 70 to 90% without losing quality.The shift this representsFor a long time, software costs were predictable. A user logs in, a server handles it, the bill comes at the end of the month. AI flipped that. Now the cost depends on how much text flows in and out, which depends on what the user asks for, which depends on the conversation, which depends on the prompts your team wrote.This is uncomfortable for finance teams, because the cost is shaped by engineering choices that nobody is auditing yet. The teams that win at AI cost are the ones who treat tokens like an engineering metric. They watch them. They optimize them. They build dashboards around them.So the next time you see your AI bill, do not just look at the total. Look at the tokens. Look at the split between input and output. Look at which features are eating the most.That is where the money actually goes.This story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories.Subscribe to our newsletter and YouTube channel to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!What Are Tokens? The Hidden Meter Behind Every AI Bill was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.Source: Generative AI Pub — Published — Category: Image AI