I have used AI tools in my practice every working day for eighteen months and built a legal intelligence platform on top of them. There is one mechanical fact about how these models work that governs cost, reliability and confidentiality in all of it, and almost nobody outside engineering ever hears it stated plainly. This is that fact, and how my practice is built around it.
The call
A model has no memory between messages. Every reply is a fresh call that receives one package of text and returns one block of text. What goes into the package on every message you send:
- The system prompt. Instructions from the product, your saved preferences, tool definitions. Fixed size per session.
- The chat so far. In an ordinary chat window that is every message, every reply, every pasted file and every tool result, re-sent in full. Some products select or compress the history before sending it, but the model sees only what the product puts in the package, never more.
- Your new message.
The model reads the whole package from the top, then writes the reply. It does not know it answered your last question. It reads its own earlier answer in the package the same way it reads yours. A chat is not a conversation with something that remembers. It is a transcript re-read on each turn.
In practice this is why a question like "what did we decide on the indemnity language" works in the long chat where you were drafting a form agreement and fails in a new one. The decision is in the package the first time and absent the second. Nothing was learned. The transcript was either there or it was not.
Tokens
A token is the unit the model reads and writes in, roughly three quarters of an English word. 1,000 tokens is about 750 words. A 20 page commercial lease is around 15,000 tokens. A full discovery production is hundreds of thousands.
The meter counts two things on every call: input tokens, which is the whole package, and output tokens, which is the reply. Output is priced several times higher per token, but in a long chat the input dwarfs it, because the history is re-sent in full each time.
Subscription plans do not show you the token count. They convert it into a usage allowance that drains with every call. If I paste one of my own form leases into a chat to work on the assignment clause, that lease rides on every question I ask afterward in that chat, whether or not the next question is about the lease. The allowance pays for it each time.
The context window
The context window is the maximum size of the package: system prompt, history, your message and the reply together. Current frontier models run about 200,000 tokens, some to a million. That is the hard ceiling on what the model can see at once.
When a chat approaches the ceiling, the product has three choices: refuse the next message, drop the oldest history, or compact it, replacing earlier exchanges with a shorter summary. Compaction keeps the gist and loses the detail. Either way the model no longer has the start of the thread as it was written. A fact established on day one may survive as a line in a summary, or not at all.
Quality falls before the ceiling. A model reading 150,000 tokens of history finds the one relevant line less reliably than a model reading 5,000. The answer gets vaguer as the window fills, independent of cost.
This is the drift every heavy user has felt. Early in a working chat you tell the model how you want business correspondence written: no preamble, no "we", plain register. Forty turns later the drafts come back with preambles. The instruction did not stop mattering. It either left the window or got buried under enough material that the model stopped weighting it. Arguing with the model in that chat spends more of the window. Restating the rule in a fresh one is the cheaper fix.
Why long chats burn the week
The cost of a message is the size of the whole package, not the size of the message. A ten word question at the end of a 150,000 token chat costs 150,000 input tokens. The same question in a fresh chat costs the system prompt plus ten words. Over a session that compounds. Twenty turns in one chat cost about twice what the same twenty turns cost split across two chats, all else equal, and the gap widens the longer the single chat runs.
A working chat carried for weeks, with templates and outputs pasted in along the way, means every question in the fourth week pays for everything from the first. That is how a weekly allowance on a top tier plan can be gone by Tuesday while the plan looks undersized. Pasted documents stay in the active context long after the question that needed them, a heavier model multiplies the same count by a higher rate, and an agentic coding session makes hundreds of calls on one growing log. Providers cache the repeated prefix and charge less for it, which softens the curve but does not flatten it. The lever you control is the size of the package.
Memory, files, tools and agents
None of these change the basic rule. They are all ways of putting text into the package or taking text out of it. Memory is a few pages the product writes after a chat and pastes into the next one. An uploaded document, depending on the product, is either inserted whole, so that a 300 page PDF rides on every later turn, or searched, with only the excerpts the search returns entering the package. A tool call is a request written as text and the result pasted back. An agent is that loop run many times unattended, its history growing with every step. A subagent is a separate call with a fresh package that reports back a summary, which is how long work is kept cheap.
FLIS, the platform I built for my practice, exists because of this. A public chat window is not where client material goes, full stop. Chat memory holds a few pages about how I work, and that is all it should hold. The matter record lives in a system I control, and when a model is used on that record it is through the platform, on terms and controls built for the purpose, receiving only the slice a given task needs. Each call is small, current and scoped.
The same principle moves work between tools. When my Claude allowance ran out mid build last week, a different coding agent picked up from written specs, a results file and a hold note, not from any chat transcript. The transcript was never the record. The record was on disk. For a lawyer that pattern should feel familiar. It is a file memo.
What this changes
- One chat per job, not one chat per project. Cut over when the thread's history stops helping the next question. A few days of work is the ceiling, not a few weeks.
- Carry forward a file memo, not a transcript. Open items, decisions, a short catch-up. The old chat stays searchable; it does not need to ride along.
- Put a document in only when the turn needs it, in a chat you will close. A form pasted for one question costs on every question after it.
- Match the model to the task. Drafting, lists and billing entries do not need the heaviest model. Save it for analysis and long reads.
- Expect drift in a long thread and do not fight it. When an instruction stops landing, assume it left the window or got buried, and restate it in a fresh chat.
- Keep the matter record outside the chat. Memory and preferences are for how you work. The facts of a matter belong in a system you control, fed to the model one slice at a time.
And one for confidentiality, which the mechanics inform but do not settle. Client material stays out of general-purpose consumer chats. Part of the reason is what the package shows: whatever is in it is part of every call, not just the first, and it is held under the terms and settings governing that service. The rest is the ordinary work of vendor terms, retention, access and configuration, which no architecture choice makes unnecessary. Where a model touches client material at all, it does so through tooling where those terms are set for the purpose. That is what the platform is for.
This article is for general information and is not legal advice. Reading it does not create an attorney-client relationship.