Why Understanding These Mechanics Matters
You can use an AI tool without understanding how it works. Millions of people do every day. But understanding how it works—the levers you can pull, the constraints you are operating within, the reasons why it behaves the way it does—transforms you from a passive user into an intentional one. You move from "the AI gave me a bad answer" to "I need to adjust my approach to get a better answer." This chapter teaches you those levers and constraints.
These are not advanced topics. You do not need a PhD in machine learning to understand context windows and temperature. But they explain the most common frustrations people have with AI tools, and once you understand them, those frustrations become solvable problems.
Tokens: How AI Tools Count Words
What Is a Token?
A token is how AI tools count language. It is not exactly a word, though it is close. Some words are one token. Some are multiple tokens. Punctuation is a token. Spaces are tokens. Roughly, you can think of one token as about 0.75 words, or one token per four characters.
Why does this matter? Because every limit in AI tools is measured in tokens, not words. When ChatGPT says it has a 128,000-token context window, that translates to roughly 96,000 words. When Copilot limits you to 2,000 tokens per response, that is about 1,500 words. Understanding tokens helps you understand AI limits.
How Tokens Affect Your Experience
Every AI tool charges you in tokens if you are on a paid plan. More input tokens = more cost. Longer responses = more cost. This is why some platforms offer cheaper tiers: they limit you to fewer tokens per month or per day. It is also why the same conversation on ChatGPT and Claude might have different costs, even if you make similar requests: different models tokenize language slightly differently.
For your learning purposes right now, just keep tokens in mind. If an AI tool says you have exceeded context limits or hit rate limits, it is actually talking about tokens. The concepts you will learn in this chapter all relate back to tokens.
Context Windows: How Much the AI Remembers
What Is a Context Window?
A context window is the amount of conversation history an AI tool can "see" at once. Imagine you are having a conversation with a person who has a limited memory of recent events. They can remember what you said five minutes ago, but not what you said three months ago. An AI's context window is similar.
When you start a conversation with ChatGPT, the entire conversation (from the beginning) is in its context window. As the conversation gets longer, more of it fits in the window. But there is a limit. Once the conversation exceeds the context window size, the AI can no longer see the earliest messages.
Context Windows Across Platforms
ChatGPT (GPT-3.5): 4,096 tokens. This is about 3,000 words. In a typical conversation, this means you can go back about 20-30 exchanges before the AI starts forgetting the earliest messages.
ChatGPT (GPT-4): 128,000 tokens. This is about 96,000 words—the length of a small book. You can have very long conversations and the AI remembers everything you said.
Claude 3 Sonnet (free): 200,000 tokens. Even larger than GPT-4. Claude has prioritized large context windows because they enable more interesting capabilities.
Copilot: Around 4,000 tokens. Similar to GPT-3.5. Conversations have practical length limits.
Gemini: 1,000,000 tokens (as of 2026). Genuinely enormous. Google has prioritized context window size, letting you load entire documents, code repositories, or books into a single conversation.
How Context Windows Affect You
A small context window means you need to remind the AI of important context frequently. If you are working on a project over multiple days, a tool with a small context window will not remember your project details from yesterday. A large context window means you can have long, continuous conversations where the AI has full context.
This is why Claude is often preferred for long-form writing or complex projects: its large context window means the AI maintains continuity throughout. For quick, simple interactions, context window size matters less.
Many people assume that if they are paying for an AI tool, they are always working within the same context window. This is not true. By default, most platforms start a new conversation (and thus a new context window) each time you click "New Chat" or refresh the page. Within a single conversation, context accumulates. Across multiple conversations, the AI has no memory of the previous conversation unless you explicitly reference it.
Temperature: Controlling AI Creativity
What Is Temperature?
Temperature is a parameter that controls how creative or deterministic an AI is when generating responses. On a scale from 0 to 2 (in most tools):
- Temperature 0: Completely deterministic. The same prompt always produces the exact same response. This is useful for tasks where you want consistency above all else: writing code, performing mathematical calculations, following strict instructions.
- Temperature 0.5-0.7: Balanced. Somewhat predictable but with variation. Most AI tools default to something in this range. Useful for most general-purpose tasks.
- Temperature 1.0: Neutral high temperature. The AI is creative but still somewhat grounded. Good for writing, brainstorming, and creative tasks where you want variety.
- Temperature 1.5-2.0: Highly creative and random. The AI takes more risks and produces more unexpected outputs. Useful for brainstorming and creative exploration, but might produce less coherent results.
Temperature in Practice
Consider this prompt: "Write me a subject line for an email thanking a customer for their business."
At temperature 0, you get: "Thank you for your business." Same every time.
At temperature 0.7, you might get: "Thank You for Trusting Us with Your Business" or "Your Loyalty Means Everything" or "Gratitude for Your Continued Support." Variation, but all appropriate.
At temperature 1.5, you might get: "Infinite Gratitude, Infinite Growth" or "You Elevate Our Universe" or something more unusual. Creative, but might not fit your professional context.
Where to Adjust Temperature
ChatGPT: Click your profile icon, go to Settings, then click "Beta features" to enable the Developer mode. This gives you access to a temperature slider for adjusting model creativity.
Claude: Claude does not expose temperature controls in the web interface. You would need to use the Claude API if you want to adjust temperature.
Copilot: You can adjust conversation style with "Creative," "Balanced," or "Precise" buttons at the bottom. These are presets for different temperature ranges, not direct temperature adjustment.
Gemini: Temperature adjustment is not exposed in the free interface. You would need to use Gemini's API.
System Prompts: The Hidden Instructions
What Is a System Prompt?
A system prompt is invisible instructions that the platform gives to the AI before you even start your conversation. You cannot see your platform's system prompt, but it is there, shaping how the AI behaves. A system prompt might say something like: "You are a helpful assistant. Prioritize harmlessness. Refuse requests for illegal activities. Be honest about your limitations."
Different platforms have different system prompts. This is one reason why the same prompt produces different results on ChatGPT versus Claude. ChatGPT's system prompt emphasizes being helpful. Claude's system prompt emphasizes being harmless and honest (sometimes at the expense of being maximally helpful). These different priorities create different behaviors.
Custom System Prompts (When Available)
Some platforms let you write your own system prompt or custom instructions. This is powerful. If you tell ChatGPT "Always use a professional tone" or "Always explain things as if I am a beginner," you are essentially giving it a custom system prompt that shapes all your subsequent conversations.
ChatGPT: Go to Settings > Custom instructions and write instructions that apply to all your conversations. For example: "I am a product manager. Explain technical concepts in business terms. Focus on customer impact."
Claude: You can include system prompt-like instructions at the beginning of conversations. For example: "You are an expert copywriter. All responses should be casual and conversational. Avoid corporate jargon."
Conversation History: Building on Previous Messages
How Conversation History Works
As you have a conversation with an AI, each message you send and each response it provides becomes part of the conversation history. When you send the next message, the AI sees the entire history up to that point (within the context window limits). This allows the AI to build on previous messages, remember what you said earlier, and maintain context.
This is more nuanced than it seems. The AI does not "understand" conversation history the way humans do. It sees it as tokens in its context window. It recognizes patterns: if you asked a question, then corrected yourself, the AI generally figures that out. But it can also miss context or misinterpret what you meant.
Best Practices for Using History
Start conversations fresh when context has changed. If you are working on Project A, have a conversation with the AI. Then switch to Project B. Instead of continuing the same conversation and reminding the AI about Project B, start a new conversation. A fresh conversation is cleaner and avoids confusion.
Be explicit when correcting the AI. If the AI misunderstood, do not just say "you got it wrong." Say "I was unclear. What I meant was..." This makes the correction explicit and helps the AI understand the context.
Refer back to previous messages when useful. You can ask the AI "Remember what I said earlier about..." or "Based on the document I pasted three messages ago..." The AI will reference the history to answer you.
Clear conversations regularly. If a platform stores your conversation history and you are concerned about privacy, periodically delete conversations you no longer need. Most platforms let you delete conversations individually or in bulk.
Conversation Branching: Exploring Multiple Paths
What Is Conversation Branching?
Conversation branching is a feature some platforms offer that lets you "branch off" from a previous message, explore a different direction, and then return to the main conversation thread. Instead of a single linear conversation, you can have multiple branches exploring different ideas.
Branching Across Platforms
ChatGPT: Supports branching. Hover over any AI response and you see a button to "Show variations" or "Regenerate response." This creates a branch. You can edit previous messages in the conversation, and the conversation regenerates from that point onward.
Claude: Limited branching support in the web interface. You can regenerate responses, but editing previous messages is not supported.
Copilot: Limited branching. You can ask follow-up questions but cannot edit previous messages.
Gemini: Supports editing previous messages and regenerating responses from that point forward, similar to ChatGPT.
Why Branching Matters
Branching is useful for exploration. If an AI gives you an answer but you want to see how it would approach the problem differently, you can branch and ask "What if I approached this from a marketing angle instead?" without losing the original technical approach. You can compare different approaches side by side.
File Uploads: Giving AI Your Documents
File Upload Basics
All four major platforms support uploading files. You can paste or upload documents, code, PDFs, images, or data, and the AI analyzes them. This is incredibly powerful: instead of copying and pasting a long document, you upload it. Instead of describing a spreadsheet, you upload it. The AI reads the file and works with it.
File Upload Features
ChatGPT (Plus): Supports text files, PDFs, images, and code. You can upload a file, and GPT-4 reads and analyzes it. File size limits exist but are generous (around 20MB per file).
Claude: Supports text, PDFs, images, and other formats. Claude's large context window means it can handle very long documents. You can paste or upload files directly.
Copilot: Limited file upload support. You can reference documents in your recent browsing, but direct uploads are not available in the free version.
Gemini: Supports uploading images and documents. Within Gmail and Docs, you can reference content directly. On the web interface, file uploads are available but somewhat limited.
Common File Upload Use Cases
- Document summarization. Upload a long report or article, and ask the AI to summarize it or extract key points.
- Code review. Paste or upload code, and ask the AI to review it, suggest improvements, or explain what it does.
- Data analysis. Upload a CSV or spreadsheet, and ask the AI to analyze trends, identify patterns, or answer questions about the data.
- Image analysis. Upload an image, and ask the AI to describe it, extract text, or answer questions about its content.
- Comparative analysis. Upload multiple documents and ask the AI to compare them, highlight differences, or synthesize information.
Why the Same Prompt Gives Different Results
Sources of Variation
Different underlying models. ChatGPT (GPT-4) is trained by OpenAI. Claude is trained by Anthropic. Copilot is Microsoft's integration of OpenAI's technology. Gemini is trained by Google. Different training data, different training priorities, and different architectures create fundamentally different responses.
Different default temperature. Each platform has a default temperature setting. Some are more creative by default; others are more conservative. This affects variation in responses.
Different system prompts. Each platform has different invisible instructions shaping behavior. ChatGPT's system prompt is different from Claude's, which is different from Gemini's.
Different context window sizes. A platform with a small context window might not see all your previous messages. A platform with a large context window sees more context and makes better decisions.
Random variation within the model. Even on the same platform, the same prompt might produce slightly different responses. This is inherent to how language models work. They are probabilistic systems, not deterministic. Even at low temperature, there is some randomness.
Leveraging These Differences
Instead of seeing these differences as frustrating, see them as features. Different AI tools have different strengths. When you need a creative solution, use the tool that defaults to higher temperature. When you need code review, use the tool known for being most careful about correctness. When you need web search, use Copilot or Gemini which have built-in search. When you need the largest context window, use Gemini or Claude.
Practical Implications: How to Use This Knowledge
For long-form work: Use Claude or Gemini because of their large context windows. You can paste entire documents and the AI remembers the full context.
For coding tasks: Use ChatGPT (GPT-4) or Claude because both excel at code. Consider lowering temperature for code (you want predictability).
For research tasks requiring current information: Use Copilot or Gemini because they have web search built in. ChatGPT free tier does not have search (GPT-4 Plus does).
For creative brainstorming: Use higher temperature (if accessible) or Claude because it is known for creative thinking. Try different tools to see which you prefer.
For tasks where you care about privacy: Use Claude because Anthropic does not use free-tier conversations for model training.
Key Takeaway
AI tools are not magic black boxes. They have mechanics: context windows that limit memory, temperature that controls creativity, system prompts that shape behavior, conversation history that provides context, and file upload capabilities that expand what you can work with. Understanding these mechanics lets you use AI tools more intentionally. You become not just a user, but someone who understands how the tool works and can adjust your approach to get better results.
What Comes Next
Now that you understand how AI tools work mechanically, Chapter 3.3 puts theory into practice. You will complete hands-on exercises that reveal AI capabilities and limitations through direct experience. You will draft emails, summarize documents, analyze data, brainstorm solutions, and conduct research. You will see these concepts from Chapter 3.2 in action and develop the practical intuition that only comes from doing.