Prompt Engineering for

Geniuses
 What is prompt engineering all about?

• Based on transformer
architecture from NLP

• Pays attention to different

parts of the input text,
capturing relationships
between words and phrases
• They can remember
information from earlier in
the prompt
Same Diagram in Plain English
Why do we care about attention?
 Tokenization and Context

LLMs don't read raw letters; they break input into tokens (word pieces or
characters).

For example, the sentence “LLMs are amazing” might be split into tokens like:

[“LL”, “M”, “are”, “amazing”]

Each token is a number that the model processes

The model has a context window, which is like its short-term memory
 Context Window

You can think of the context window as the span of text the model “sees” when
generating a response.

Larger Context Window Smaller Context Window

Can process entire documents Might not “remember” the

or long conversations beginning of long prompts

The context window is measured in tokens (not words)

If your input exceeds the context window, the model can't use the overflow
to inform its answer
 How LLMs Work
You can think of the context window as the span of text the model “sees” when
generating a response.
If you prompt it with the “The capital of France is”
It will likely complete with “Paris”

Under the hood, the model represents words as vectors (lots of numbers) that
capture meaning

close far
cat dog banana

In short, an LLM is a predictive text engine on steroids, guessing the most

probable continuation of your input
 Why Prompt Engineering Matters
Because LLMs will answer something no matter what, the way you ask matters a
lot
These models don't really think or know our goals -- they rely on cues in our
prompt

Poorly Worded Prompt Well-Structured Prompt

Confuses the AI or leads to Sets the model up for

irrelevant output success

We'll cover techniques on how to help the models reason through

complex tasks, retrieve facts accurately, and then critique its own
answers
Prompt Engineering
Fundamentals
 Zero-shot Prompting (just ask)
What it is:
Zero-shot prompting means giving the model a prompt without any examples
or prior demonstrations.

How to use it:

Simply state your request or question clearly in plain language.

“Explain why the sky is blue in one paragraph”

Model relies on internal knowledge with no examples given

When to use:

• Quick queries or well-known tasks

• Straightforward or generic instructions
• When you’re exploring what the model can do
 Zero-shot Prompting (just ask)
Example:

User prompt:
User “Translate the following to French”
Good morning, how are you?

Model response:
AI
“Bonjour, comment allez-vous?”

No examples were given; the model uses its pretrained knowledge

 Few- Shot Prompting (Providing Examples)
What it is:
Few-shot prompting means you include a few examples of the task in your
prompt to show the model how what you expect

“Here's how I want it done in similar cases. Now do it for this new case”

This leverages the model's ability to learn patterns in the moment from the
prompt itself

(this is called in-context learning)

 How to Use Few-Shot Prompting
1. Start with 1-5 Relevant Examples
2. Show input → output pattern

Input 1: [Example Input] → Output 1: [Example output]

Input 2: [Example Input] → Output 2: [Example output]

Input: [Your actual question] → Output 2: ???

3. Model infers the pattern and applies it to your input

 When to Use Few-Shot Prompting
Few-shot prompting is useful when you need:
• A specific format or style
• To handle unusual tasks the model might misinterpret
• To improve results when zero-shot performs poorly

Great for:
Specific text
Custom Classification Following style guides
transformations

Specific output format Consistent Tone/Voice

Remember: Examples use up context window space

Don’t overload with too many examples
 Few-Shot Example
Converting movie names to “superhero names”

Input: The Godfather

Superhero name: The Shadow Don

Input: Jurassic Park

Superhero name: The Prehistoric Garden

Input: The Matrix

Superhero name: The Digital Liberator

Input: Titanic
Superhero name: [the model will generate a superhero name]

Model learns the pattern from examples and applies it

 Iterative Prompting and Re-asking
What it is:
Even humans often don't ask a perfect question on the first try

Iterative prompting means that you follow up on the model's response with
another prompt, essentially having a back-and-forth to clarify or approve the
result.

Initial question

Initial response

Follow-up question
 How to Use Iterative Prompting
Treat the interaction as a conversation

That's not what I expected, Dash. Can you consider XYZ and try again

For redirecting when the answer is off-track

Please correct any errors in your last answer

For prompting self-correction

Double-check the calculation above step-by-step and correct any mistakes

For verifying detailed work

The model can often catch its own mistakes when prompted to review or
when asked in a clarified manner
 When to Use Iterative Prompting
Use iterative prompting whenever the first output isn’t satisfactory

• When the answer has a minor error

• Ask it to fix that specific part

• When the answer is too general

• Ask for more details or a specific format

• When you suspect hallucination

• Ask “Can you verify that answer?” Is it correct?”

Don't be afraid to prompt again. Each prompt can give the model more
information about what you want and what you're looking for
 Iterative Prompt Example

User What’s 53,467 * 76,935?

AI 4,114,576.045

User Is that answer correct? If not, why?

AI
I made a mistake. Let me recalc.
The correct answer is 4,113,520,845
Iterative prompting turns a single-shot Q&A into an interactive session
that improves results
Intermediate Prompting
Techniques
 Chain-of-Thought Prompting
What it is:
Chain of thought prompting is a technique where you prompt the model to
produce its reasoning process step-by-step rather than spitting out the final
answer

Chain-of-Thought Output
Traditional Output “First, I’ll calculate X…”
“The answer is 42” “Then, I need to consider Y…”
“Therefore, the answer is 42.”

In essence, you encourage the AI to “show its work”

 Chain-of-Thought Prompting
Two approaches to invoke Chain-of-Thought:

1. Few-Shot CoT
Include examples with the reasoning steps in your prompts
Q: If John has five apples and needs to go, how many remain?
A: John starts with five apples. He eats two apples. Five minus two equals three,
so three remain.

2. Zero-Shot CoT
Simply add a prompt to think step-by-step
Think step-by-step
Let's reason this out one step at a time
 Why and When to Use CoT
Chain-of-thought is extremely useful for:

• Complex reasoning tasks: multi-step math problems, logical puzzles, combining

multiple facts.

• Error reduction: The model has to justify its answer, making it more likely to catch
contradictions.

• Increased transparency: You see why and how the model comes up with the answers
it does

CoT is basically baked into reasoning models, so you want to use this
when you need a couple of extra steps of double-checking criteria or
ensuring you're respecting, let's say, a word count, which is impossible to
do when you have a non-reasoning model.
 Chain-of-Thought Example
Prompt to the COT trigger:
A farmer has 17 cows, all but five (12) and all but nine (8) die. How many are left?

Model response with CoT:

Let me think through this step-by-step:

1. The farmer starts with 17 cows

2. All but nine die means that nine cows are still alive
3. If nine cows are still alive then 17 minus 9 equals 8 cows died
Therefore, 9 cows are left

Without CoT, a model might misinterpret this tricky question and

answer “8 cows” (incorrectly subtracting 9 from 17)
 Scaffolding
What it is:
Scaffolding is a prompt design pattern where you break a complex task into
smaller or manageable steps, prompting the model through each step

Single Prompt Scaffolded Approach

“Write a full essay about climate 1. Create an outline
change” 2. Write introduction
3. Write each section
Potential for wondering 4. Create conclusion

Like construction scaffolding - providing support structure

 How to Use Scaffolding
Two approaches to scaffold your prompts:

1. Multi-Prompt Scaffolding
Break the task into a series of sequential prompts
Prompt 1: Give me an outline for an essay about climate change.
Prompt 2: Now write the first section about impacts on agriculture.
Prompt 3: Now write the section about impacts on oceans

2. Single Prompt Scaffolding

Guide the model with a structured format in one prompt
First, list key points about climate change. Then, for each point provide a
detailed explanation with evidence. Finally, conclude with recommendations
for addressing climate change
 Why and When to Use Scaffolding
You use scaffolding when a task is too complex to expect a good answer in
one shot

Ideal for:

• Long-form content (essays, reports, stories)

• Multi-part analyses, or explanations
• Content that benefit from a clear structure or sequence

The benefits of using scaffolding is it reduces wandering and keeps the

model on topic. It maintains coherence in the long-form content and
gives you control over the structure
 Scaffolding: SWOT analysis
Single prompt approach
“Give me a SWOT analysis of Company X”

Scaffolded approach:

Prompt 1: List the strengths of Company X.

Prompt 2: List the weaknesses.
Prompt 3: List them for opportunities.
Prompt 4: Now list the threats.
Prompt 5: Combine the above into a full SWOT analysis report.

More thorough and organized analysis

 Utilizing Context and Examples Effectively

Models operate in the here-and-now of the prompt.

For multi-turn scaffolding, remind the model of key points:
“Continuing the essay on climate change impacts, next discuss impacts on oceans…”

For few-shot examples:

Ensure examples are representative of edge cases if needed

The model will pattern-match your examples

Order matters in your prompt

Put important instructions at the beginning or end of the prompt
Explicitly number steps, or use bullets to help the model follow structure
Tree-of-Thoughts Prompting (Branch Out Ideas)
What it is:
Tree-of-Thoughts (ToT) is a prompting approach where the model explores
many possible solutions by branching out like a tree

Like a brainstorming session where you don't stop at the first idea
 Why and When To Use ToT
Tree-of-Thoughts is powerful for very complex or creative problems
Ideal for:

• Complex puzzles or problems with multiple solution paths

• Strategic planning with different possible approaches
• Creative writing with multiple plot options or directions

Benefits:
1. Performs more thorough research of possible solutions
2. Helps avoid getting stuck on wrong assumption
3. Research shows it outperforms chain-of-thought in some cases

Trade-off: is more involved in requiring prompts than simpler methods

 Tree-of-Thoughts Example
Problem
You have a 3-gallon and 5-gallon jug, how can you measure exactly 4 gallons of water?

The model explored a space of solutions rather than one reasoning line
 Self-Reflection and Critique Prompting
What it is:
Self-reflection or self-critique prompting is a technique where the model is
asked to review and critique its own output before finalizing an answer.

Self-Reflection Output
Traditional Output Question →
Question → Answer Initial Answer →
Self-Critique →
Improve Answer

It's like the model taking a moment to check its work -- writing a draft, and
then proofreading it
 How To Use Self-Reflection

1. Multi-turn follow-up:
After getting an answer, prompt:

“Do you spot any mistakes or areas to improve your answer? If so, please correct them. “

2. Single Prompt with Multiple Steps:

Include instructions of self-review in the initial prompt:
“First, answer the question. Then, analyze the answer to see if it could be wrong or
improved and if the answer needs revision, provide a revised answer”

3. Advanced Reflection Approach

The model produces multiple pen attempts, each guided by a critique of
the previous attempt
 Why and When to Use Scaffolding
Useful for improving reliability and quality of the output
Models often recognize a mistake when asked to reflect, even if they made
that mistake in the first pass

Ideal for:

• Complex reasoning tasks where errors are likely

• Creative writing that needs refinement
• Scenarios where incorrect answers could be costly

Watch Out: Sometimes the model might “critique” a perfectly fine answer
and make it worse. This approach is not foolproof
 Self-Reflections Examples
Math Problem Example
Initial prompt: “Provide a step-by-step solution to a train leaves city A...”
Follow-up: “Check your solution for any errors and explain.”
“Upon reviewing, I realize I assumed the trains traveled at the same
time, which was incorrect. The mistake is in step 3…. etc.”
[Model then provides the corrected solution]

Creative Writing Example (Single Prompt)

“Write a short poem about technology. Then critique its style and tone and produce a final
improved poem.”
1. Model writes initial poem.
2. Model critiques – “the tone is a bit dry, could be more playful.”
3. Model provides revised poem with a more playful tone.

Self-critique makes LLM outputs more robust and trustworthy

 Advanced Self-Reflections Examples
Constitutional AI (Anthropic)
The model follows a set of principles to critique and refine responses

After an answer it checks: “Does this follow the principles of honesty

and harmlessness?” and revises if needed

Self- Consistency Approach

Generate multiple answers and see if they agree
Run the prompt several times (with slight randomness) and choose the
answer that the most outputs converged on

Multiple perspectives (even from the same model) lead to better answers
Context Window &
Sensitivity
Context Window Limits & Lost-in-the-Middle Logic
The Issue:
LLMs have a fixed context window, and they pay most attention to the
beginning and end of the prompt

The main problem here is that the model might miss details in the middle
of long inputs and content beyond the context limit is truncated
 Mitigation Strategies (Part 1)
1. Chunking and Summarization
• Break long input into smaller chunks that fit the model's context window
• Summarize or extract key points from each chunk
• Feed those summaries to a final prompt
To analyze a 50-page report: summarize each chapter then ask
questions against the combined summary

2. Emphasize Important Information

• Reiterate crucial details towards the end of your prompt
• Explicitly draw the model's attention to key points.

“Note: The most important facts to remember are X, Y, Z”

 Mitigation Strategies (Part 2)
3. Use headings or structured format
• Break the prompt into sections with clear headings
• Use formats like “Context: [text] User Question: [question]”

4. Leverage Long-Context Models

• Use models with larger context windows (Claude 100K, Gemini 1M)
• Even with large windows, structure the prompt well
• Here's a long document, make sure to scan the whole text.

5. Verify Context Usage

• After getting an answer, explicitly ask the model to check if it is using information from the
entire context.
Did you use information from the entire context? Consider if
something in the middle of the document was relevant
Practical Example
 Prompt Sensitivity (Part 1)

Small changes in prompts lead to very different output styles

Prompt Sensitivity (Part 2)
Prompt Sensitivity (Part 3) - Temperature
Prompt Sensitivity (Part 3) - Temperature
Prompt Sensitivity (Part 4) – Top P
Comparing LLM
Performance
LLM Model Comparison
 GPT-4o & GPT-4.5 Optimization
System Message

Set role, behavior, and consistent requirements.

You are a financial advisor who provides concise, actionable advice.

User Message Formatting Tips

Clear instructions Request step-by-step reasoning
Specify format if needed Use lists or tables for clarity
Provide necessary context Ask for specific output structure

System Message
GPT-4o: Reliable with structure, excels at GPT-4.5: More conversational, may need
detailed reasoning tasks explicit requests for detailed reasoning
Comparing GPT vs Claude
Comparing GPT vs Claude
Gemini Models
Context Window Comparison
LLM Family Comparison: GPT vs Claude vs Gemini

Format

Strengths

Weakness

Best For
Use Cases and
Applications
TLDR
Business Writing → Professional Communications

Few-shot examples establish tone and style

• Show sample emails → model imitates
• Include style guides in prompt
• Role prompting ("Act as HR manager...")

Scaffolding for complex documents

• Get outline first → approve → fill sections
• RAG prevents hallucinations on facts

Marketing needs creativity

• Brainstorm ("Give 5 tagline ideas...")
• Self-critique ("Pick best one and explain why")
Code Generation & Debugging

Be specific about requirements

• Language, libraries, constraints
"Write Python function that sorts list without built-ins"

Chain-of-thought improves correctness

• Explanation first → code second
"Explain how you'd solve X, then code it"

Iterative prompting for debugging

• Feed errors back to model
"Code gives error in line 5, please fix"

Scaffolding for complex projects

• Outline solution approach first
• Generate code for each component
Automation & Workflow Orchestration

Structured output prompting → key

• Request specific format (JSON, XML)
• Easily parsed by next system in pipeline

Few-shot demonstrates expected format

• Show examples of input/output pairs
• Helps model learn the pattern

Function calling for system integration

• Direct outputs to specific actions
• track_order("123") vs lengthy explanation

Divide and conquer

• Use model for individual workflow steps
• Extract info first → format reply second
• More reliable than one giant prompt
Creative Writing → Leverage Imagination

Zero-shot with rich prompt works well

"Write story about dragon who can't fly..."
• Set constraints but allow freedom

Tree-of-Thoughts enhances creativity

"Give three possible endings..."
• Encourages divergent thinking

Role-playing anchors voice

"You are Charles Dickens, reincarnated..."
• Creates consistent style

Self-refinement improves quality

"Write draft → critique → rewrite"
• Model improves its own work
Summarization → Focused Information

Be clear about focus

"Summarize meeting transcript, emphasizing decisions..."
• Steers summary to be useful

Scaffolding for long documents

• Summarize each section
• Then summarize the summaries

Information extraction needs structure

"Extract names, dates as JSON..."
• Direct prompt gets direct results

Chain-of-thought improves coverage

• Outline document first → then compress
• Ensures structure is considered
Customer Service & Chatbots

System prompt defines persona and rules

• "You are customer assistant for ACME Corp..."
• Sets tone and boundaries

Few-shot examples show handling patterns

• Difficult questions
• Escalation protocols

Function calling for specific actions

• reset_password()
• Consistent handling of common requests

Context maintenance across conversation

• Occasionally reprompt with situation summary
• Prevents forgetting in long exchanges
Advanced Reliability Strategies

Combine techniques for best results:

RAG + self-check → factual accuracy

"Ensure answer matches policy snippet"

Chain-of-thought + self-critique → better code

Catches bugs before presenting

Tree-of-thought → interesting creative content

Multiple options → select best

Multiple passes → comprehensive summaries

Extract → verify → refine
Applications