Dynamic Templates & Evaluation

pip install jinja2 openai

import os
from jinja2 import Template
from openai import OpenAI

# Initialize the client (ensure your API key is set in environment variables)
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

print("Libraries loaded successfully.")

Step 2: Your First Template

Let's create a template string. Notice how clean it looks compared to the concatenation method in "The Problem."

# Define the template string
# We use triple quotes (""") for multi-line strings in Python
email_template_str = """
You are a world-class sales copywriter.
Write a cold email to {{ name }}, who is the {{ role }} at {{ company }}.

The goal of this email is to schedule a demo of our product, 'SpeedyAI'.
Focus heavily on solving their main problem: {{ pain_point }}.

Keep the tone professional but friendly.
"""

# Create a Jinja2 Template object
template = Template(email_template_str)

# Define our data (the context)
data = {
    "name": "Marcus",
    "role": "CTO",
    "company": "TechFlow",
    "pain_point": "high cloud infrastructure costs"
}

# Render the template
# This fills in the slots with the data
final_prompt = template.render(data)

print("--- GENERATED PROMPT ---")
print(final_prompt)

Step 3: Adding Logic to Templates

Let's update our template to handle different roles differently.

# A more advanced template with logic
advanced_template_str = """
You are a sales assistant. Write an email to {{ name }} at {{ company }}.

{% if role == "CEO" %}
CRITICAL INSTRUCTION: The recipient is a CEO. 

Keep the email extremely short (under 75 words).
Focus on ROI and bottom-line value.
Do not use technical jargon.

{% else %}
The recipient is a technical leader.

You can explain the technical implementation details.
Feel free to use industry terminology.
Length can be around 150 words.

{% endif %}

Problem to solve: {{ pain_point }}.
Sign off as: The SpeedyAI Team.
"""

template = Template(advanced_template_str)

# Test 1: CEO
ceo_data = {
    "name": "Sarah",
    "company": "Innovate Inc",
    "role": "CEO",
    "pain_point": "slow time-to-market"
}

# Test 2: Engineer
dev_data = {
    "name": "Mike",
    "company": "DevCorp",
    "role": "Senior Engineer",
    "pain_point": "messy documentation"
}

print("--- CEO PROMPT ---")
print(template.render(ceo_data))

print("\n--- ENGINEER PROMPT ---")
print(template.render(dev_data))

Step 4: Connecting to the LLM

Now that we have a perfect prompt generator, let's feed it to the LLM to actually get our email. We will create a function to handle this.

def generate_email(person_data):
    # 1. Define the Template
    prompt_template = """
    Write a cold outreach email to {{ name }}, the {{ role }} at {{ company }}.
    
    {% if role == "CEO" %}
    tone: Direct and value-focused. Max 3 sentences.
    {% else %}
    tone: Helpful and consultative. Can be longer.
    {% endif %}
    
    Pain point: {{ pain_point }}
    Product: SpeedyAI (automates code documentation).
    """
    
    # 2. Render the Prompt
    template = Template(prompt_template)
    filled_prompt = template.render(person_data)
    
    # 3. Call the LLM
    response = client.chat.completions.create(
        model="gpt-3.5-turbo", # Using 3.5 for speed/cost
        messages=[
            {"role": "system", "content": "You are a helpful sales assistant."},
            {"role": "user", "content": filled_prompt}
        ],
        temperature=0.7
    )
    
    return response.choices[0].message.content

# Let's try it out
prospect = {
    "name": "Diana",
    "role": "CEO",
    "company": "Marketing Wizards",
    "pain_point": "spending too much time on manual reporting"
}

email_draft = generate_email(prospect)

print(f"--- EMAIL FOR {prospect['name']} ---")
print(email_draft)

Step 5: Introduction to Evaluation (LLM-as-a-Judge)

We have a generator. But is it good?

In AI, we can't easily assert that an email is "good."

However, we can use an LLM to grade the output. This is called "LLM-as-a-Judge." We will ask GPT-4 (or a strong model) to score the email generated by GPT-3.5.

def grade_email(original_prompt, generated_email):
    evaluation_prompt = f"""
    You are a strict editor. I will provide you with an Email Draft.
    
    Please grade the email on a scale of 1 to 5 based on these criteria:
    1. Is it polite?
    2. Does it address the specific pain point mentioned?
    3. Is the length appropriate?
    
    Email Draft:
    "{generated_email}"
    
    Return ONLY the number (1-5).
    """
    
    response = client.chat.completions.create(
        model="gpt-4o", # Ideally use a stronger model for grading
        messages=[{"role": "user", "content": evaluation_prompt}]
    )
    
    return response.choices[0].message.content

# Run the grader
score = grade_email(prospect, email_draft)
print(f"\nQuality Score: {score}/5")

Now You Try

You have the basic engine. Now, let's make it robust.

Challenge Project: The Automated Grader

Your challenge is to build a script that objectively measures the accuracy of an LLM on a specific task.

* The Judge prompt should look like: "Question: [Q]. Correct Answer: [A]. Student Answer: [Student Answer]. Is the student correct? Answer YES or NO."

What You Learned

Today you moved from "hardcoding" to "engineering."