NoSQL Concepts & Document Storage

pip install tinydb

Now, let's create a database and save our first simple message.

from tinydb import TinyDB, Query

# 1. Create (or connect to) the database file
# This will create a file named 'chat_db.json' in your folder
db = TinyDB('chat_db.json')

# 2. Clear the database so we start fresh every time we run this script
# (Useful for learning, but you wouldn't do this in a real app!)
db.truncate()

# 3. Insert a simple text message
# Note: We just pass a Python dictionary. No SQL commands needed.
db.insert({
    "role": "user", 
    "type": "text", 
    "content": "Hello, AI!"
})

print("Database created and first message saved!")
print("Check your folder for 'chat_db.json'.")


Step 2: The Power of Flexibility

Now for the magic. We will insert a message with a completely different structure—an image generation result. We don't need to change any settings. We just insert it.

from tinydb import TinyDB

db = TinyDB('chat_db.json')

# 1. Insert a standard text response
db.insert({
    "role": "assistant",
    "type": "text",
    "content": "Hello! I can generate images for you."
})

# 2. Insert a COMPLETELY different structure (Image)
# It has 'url' and 'resolution' instead of 'content'
db.insert({
    "role": "assistant",
    "type": "image",
    "url": "https://myserver.com/robot_art.png",
    "resolution": "1024x1024",
    "metadata": {
        "model": "dall-e-3",
        "cost": 0.04
    }
})

print("Inserted mixed data types successfully.")
print("-" * 20)

# 3. Print all data to prove it's there
all_messages = db.all()
for msg in all_messages:
    print(msg)


Why this matters: Notice the metadata field in the second message? It's a dictionary inside the message. SQL struggles with nested data like this. NoSQL loves it.

Step 3: Querying (Searching) Data

Storing data is useless if you can't find it. In SQL, you use SELECT * FROM table WHERE.... In TinyDB, you use a Query object.


Think of the Query object as a "search template."


from tinydb import TinyDB, Query

db = TinyDB('chat_db.json')

# 1. Create a Query object
# This acts as a placeholder for our search
Message = Query()

# 2. Search for all messages where the role is 'assistant'
print("--- Assistant Messages ---")
assistant_msgs = db.search(Message.role == 'assistant')

for msg in assistant_msgs:
    print(f"Found one: {msg}")

# 3. Search for specific types (e.g., images)
print("\n--- Image Messages ---")
images = db.search(Message.type == 'image')

for img in images:
    print(f"Image URL: {img['url']}")


Step 4: Updating Data

Imagine the user liked the image generated by the AI. We want to add a liked: True flag to that specific record.


We can update records based on a search query.

from tinydb import TinyDB, Query

db = TinyDB('chat_db.json')
Message = Query()

# 1. Find the image message and add a 'liked' field
# We search for the specific URL to identify the message
target_url = "https://myserver.com/robot_art.png"

# The update function takes two arguments:
# 1. The data to add/change
# 2. The condition to find which documents to change
db.update({"liked": True}, Message.url == target_url)

# 2. Verify the change
updated_msg = db.search(Message.url == target_url)
print("--- Updated Record ---")
print(updated_msg)


Step 5: Handling Missing Keys (The Solution)

Now that we have mixed data, how do we display it without crashing? We use the .get() method, which is safe to use on dictionaries.


dictionary.get("key", "default_value") attempts to find the key. If it fails, it returns the default value instead of crashing.

from tinydb import TinyDB

db = TinyDB('chat_db.json')

print("--- Safe Chat History Display ---")

for msg in db.all():
    role = msg.get('role', 'unknown')
    
    # Check the type to decide how to print
    msg_type = msg.get('type', 'text')
    
    if msg_type == 'text':
        content = msg.get('content', '')
        print(f"[{role}]: {content}")
        
    elif msg_type == 'image':
        url = msg.get('url', 'no url')
        liked = " (Liked!)" if msg.get('liked') else ""
        print(f"[{role}]: [Image displayed from {url}]{liked}")


Output:
--- Safe Chat History Display ---
[user]: Hello, AI!
[assistant]: Hello! I can generate images for you.
[assistant]: [Image displayed from https://myserver.com/robot_art.png] (Liked!)


We now have a robust system that handles text, images, and metadata without a single crash, all stored in a permanent file.

Now You Try

It is time to extend your database skills. Create a new script for these tasks.

1. The Timestamp Extension
When inserting data, import the datetime module. Add a timestamp field to every message you insert containing the current time (string format).

Hint: import datetime; str(datetime.datetime.now())

2. The "Mark as Read" Feature
Write a script that updates every message in the database to have a new field: read_status: True.

Hint: In db.update, if you omit the second argument (the condition), it applies to ALL documents.

3. The Cleanup Crew
Write a script that deletes all messages where the type is "text". Keep the images.

Hint: Look up db.remove(...) in the TinyDB documentation or use your IDE's autocomplete to see how it works. It works very similarly to search.

Challenge Project: The OpenAI Logger

When you build real AI apps, you send a prompt to OpenAI and get back a huge, complex JSON object containing the answer, the number of tokens used (cost), the model version, and finish reasons.

Your challenge is to build a Conversation Logger.

Requirements:
 Create a function log_interaction(user_text, ai_response_json) that saves data to logs.json.

 The user_text should be stored as a simple string field.

 The ai_response_json will be a nested dictionary (simulating a real API response). Store this entire structure inside your database document under a field called raw_response.

 Add a top-level field called cost_tokens that extracts the total_tokens from the nested JSON.


Example Input Data (Use this to test):

# Simulating what OpenAI sends back
mock_openai_response = {
    "id": "chatcmpl-123",
    "object": "chat.completion",
    "created": 1677652288,
    "model": "gpt-3.5-turbo",
    "choices": [{
        "index": 0,
        "message": {
            "role": "assistant",
            "content": "Python is awesome!"
        },
        "finish_reason": "stop"
    }],
    "usage": {
        "prompt_tokens": 9,
        "completion_tokens": 12,
        "total_tokens": 21
    }
}

user_input = "What do you think of Python?"


Desired Database Document Structure:
Your database should contain a document that looks like this:

{

"user_prompt": "What do you think of Python?",

"timestamp": "2023-10-27 10:00:00",

"cost_tokens": 21,

"raw_response": { ... the full dictionary from above ... }

}