Interactive Field Guide · From Jason's Forge

Why AI Code
Lies to You
— and How to Catch It

Claude writes confident code. Your server doesn't care about confidence. This lesson teaches you the exact system to validate AI output before it wastes your time.

6
Lessons
4
Scenarios
10
Quiz Qs
~20
Minutes
↗ View the Step-by-Step Guide
Lesson 1 of 6
The Core Problem
Why AI Code Fails (Even When It Looks Right)
Before we fix anything, understand the exact failure pattern. It's not random.
The Setup

You asked Claude Code for a cron job to post to X. It gave you code in under 30 seconds. It looked clean. It failed silently at 9am and you had no idea why.

There are exactly 3 reasons this happens. Click each one:

🌍 Environment Blindness
🔇 Silent Failures
🔑 Auth Assumptions
Environment Blindness — When you run code in your terminal, your personal settings load automatically: Python path, API keys, folder shortcuts. When a cron job runs your code, it starts with a completely blank slate. No settings. No paths. No environment variables. Claude writes code for your terminal. Cron runs in a desert.
Silent Failures — By default, a script that crashes at 3am just dies quietly. No email, no log, no notification. You assume it worked and find out days later. Claude doesn't add logging unless you explicitly ask — and most people don't know to ask.
Auth Assumptions — X/Twitter's API requires specific permissions: your app must have Write access enabled (not just Read), and your API tier must allow posting. Claude assumes you've set this up. It hasn't verified anything — it just wrote the connection code and hoped.
🔭
The Real Insight

AI isn't lying — it's writing for an ideal environment that doesn't exist on your machine. Every gap between "what Claude assumed" and "what your server actually has" is a failure waiting to happen.

⚡ Quick Check
You run python3 post.py in your terminal — works perfectly. You set up a cron job for the same script — fails every time. What's most likely wrong?
Lesson 2 of 6
Step 1 of the System
Write the Spec Before the Code
The PRD doesn't prevent all failures — it eliminates the assumptions Claude makes about your environment.
Jason's Forge Rule

Claude Code executes against a spec. If you don't give it one, it invents one — based on ideal conditions, not yours. A PRD is just a list of facts Claude can't guess.

A Forge-standard PRD answers 5 questions. Reveal each:

Why It Matters

Cron uses the system Python, not yours. Run which python3 to get the full path — use that exact path in cron, not just python3. If you installed tweepy via pip, you're using a virtual environment and need the venv's Python path: which python3 will show it if the venv is active.

Why It Matters

Keys go in a .env file, never in the code. Claude needs the absolute path to load it explicitly. But first — verify at developer.twitter.com that your X app has Read and Write permissions. The default is Read-only. A perfectly written script fails silently with the wrong permissions.

Why It Matters

Without logs, failures are invisible. Use an absolute path: not ./logs/post.log — that's relative and cron won't find it. Use /home/joe/scripts/logs/post.log. Cron has no working directory to resolve relative paths from.

Why It Matters

Cron uses 5-field syntax that's easy to get wrong. Tell Claude the schedule in plain English — "every day at 9am" — and it generates the correct expression. You don't need to know cron syntax.

Why It Matters

Should it retry? Notify you? Log and exit? Claude will make a default decision — make sure it's your decision. The most useful option for beginners: log the error and exit cleanly so you can diagnose from the file.

⚡ Quick Check
Which path is correct for the log file in a cron job?
Lesson 3 of 6
Step 2 of the System
Use a Second AI to Audit the First
A fresh prompt with critic framing finds what the builder missed.
The Problem

The AI that wrote the code was primed to produce it. A second prompt — even to the same model — with critic framing produces different, more skeptical output. You need a critic, not a builder.

🔭
2nd Order Effect

Opening a new chat means zero priming toward the code being good. It will find environment issues, missing error handling, and auth gaps the original window glossed over.

Code Critic Prompt — New Chat
You are a code critic. NOT a code writer.

This script runs via cron and posts to X/Twitter.

Find every way this FAILS:
1. Will it work when cron runs it with no environment loaded?
2. Are ALL file paths absolute (start with /)?
3. Is the .env file loaded with an absolute path?
4. Is auth validated BEFORE attempting to post?
5. Does it log errors or crash silently?
6. Does it use the correct Python path for this setup?
7. What happens on failure?

For each issue: exact line number and the fix.

[PASTE CODE HERE]
🐛 Spot the Bug
Claude generated this script. You're the critic. Which line is the most dangerous failure?
1import tweepy
2import os
3API_KEY = "AaBbCcDd1234"
4auth = tweepy.OAuth1UserHandler(API_KEY, API_SECRET)
5api = tweepy.API(auth)
6api.update_status("Good morning!")
Lesson 4 of 6
Step 3 of the System
Make Failures Scream — The Logging Wrapper
Without logs you're flying blind. With them, every failure hands you a diagnosis.
The Mental Model

Imagine running your store at night with no security cameras. Logging is the camera. Every time the script runs — success or failure — it writes a timestamped record you can read the next morning.

The Logging Wrapper
import logging
import sys
from dotenv import load_dotenv

# ABSOLUTE path to .env — not ./ or ~/
load_dotenv('/home/joe/scripts/.env')

# ABSOLUTE path to log file
logging.basicConfig(
    filename='/home/joe/scripts/logs/post.log',
    level=logging.DEBUG,
    format='%(asctime)s | %(levelname)s | %(message)s'
)

logging.info("=== Script started ===")

try:
    # YOUR CODE GOES HERE

    logging.info("Post successful")

except Exception as e:
    logging.error(f"FAILED: {e}")
    sys.exit(1)  # signals failure to cron
🐛 Spot the Bug
This logging setup has one problem that silently breaks it on a cron job. Which line is wrong?
1import logging, sys
2load_dotenv('.env')
3filename='/home/joe/scripts/logs/post.log'
4logging.info("=== Script started ===")
5sys.exit(1)
🔭
2nd Order Effect

Once logging is in place, every future failure becomes a one-line diagnosis. You stop debugging with your brain — you paste the log into Claude and get the fix. That's the real leverage here.

Lesson 5 of 6
Steps 4 + 5 of the System
Test in the Real Environment, Then Declare Success
One command exposes the environment gap. One declaration makes success binary.
The Test Most People Skip

This single command runs your script the same way cron will — with no environment loaded. If it fails here but works in your terminal, you just found the cron gap before it went live.

Step 1: Find your exact Python path first. This is critical if you used pip to install tweepy:

Find Python Path
which python3

Step 2: Run the simulation using that exact path:

Cron Simulation Command
# Standard Python (replace with your path from 'which python3')
env -i /usr/bin/python3 /absolute/path/to/script.py

# If you used pip / virtual environment:
env -i /home/joe/myproject/venv/bin/python3 /absolute/path/to/script.py
⚠️ Virtual Environment NoteIf you installed tweepy with pip install tweepy, you have a virtual environment. Using the wrong Python path in this test (or in your cron job) means Python can't find tweepy and the script crashes with ModuleNotFoundError. Use the output of which python3 while your venv is active.
ℹ️ macOS UsersOn modern macOS, cron is deprecated. If crontab -e doesn't fire your job, use launchd instead. Ask Claude: "Convert this cron schedule to a macOS launchd plist file" — it will generate the correct file and the commands to load it.
⚡ Quick Check
Your cron simulation passes. Your smoke test says the tweet should appear within 2 minutes. After 10 minutes — nothing on X. But the log says "Post successful." What happened?
Lesson 6 of 6
Step 6 of the System
The Feedback Loop — Logs Beat Descriptions
Once this is built, you never debug manually again.
The Leverage Point

When something breaks, most people describe the problem in English to Claude. Stop doing that. Paste the log. Paste the script. Claude diagnoses from actual error output far more accurately than from your interpretation of what you think happened.

Failure Prompt — Use Every Time
Here is my error log.
Here is the script.

Tell me:
1. The exact line that failed
2. Why it failed
3. Give me the corrected script only — no explanation

LOG:
[paste log here]

SCRIPT:
[paste script here]
🔭
2nd Order Effect

Every failure you process through the log→paste→fix loop makes the next one faster. You stop debugging with your brain — you debug with evidence. That's how diagnostic systems work in every professional field.

The Complete System — In Order

1
Write the PRD — eliminate Claude's assumptions
2
Second AI audits the code — find failures before running
3
Add logging wrapper — failures scream instead of disappear
4
Cron simulation test — verify in the real environment
5
Declare expected outcome — make success binary
6
Log → paste → fix loop — compound speed over time

Final Quiz

10 questions · Wrong answers reveal the correct one

0 of 10 answered
0 correct
0
/ 10
Open the Step-by-Step Guide →