Case Study
Training Prep Agent
A real AI agent that drafts, critiques, and finalizes technical training modules for gas fitter training — one that decides its own next step based on what it actually finds, instead of following a fixed script.
The Problem
I prepare technical training modules for gas fitter skills training — theory, store collection, on-site installation, stock returns, emergency response — roughly every 4–6 weeks. Drafting one by hand in Word takes about 2 hours, and there's no built-in check on quality: no one forcing a second look for missing safety content, no consistency check against past modules.
What I Did
I built a Training Prep Agent as a Claude Project. The first version was a fixed workflow — draft, critique, revise, format, always in that exact order, regardless of what the critique actually found. The real upgrade was making it an agent: it now decides its own next step. If a critique only finds minor gaps, it revises once. If it finds something severe — like missing life-safety content — it runs a second critique-revise loop on its own, and says explicitly that it's doing so and why. If a real code or regulation reference is needed, it checks with web search instead of guessing. If something is genuinely ambiguous even after checking, it stops and asks me directly rather than finalizing a guess.
How It Works Now
I type "Run pipeline: [topic]" inside the Claude Project → it drafts a short outline and slide bullets → critiques its own draft as a skeptical instructional designer → decides whether one revision is enough or a second critique-revise loop is needed → checks Google Drive for my past training docs for consistency → verifies any real code/regulation reference via web search rather than stating an unverified number as fact → formats a final, numbered, timed training document.
Real Run: What Actually Happened
On a real run ("Installation & Use On Site"), the agent's self-critique caught two severe gaps on its own — no standalone CO-poisoning/life-safety block, and no explicit stop-work criteria for refusing to commission an unsafe installation. Per its own instructions, it ran a second critique-revise loop specifically because these were safety gaps, not polish issues, and said so explicitly.
It then searched for a real installation-practice code to replace a vague placeholder phrase, found that no fitter-level code is publicly indexed, and refused to invent one — flagging it as an open item in the final document instead. It also noticed, unprompted, that unrelated documents in my Drive used a consistent house style (header block, bilingual delivery, an ethics section), and asked whether this new module should match that style for consistency across my real course library. I hadn't instructed it to look for that.
[You type: "Run pipeline: <topic>"]
|
v
[DRAFT] — short outline + slide bullets
|
v
[CRITIQUE] — self-review as skeptical instructional designer
|
v
[DECIDE severity]
| | |
minor gaps severe gap needs a real
| | code/reg reference
v v |
[REVISE once] [REVISE, then [web search to verify]
| CRITIQUE again] |
| | still uncertain?
| v |
| [REVISE again] v
| | [STOP, ask user directly]
'---------------'
|
v
[FORMAT] — numbered, timed, consistent final module
The agent's actual decision logic — not a stock diagram, this is what the real Claude Project instructions define.
What Came of It
A complete, formatted training document — six timed sections, a dedicated life-safety block added specifically because of the second critique pass, an explicit "open verification item" box instead of a guessed citation, and a 20-mark pre-test plus 30-mark post-test matching my real course house style. Every run ends with the module marked "pending review, not approved for delivery" — a deliberate guardrail, since the agent drafts and checks its own thinking, but I own the final call before anything reaches a real trainee.
Full documentation — setup instructions, real eval results, known limitations, and a narrated live demo — is in the project README.
Watch the demo video →Get In Touch
Want to see how I'd build something like this for your team?
Contact me →