Teaching blueprint

Start from a real confident-but-wrong failure

Explain the boundary between model ability, evidence, authority, and tools

Force the learner to rewrite one task as a decision ladder

Separate fluent output from real reliability

Route tasks into direct answer, clarification, retrieval, tool use, and refusal paths

Turn one confident failure into a safer workflow boundary

Start from a real failure where downstream systems had to guess meaning

Explain the distinct roles of framing, schema, and action execution

Force the learner to convert a free-form output into a contract

Know when to use natural-language instruction, structured outputs, or function calling

Rewrite one real task as an output contract with clear fields and types

Understand why failures must be visible instead of silently guessed through

Start with the real problem of prompts getting longer and worse

Explain the relationship between tokens and context

Give one workflow decomposition exercise

Understand why token budget shapes product boundaries

Separate persistent information from on-demand injection

Start reading your workflow through a budget lens

Show a failure caused by an oversized prompt

Break down a three-layer context structure

Ask the learner to design their own context split

Separate system rules, task state, and live evidence

Diagnose whether a failure is a prompt issue or an architecture issue

Start rewriting giant prompts into structure

Start from a sourced answer that was still wrong because the source was outdated

Break freshness into classes, metadata, ownership, and fallback behavior

Force the learner to govern one real knowledge source instead of discussing RAG in the abstract

Separate retrieval quality from freshness and governance quality

Define owners, review cadence, and expiry thresholds for real knowledge sources

Diagnose whether a grounded answer failed because the evidence was stale, mixed, or unmanaged

Start from a confident answer that should never have been given without evidence

Explain evidence routing, freshness, and provenance as system responsibilities

Force the learner to turn one real question into a retrieval chain

Separate retrieval design from simply making the context longer

Design a real workflow with query, filtering, injection, and citation steps

Diagnose whether a failure came from missing evidence, stale evidence, or noisy retrieval

Start from a workflow that feels magical until it touches a real external action

Break the chain into routing, authority, execution, and recovery responsibilities

Require the learner to rewrite one workflow as an action-ready operating design

Separate model reasoning from tool execution and confirmation logic

Design one real workflow across clarify, retrieve, decide, act, verify, and handoff stages

Turn repeated operator behavior into a reusable SOP or skill

Start from real failures, not abstract benchmarks

Give a minimum eval loop template

Ask the learner to convert their failures into eval samples

Collect real failure samples

Define a minimum eval set

Tie eval results to launch or rollback decisions

Start from the real problem of demos that become slow and expensive in production

Explain critical path, caching, batching, async work, and graceful downgrade

Force the learner to audit one workflow for latency and cost

Separate user-perceived latency from total system latency

Spot waste in duplicated requests, bloated context, oversized outputs, and missing caching

Draft a minimum latency and cost audit for one workflow

Start from a real injection or unauthorized-action scenario

Explain trust boundaries, action boundaries, and graceful downgrade

Force the learner to run a guardrail audit on their own workflow

Separate untrusted text, system rules, and real-world actions by trust level

Recognize prompt injection, prompt leak, and unauthorized-action paths

Design input isolation, confirmation, and downgrade behavior for one real workflow

Start from the cost and reliability damage caused by sending everything through one answer path

Break routing into task classes, thresholds, and fallback logic

Require the learner to write one explicit unsupported-answer policy

Design a routing matrix for different task classes instead of defaulting to one model path

Define when the system should answer directly and when it should not answer yet

Judge routing policy with evals, abstention quality, latency, and cost

Start from a production failure the team cannot reproduce

Explain the roles of traces, replay, and failure labels

Force the learner to redesign one bad case as a replayable debugging record

Know what a minimum useful trace must contain

Replay failures before deciding whether to change prompting, retrieval, tool use, or orchestration

Turn bad runs into assets for later evaluation and prioritization with failure labels

Start from a workflow that everyone says is improving but nobody can score clearly

Break evaluation into named dimensions, anchors, and override rules

Require the learner to build one rubric that another reviewer could actually apply

Turn one abstract quality goal into a rubric with dimensions and score anchors

Keep dimension scores and hard-stop rules instead of relying on one total score

Use rubric evidence to decide what to fix first

Start from a workflow that currently over-answers because nobody wants to escalate

Break the problem into hard-stop triggers, ownership, SLA, and handoff evidence

Force the learner to design one live review queue instead of talking about human-in-the-loop in slogans

Define escalation triggers for risk, authority, evidence, and policy-sensitive requests

Design a handoff packet another human can act on immediately

Treat escalation as a quality path instead of product embarrassment

Set up product, portal, and webhook in test mode first

Map them to the app routes and sync logic

Validate with in-app entitlements and DB state

Create a test product and wire env vars correctly

Run Creem Checkout, Portal, and local webhook forwarding

Understand why you cannot trust success_url alone

State the final artifact, environment requirements, and common failures up front

Drive every step with a checklist

Define success criteria and troubleshooting hints for each step

Finish environment prep, config fill-in, and startup verification

Understand what 3 to 5 critical config items actually control

Know what to check first when things fail

Show the end state first

Then wire tables, env vars, and helpers step by step

Finish with post-login page behavior

Create user tables and RLS rules

Get sign-in, sign-out, and session refresh working

Understand why auth cannot live only in the frontend

Start from one workflow that looks grounded but is vulnerable to stale evidence

Drive the audit through source class, metadata, ownership, and fallback behavior

Require artifacts another operator could use in launch review

Produce a freshness register that classifies source types and expiry thresholds

Define document-governance rules for version, owner, and approval status

Create a stale-content triage path for clarify, refresh, or escalate decisions

Start with the threat-model order before touching prompts

Drive the review with an audit ladder and live red-team attempts

Require a signed-off artifact instead of a discussion-only outcome

Map trusted and untrusted content instead of treating all text as equal

Inspect where tool actions can be triggered by risky inputs

Produce a real pre-launch guardrail review report

Start from one workflow that still answers when it should stop

Drive the review through escalation triggers, owner, SLA, and packet design

Require deliverables that make escalation auditable instead of hand-wavy

Define hard-stop triggers that force escalation instead of unsupported answers

Assign queue ownership and review expectations

Produce a handoff packet template that preserves the evidence a human needs

Establish a baseline before discussing optimizations

Use an audit ladder that starts with waste, not model swaps

Require a reusable report the learner can take back to their product

Separate user-perceived latency from total background work

Expose waste in request count, prompt size, output size, and retrieval volume

Produce a ranked optimization plan with a performance budget

Start from a broken downstream integration instead of prompt-writing tips

Use a contract ladder from framing to schema to acceptance checks

Require a spec and checklist that another operator could review

Separate free-form reasoning from data the rest of the system must trust

Define fields, enums, null behavior, and explicit failure paths for one real task

Produce a contract spec that engineering or ops can actually implement

Start from an answer that should have been grounded but was not

Run the audit through evidence routing, freshness, and citation checks

Require a report that could be used in a launch review

Map where the system should retrieve, cite, abstain, or escalate

Spot stale evidence, noisy retrieval, and missing provenance before launch

Produce a retrieval review report and evidence-chain checklist for a real workflow

Start from one workflow that currently over-answers or over-spends

Drive the review through task classes, routing thresholds, and unsupported-answer handling

Require policy artifacts that another operator could enforce and review

Define a routing matrix that matches task value, risk, evidence need, and budget

Write an unsupported-answer policy that covers clarify, retrieve, abstain, and escalate paths

Produce a fallback ladder instead of letting the system improvise under uncertainty

Start from a workflow currently judged by gut feel

Drive the audit through dimensions, anchors, hard stops, and calibration

Require artifacts that make quality reviewable by a second operator

Write one rubric with dimensions and scoring anchors for a live workflow

Define grader instructions and hard-fail rules instead of relying on overall impressions

Leave with a calibration sheet that makes scoring disagreements visible

Start with one workflow that currently feels agentic but is still fragile

Drive the audit through routing order, tool boundaries, and recovery order

Require a reusable artifact that survives beyond one careful operator

Map the workflow into explicit clarify, retrieve, act, verify, and handoff stages

Define which actions need evidence, which need confirmation, and which should never auto-run

Produce a routing sheet, a tool-boundary checklist, and one operator skill brief

Define project scope and acceptance criteria

Advance in stages across content, auth, billing, data, and trust

Require a final demo and recap

Build the complete product loop independently

Explain your architecture choices

Show a real working result