Failure-mode competency scope

This document maps each AdCP failure-mode scenario to the certification module(s) where it belongs, specifies the depth target per tier, and defines the assessment approach. It is a scoping artifact — authoring of the actual content in each module is tracked in follow-up issues per module. Depth levels used below:

Level	Meaning
Surface / recognize	Knows the failure mode exists; can name it when prompted
Diagnose / explain	Can describe the cause, the affected protocol surface, and the correct recovery path
Resolve / demonstrate	Can execute recovery hands-on using protocol tools against a sandbox agent
Evaluate / create	Can reason about multi-domain conflicts, adjudicate competing rules, and construct novel scenarios

FM-1 — Idempotency replay / conflict / expired

Status: Partially covered. #2346 and #2367 established idempotency in the training agent. This entry consolidates the assessment scope.

Module	Depth	Assessment approach
S1 (Media buy)	Resolve / demonstrate	Spot-the-error: three-call transcript — (1) retry with same key + same payload → `replayed: true`; (2) re-plan with new payload + same key → `IDEMPOTENCY_CONFLICT`; (3) key used after TTL → `IDEMPOTENCY_EXPIRED`. Learner identifies which caller violated the contract and explains correct behavior for all three.
A2	Surface / recognize	Learner explains that `idempotency_key` makes agent retries safe for real money — no deeper troubleshooting required.
C4 (Build project)	Resolve / demonstrate	Learner’s submitted agent generates idempotency keys correctly and handles retries without re-planning.

Gap to close before authoring: Confirm S1 lab exercise 7 (“Lifecycle management”) stages all three error states in sandbox, not just NOT_CANCELLABLE. If not, extend exercise 7 to cover IDEMPOTENCY_CONFLICT and IDEMPOTENCY_EXPIRED.

FM-2 — Creative compliance failure post-launch

Status: Not yet in any module. Prerequisite reading is present in S2 and S4, but no lab exercise walks the post-launch discovery path.

Module	Depth	Assessment approach
S2 (Creative)	Diagnose / explain + demonstrate	Scenario: a creative passes `preview_creative` compliance at build time, but `validate_content_delivery` returns a violation after flight start (e.g., served variant omits a regulatory disclosure present in the preview render). Learner uses `get_media_buy_artifacts` to retrieve the audit artifact, explains the discrepancy, and describes the remediation options (pause and swap vs. cancel).
S4 (secondary)	Diagnose / explain	Learner explains how `get_media_buy_artifacts` and `validate_content_delivery` connect to the governance audit trail.
C2	Surface / recognize	Learner knows that creative compliance can fail post-launch and that the discovery path is distinct from pre-launch checks.

Gap to close before authoring: Add a numbered lab exercise to S2 that explicitly stages a post-launch compliance failure using validate_content_delivery. The prerequisite reading card is present; the exercise is not. Without a staged scenario, assessment relies entirely on conversation and cannot satisfy IACET Element 7 (demonstrable competency evidence). The authoring issue must specify three things to prevent a reading card from being filed as a lab exercise: (a) the sandbox agent must be configured to return a validate_content_delivery violation on a specific creative ID after a simulated flight-start event; (b) the learner must invoke get_media_buy_artifacts within the same session and correlate the audit artifact to the violation; (c) this constitutes a required demonstration with a stable criterion ID (e.g., s2_postlaunch_sc0) — the recertification machinery only fires if the ID exists.

FM-3 — Payment / settlement reconciliation differences

Status: Not yet in any module. Schema being finalized in #2391 (billing reconciliation, AdCP 3.1). Scoped now against current delivery and accountability-terms surfaces; depth will expand when #2391 lands.

Module	Depth	Assessment approach
S1 (Media buy)	Diagnose / explain	Scenario: `get_media_buy_delivery` shows reported impressions 12% below the guaranteed commitment at flight midpoint, and a `billing_measurement` vendor entry that differs from the `measurement_terms` accepted at buy creation. Learner (a) identifies the discrepancy type — delivery shortfall vs. measurement vendor mismatch; (b) names the correct remediation for each — `update_media_buy` to adjust pacing vs. escalating the measurement vendor discrepancy per `measurement_terms` negotiation; (c) identifies which protocol artifact is the settlement record.

Flag for authoring: Assign a stable criterion ID (e.g., s1_recon_sc0) at authoring time. When #2391 ships a reconciliation schema, S1 credentials issued under the current criteria must be flagged for targeted recertification — the recertification machinery in the instructional design framework only fires if the criterion ID exists. Do not leave this implicit. Depth TBD pending #2391: Once the billing reconciliation schema lands, add a second criterion (s1_recon_sc1) covering the new settlement fields. Credentials issued before that addition are candidates for recertification per the protocol-triggered recertification policy.

FM-4 — Governance token mismatch / authorization revoked mid-lifecycle

Status: Partially covered. S4 covers GOVERNANCE_DENIED recovery, the 15-step JWS verification, and the governance_context correlation model. Mid-lifecycle revocation is distinct from denial at check time and is not yet named as a scenario.

Module	Depth	Assessment approach
S4 (Governance)	Resolve / demonstrate	Scenario: a `governance_context` token was issued at campaign launch; mid-flight, the governance agent revokes authorization (brand exits a market, rights grant expires). Seller’s execution-phase `check_governance` returns a revocation status. Spot-the-error: a seller implementation that silently continues serving after receiving revocation. Learner identifies the failure, explains correct behavior (halt execution, webhook orchestrator, re-run `sync_plans` with updated parameters), and explains which step in the 15-step JWS verification catches key compromise vs. a valid revocation.
C2	Surface / recognize	Learner knows that a governance token can be revoked mid-lifecycle and that the seller’s obligation is to halt, not continue.

Gap to close before authoring: S4’s “What you’ll demonstrate” section covers the 15-step JWS verification and governance_context correlation model but does not name mid-lifecycle revocation as a discrete scenario. Add it as a demonstration item and extend lab exercise 7 (“GOVERNANCE_DENIED recovery”) with a revocation-during-execution variant.

FM-5 — Lifecycle state stuck (media buy, creative, account, SI session, catalog)

Status: Not yet in any module as an explicit failure-mode scenario. S1 lab exercise 6 covers forced rejection from pending_start but not timeout without response. S5 covers normal SI session management but not stuck or expired sessions.

Module	Depth	Assessment approach
S1 (Media buy)	Resolve / demonstrate	Scenario: a media buy has been in `pending_start` for 36 hours with no seller-initiated transition and no webhook. Learner (a) uses `get_media_buys` to read `valid_actions` and confirm state; (b) identifies that `cancel` is available from `pending_start`; (c) explains why `pause` is not valid from `pending_start`; (d) describes the webhook the seller MUST send at flight start and what happens when it is absent.
S2 (Creative — sync stuck)	Diagnose / explain	Scenario: `sync_creatives` call returns `accepted` but creative approval status never updates and the buy stays in `pending_creatives`. Learner reads `valid_actions` on the buy, identifies the seller-side obligation, and describes the escalation path.
S5 (SI — session stuck)	Diagnose / explain	Scenario: an SI Chat Protocol session has no `session_end` event after the expected TTL. Learner explains session expiry semantics, what the host must do, and what state risk a non-terminated session creates.
D1 / D3 (Platform track)	Surface / recognize	Learner knows that async protocol operations can stall and explains the polling-vs-webhook reconciliation pattern.

Gap to close before authoring:

S5’s current “What you’ll demonstrate” covers normal session management only. Add session expiry and stuck-session recovery as explicit demonstration items.
S1 lab exercise 6 should be extended (or a variant added) for the timeout-without-response case, distinct from the forced rejection already staged.

FM-6 — Webhook delivery failure / retries

Status: Not yet in any module as a failure scenario. D3 prerequisite reading references error handling but no lab exercise or assessment dimension covers webhook failure.

Module	Depth	Assessment approach
D3 (Platform)	Diagnose / explain + configure	Scenario: a seller’s `active` → `paused` transition is lost — the webhook endpoint returned 503 on first attempt and the retry backoff exceeded the buyer’s expectation window. Learner (a) describes correct retry semantics (exponential backoff, idempotency of delivery events); (b) identifies what the buyer agent should do when the expected webhook doesn’t arrive (poll `get_media_buys`); (c) explains the tradeoff between webhook-driven and poll-driven state reconciliation.
S1 (secondary)	Diagnose / explain (consumer perspective)	Learner explains how to detect a missing webhook, when to poll instead, and how this affects campaign state management.
B3 (Publisher track)	Surface / recognize	Learner knows that webhooks can fail and that sellers must implement retries.

Gap to close before authoring: Add a scenario within D3’s error-handling discussion (not necessarily a full new exercise) that walks a webhook delivery failure end-to-end. D3 currently lists the error-handling docs as prerequisite reading but has no lab exercise or assessment item that tests operational recovery.

FM-7 — Signed-request validation failure

Status: Not yet in any module as a failure scenario. S1 covers the buyer-identity resolution chain (signature → JWKS → agent entry → brand.json) conceptually, but no module tests a fail-closed implementation.

Module	Depth	Assessment approach
D2 (Platform)	Resolve / demonstrate (implementation)	Spot-the-error: a seller implementation accepts a request where `iss` matches a known brand but the JWKS fetch fails with a network error, and the implementation falls back to trusting the `iss` claim. Learner (a) identifies the vulnerability — accepting identity claims without key verification; (b) explains correct behavior — fail closed, reject the request, do not fall back; (c) describes the SSRF risk in the JWKS fetch and the mitigation.
S1 (secondary)	Diagnose / explain (reasoning)	Learner explains what each link in the identity chain defends against and why a JWKS fetch failure must not trigger a fall-through to bare `iss` trust.
B2 (Publisher track)	Surface / recognize	Learner knows that incoming requests must be signature-verified and that validation failures must reject, not accept.

Note on criterion IDs: Assign separate criterion IDs for D2 (d2_sig_sc0) and S1 (s1_sig_sc0) at authoring time so that a future change to RFC 9421 request signing triggers recertification in both modules independently. Related (FM-C below): adagents.json / brand.json authorization failure at agent discovery is a closely related onboarding failure mode scoped separately.

FM-8 — TMP provider integration failure

Status: Reclassified from “TMP attestation failure.” TMP cryptographic attestation is a SHOULD (not a MUST) in AdCP 3.0 and is marked “future enhancement” in the specification — the current conformance model is publisher-attested via adagents.json over HTTPS. Teaching cryptographic attestation failure as a certification topic before the mechanism is stable would credential knowledge of a future feature rather than current protocol behavior. The operationally real failure modes today are: (1) adagents.json binding failure — provider not listed, seller_agent URL mismatch, or bypass-mode misconfiguration; (2) TMP Router provider configuration failure; (3) Identity Match returning no result due to integration misconfiguration, causing the frequency-cap logic to fall back to no-cap behavior. Home is S1, not S3 — TMP is a media buy execution mechanism, not a signals/audiences topic. Deferred: Cryptographic attestation failure scenarios will be added when TMP moves out of experimental status.

Module	Depth	Assessment approach
S1 (Media buy)	Diagnose / explain	Scenario: a TMP Identity Match request returns no result because the `adagents.json` entry for the match provider does not list the seller’s `seller_agent` URL, or the bypass mode is misconfigured. The orchestrator’s frequency-cap logic falls back to no-cap behavior, resulting in over-delivery. Learner (a) explains what `adagents.json` binding proves — that responses come from a registered TMP provider without exposing user data; (b) describes correct fallback behavior — conservative: treat user as unknown, apply default frequency cap; (c) explains why Context Match and Identity Match are structurally separated and how this failure affects frequency capping but not creative selection.
D3 (secondary)	Diagnose / explain (router config)	Learner identifies a provider-configuration gap in the TMP Router setup: `adagents.json` entry missing or `seller_agent` URL mismatch. Explains the diagnostic steps and the configuration change to recover.
S3 (tertiary)	Surface / recognize	Learner knows that TMP integration failure degrades identity matching — not context matching — and that the fallback is conservative frequency behavior.

Gap to close before authoring: Add a failure variant to S1 lab exercise 9 — same cross-publisher suppression scenario, but Identity Match returns no result due to adagents.json misconfiguration. This is an extension of an existing exercise, not a new one.

FM-9 — Cross-protocol policy conflicts

Status: Not yet in any module. S4 covers composing governance domains (campaign, property, collection, content standards, creative) but does not include a scenario where rules from multiple domains conflict and must be adjudicated.

Module	Depth	Assessment approach
S4 (Governance)	Evaluate / create	Open-ended exam question (no lab): A campaign plan specifies `policy_categories: ['fair_housing']` with `restricted_attributes: ['zip_code']`. The creative governance feature evaluation (`get_creative_features`) passes a creative referencing a zip-code-adjacent geo signal. The signals activation (`activate_signal`) for a geo/mobility provider includes a trade-area segment derived from zip codes but not labeled with `restricted_attributes`. Which governance domain has precedence, what is the buyer agent’s obligation, and what would a correct implementation do? Learner must reason across campaign governance (plan-level constraint), creative governance (feature evaluation), and signals governance (restricted attributes on signal metadata).
S1, S2, S5	Cross-reference	Each module’s “What you’ll demonstrate” section should cross-reference S4 as the authoritative home for cross-domain policy adjudication. No independent assessment in S1/S2/S5.

Rationale for S4 (not a new cross-domain section): S4 already covers governance domains composing, including their interaction across campaign, property, collection, content standards, and creative. The cross-protocol scenario is an extension of existing S4 scope, assessable with S4 + S3 prerequisite knowledge. A new module section would require authoring scope that this issue explicitly defers. Flag for #2391: If billing reconciliation introduces a governance dimension to payment authorization (e.g., a plan that constrains spend must validate against billing reconciliation rules), add a second criterion ID in S4 for that interaction when #2391 closes.

FM-A — Account payment required blocking active buys

Status: Not in the original candidate list. Identified during curriculum review as a real, high-frequency operational failure. The account state machine allows an account to transition to payment_required, which blocks new spend but does not terminate in-flight campaigns. Buyer agents that treat all authorization failures as transient will over-retry; agents that treat them as fatal will stop managing campaigns that can still be modified. Neither behavior is correct.

Module	Depth	Assessment approach
S1 (Media buy)	Diagnose / explain	Scenario: a `create_media_buy` returns `ACCOUNT_PAYMENT_REQUIRED`. A second call on the same account to `update_media_buy` (modifying an existing buy) succeeds. Learner (a) explains the distinction between new-spend authorization failure and modification-of-existing-commitment; (b) describes the buyer agent’s correct behavior — stop new buys, do not abandon existing ones, surface the payment status to the orchestrator.

FM-B — report_usage pricing mismatch

Status: Not in the original candidate list. Identified as a billing dispute trigger: the buyer reports a pricing_option_id in report_usage that does not match the rate negotiated at buy time, and the seller agent rejects it. This is operationally painful for CPA and performance-priced campaigns where the pricing option ID changes between the buy and the reporting call.

Module	Depth	Assessment approach
S1 (Media buy)	Diagnose / explain	Scenario: a `report_usage` call returns an error because the `pricing_option_id` in the report does not match the `pricing_option_id` accepted at `create_media_buy`. Learner explains where the authoritative pricing option ID is recorded (in the accepted buy response, not the original product listing), why reporting a different ID is a protocol error, and the correct recovery (re-read the buy, extract the accepted pricing option, re-submit the report).

Scope note: If #2391 introduces a formal billing reconciliation mechanism, this failure mode may merge with FM-3 into a unified settlement module. Flag for review when #2391 closes.

FM-C — adagents.json / brand.json authorization failure at discovery

Status: Not in the original candidate list. Identified as the most common onboarding failure for new integrations: a buyer agent discovers a sales agent via get_adcp_capabilities, attempts OAuth authentication, and the seller rejects the token because the buyer’s adagents.json does not declare the correct authorized_agents[] relationship — or the brand.json entry does not match the agent identity presented in the signed request.

Module	Depth	Assessment approach
D2 (Platform)	Resolve / demonstrate	Lab scenario: a buyer agent cannot authenticate to a newly integrated seller. Learner reads the seller’s error, checks the buyer’s `adagents.json` for the `authorized_agents[]` entry, verifies the `brand.json` agent identity, and traces the resolution chain. Correctly identifies which of the three common root causes applies: missing entry, URL mismatch, or expired authorization.
S1 (secondary)	Diagnose / explain	Learner explains what `adagents.json` and `brand.json` encode in the discovery and authorization flow, why a mismatch produces a rejection, and how to diagnose it from the buyer side.
S4 (secondary)	Diagnose / explain	Learner explains how a `brand.json` authorization failure interacts with the governance token chain — specifically, why a buyer without a valid `authorized_agents[]` entry cannot obtain a governance context token and what the escalation path is.

Open items tracker

Item	Status	Unblocks
Billing reconciliation schema (`#2391`)	Active / in-flight (3.1 scoping)	FM-3 depth expansion, FM-9 `#2391` flag
Idempotency coverage confirmed in training agent (`#2346`, `#2367`)	Shipped	FM-1 authoring (verify lab exercise extension only)
Lifecycle state stuck issues (`#1612`–`#1616`)	Open	FM-5 S1/S5 scenario detail

Module impact summary

The table below shows which specialist and practitioner modules need authoring follow-ups. Each ✦ is a new lab exercise or demonstration item; each ✧ is a new exam scenario or cross-reference.

Module	New lab exercises	New exam scenarios	Cross-refs to add
S1	FM-1 (idempotency states), FM-5 (stuck `pending_start`), FM-8 (TMP provider variant)	FM-3 (delivery reconciliation), FM-6 (webhook consumer), FM-A (payment_required), FM-B (pricing mismatch)	FM-9 → S4, FM-C (adagents.json)
S2	FM-2 (post-launch compliance — required for IACET), FM-5 (creative sync stuck)	—	FM-2 → S1 (state machine cross-ref), FM-9 → S4
S3	—	—	FM-8 (TMP provider surface)
S4	FM-4 (revocation-during-execution variant)	FM-9 (cross-protocol adjudication)	FM-C (adagents.json / governance chain)
S5	FM-5 (SI session stuck/expired)	—	FM-9 → S4
D2	FM-C (adagents.json/brand.json lab)	FM-7 (signed-request fail-closed)	—
D3	FM-6 (webhook delivery failure), FM-8 (TMP Router config)	—	—
B2	—	—	FM-7 (surface/recognize)
B3	—	—	FM-6 (surface/recognize)
C2	—	—	FM-2, FM-4 (surface/recognize)
C4	FM-1 (idempotency resolve/demonstrate)	—	—
A2	—	—	FM-1 (surface/recognize)

​Failure-mode competency scope

​FM-1 — Idempotency replay / conflict / expired

​FM-2 — Creative compliance failure post-launch

​FM-3 — Payment / settlement reconciliation differences

​FM-4 — Governance token mismatch / authorization revoked mid-lifecycle

​FM-5 — Lifecycle state stuck (media buy, creative, account, SI session, catalog)

​FM-6 — Webhook delivery failure / retries

​FM-7 — Signed-request validation failure

​FM-8 — TMP provider integration failure

​FM-9 — Cross-protocol policy conflicts

​FM-A — Account payment required blocking active buys

​FM-B — report_usage pricing mismatch

​FM-C — adagents.json / brand.json authorization failure at discovery

​Open items tracker

​Module impact summary