Why You're Almost Certainly Not Doing Threat Modeling Right

Most programs fail at model structure before scaling becomes the problem. A team can apply STRIDE-LM competently, although the ‘LM’ is often dropped, and still get weak results when the model omits the relationships, identities, authorization decisions, deployment boundaries, and data characteristics that determine risk.

At one extreme, a single Lambda icon stands in for a hundred services. The arrows do not say which identity is used, where authorization happens, or what data moves. At the other extreme, one feature is modeled down to individual endpoints and queues, then left in a document that cannot be connected to the rest of the product.

Both approaches limit what reviewers can see. A useful program has to preserve enough detail for local analysis and enough structure to reason across the whole product.

Your model is too high-level

A box named backend and an arrow named HTTPS are architecture shorthand. They are not enough for a security review. One Lambda may stand for many functions with different triggers, execution roles, network paths, data classifications, and downstream permissions. Collapsing them into one symbol also collapses the distinctions that determine risk.

Granularity should be based on what changes the security conclusion.

Component metadata: environment, cloud account or project, owner, runtime, network exposure, workload identity, effective privileges, data classifications, tenancy assumptions, dependencies, frameworks, and criticality, etc.
Flow annotations: initiator, caller identity, credential type, authentication method, authorization decision point, protocol, data carried, synchronous or asynchronous behavior, trust boundaries crossed, and so on
Boundary metadata: tenant, account, identity domain, region, administrative plane, employee access path, third-party environment, etc.

Two flows can both use HTTPS and have completely different security properties. One may use mutual TLS with a unique workload identity and a narrow destination policy. The other may use a shared bearer token copied into an environment variable. A queue consumer that uses a local role is different from one that can traverse a cross-account role chain. When the model draws these relationships the same way, it hides the attack path.

Trust boundaries need the same precision. Internal is not a useful boundary by itself. Production account, tenant boundary, identity domain, control plane, third-party environment, and employee administration path each carry different assumptions. If the model does not state those assumptions, the reviewer has to invent them.

You cannot visually reason about what could go wrong until the model shows what is actually there.

Your model is too low-level

The usual correction is to model one feature in detail. That can produce a good review and still fail as a product practice. The model sits in a ticket or diagramming tool, uses local names, and has no durable connection to the services, identities, accounts, and data systems used elsewhere.

Product risk crosses feature boundaries. A security team should be able to ask:

Which public entry points can reach regulated or customer data after several service hops?
Which products depend on the same identity provider, deployment role, build system, encryption key, event bus, or administrative API?
Which workloads can assume roles into production accounts, including through delegated or transitive paths?
Where can a failure in one cloud account, region, control plane, or third party cross an apparent product boundary?

Opening hundreds of diagrams one at a time will not answer those questions. One enormous product diagram will not answer them either. It becomes unreadable and still lacks the local detail needed for a real review.

You need model chaining and a global view

Keep the feature-level detail, but make the models composable. A feature model should reference shared services by stable identifier and version. A service model should reference its identity, deployment, data, and operational models. The product-level representation should resolve those links into a graph.

For example, a checkout feature may depend on a payments API, which depends on a workload identity, a key, an event bus, and a finance account. Those dependencies should be references, not copied boxes. When the authentication platform or role policy changes, the system can identify the affected feature models and run the relevant analysis again.

AWS recommends feature-level threat modeling, reuse of existing component models or controls as dependencies, and integration with normal delivery workflows.^[1]

Model chaining also prevents each feature team from maintaining its own stale copy of the same platform. Shared assumptions, controls, and unresolved findings can be inherited, while feature-specific context stays local.

The global view should be a queryable graph, not a poster. It should support reachability, blast-radius, shared-dependency, privilege-chain, and model-coverage queries. Diagrams are views generated from that graph for a specific review.

You are modeling the application and missing the system

Many threat models concentrate on application logic and treat infrastructure as a container around it. That misses some of the most important paths in a cloud product: workload identity, CI/CD permissions, role assumption, secrets, network controls, storage policies, orchestration platforms, administrative APIs, logging, backups, recovery systems, and third-party control planes.

An API can enforce business authorization correctly while its deployment role can modify unrelated production resources. A workload can have a narrow runtime role and still inherit a broad cross-account path through the build system. A private service can be unreachable from the application network and still exposed through an administrative plane.

These are not secondary implementation details. They determine what an attacker can do after the first compromise. A threat model that stops at the application boundary cannot describe lateral movement or account-level blast radius.

You need multiple views

Teams often respond to complexity by creating separate diagrams at different levels. That helps only when the diagrams are views over one coherent model. Independently maintained diagrams drift.

Different reviewers should be able to work across application, identity, infrastructure, cloud-account, data, operational, and whole-product views without creating different versions of the system.

Each view should use the same component identifiers and relationships and display only the detail needed for that review.

Your model is out of date

Annual or biannual threat-modeling workshops are not frequent enough. The next release is where the change is and six months later the team is reviewing a diagram of a system that no longer exists.

Model updates should be part of feature review and release work. A material architecture change updates the canonical model; the model diff then determines which checks and reviews need to run. Not every release needs a full workshop. The review can be targeted to the change.

Changes that should trigger analysis include a new external interface, a trust-boundary crossing, a service identity, an authorization policy, a sensitive data type, a privileged dependency, a third-party service, a cloud account, an egress path, a tenant boundary, or an administrative capability.

AWS recommends doing threat modeling during the design of a feature or feature change and putting the work into the same sprint, backlog, and delivery tooling used by the engineering team.^[1]

An annual review still has a place. Use it to audit coverage, stale dependencies, unresolved findings, ownership, and whether the program is working. It should not be the main mechanism for keeping the model current.

You are underleveraging agents

Agentic threat modeling works now for model construction, maintenance, and first-pass analysis. Given structured inputs and access to evidence, agents can extract components and flows from infrastructure-as-code, deployment manifests, API definitions, code, diagrams, and cloud configuration; reconcile names; identify missing metadata; run targeted reviews; produce a grounded first pass of threats; verify mitigation evidence; and compare one release with the next.

This removes much of the manual setup and first-pass review work. It is not enough for autonomous risk acceptance or unreviewed release approval.

Recent spatial-reasoning gains matter because architecture review is relational and often starts with visual evidence. Blueprint-Bench 2, released in May 2026, asks agents to reconstruct apartment floor plans from roughly twenty photographs. As of July 2026, the leading model scored 0.386 on a normalized scale, compared with 0.145 for Claude Opus 4.8 and 0.586 for the reported human baseline. The benchmark measures spatial reconstruction rather than security analysis, but the relevant capability is clear: models are materially better at recovering topology and relationships from images.^[2]

Use specialized agents, not one large threat-modeling prompt

The work divides naturally. One agent can build or reconcile the model. Others can focus on identity and service-to-service authorization, cloud accounts and control planes, data and privacy, business abuse cases, or mitigation verification. A separate critic can review evidence and logic without inheriting the generator’s assumptions.

The Claude Agent SDK supports programmatic agent definitions with separate prompts, tools, skills, model choice, and reasoning effort. Each subagent starts in a fresh conversation and returns only its result to the parent. The documentation includes an AgentDefinition example and explicit support for context isolation and specialist instructions.^[3]

Fresh contexts matter. The identity reviewer should spend its context on identities and authorization paths. The privacy reviewer should not carry the cloud review’s intermediate discussion. The critic should receive the proposed finding and evidence, not the conversational momentum that produced it.

Give those agents maintained skills

A specialist agent needs more than a special name— a unique agent definition is needed when you require a bundle of specific instructions, special tools, a particular way to handle complex data, or when you need a specific persona with guard rails and tailored prompt for a specific domain.

The June 2026 SkillsBench v4 evaluation covered 87 tasks and 18 model-and-harness configurations. Curated skills raised the average pass rate from 33.9 percent to 50.5 percent, a gain of 16.6 percentage points. Gains ranged from 4.1 to 25.7 points by configuration. Expert-curated, focused skills can materially improve results, but the skill, model, and harness still need paired evaluation for the actual task.^[4]

This supports distributed ownership. The privacy team can maintain privacy templates and standards. The cloud team can maintain agents and skill pertaining to Cloud patterns. GRC can maintain control mappings and evidence requirements. Product security can own the shared schema, orchestration, validation rules, and overall quality bar.

Do not force an unsupported answer

Prompts that demand ten threats, a complete report, or a value in every field create pressure to fill gaps with invented architecture. The agent needs an explicit way to report that the evidence is incomplete.

Use only the supplied model and linked evidence. Cite component IDs, flow IDs, policies, or artifacts for each finding. Label each statement as OBSERVED, INFERRED, or UNKNOWN. Do not invent an identity, control, boundary, policy, or data flow. When evidence is missing, return UNKNOWN and state the question to resolve. Retract any finding that cannot be supported.

Anthropic’s guidance on reducing hallucinations recommends explicitly allowing the model to say it does not know, grounding claims in source material, requiring citations, and retracting claims that cannot be supported.^[5]

An unknown is useful output. ‘The model does not identify the workload identity used on flow F-218’ is a concrete security gap and a clear request for evidence.

Put deterministic checks and validation around the agents

A large part of model quality is deterministic. Run those checks before asking an agent to generate threats:

Every flow has a valid source and destination.
Every trust-boundary crossing identifies authentication and the authorization decision point.
Every sensitive asset has an owner, classification, and permitted processing context.
Every public entry point maps to a workload, identity, and policy.
Every service identity maps to effective permissions, including inherited and cross-account paths.
Infrastructure evidence and the model agree on deployed resources, exposure, and relationships.
Graph queries flag unexpected reachability, privilege transitions, and shared failure domains.

Validate generated findings as well. Referenced IDs must exist. Evidence must support the claimed preconditions and consequence. Severity must follow the program’s schema. Duplicate findings should be merged. A proposed mitigation should address the mechanism described in the finding.

Use separate generation and validation stages. Parallel specialists can review the same target from different perspectives. A critic can challenge the reasoning and evidence. A judge can score a finding against a narrow rubric. Anthropic documents routing, parallelization, voting, orchestrator-worker, and evaluator-optimizer patterns for this kind of workflow.^[6]

LLM-as-judge research reports strong agreement with human preference in some evaluation settings, along with position, verbosity, and self-enhancement biases. Use judges for narrow, calibrated checks; keep risk acceptance with security reviewers.^[7]

With structured input, specialist skills, deterministic checks, and a validation loop, an agentic pipeline can do most of the mechanical work and much of the first-pass analysis. Calling that ‘85 percent done’ is reasonable only after the program defines 85 percent of what and measures it.

Build the eval suite before trusting the pipeline

Measure the outcomes that matter: critical-threat recall, grounded-finding precision, false-positive review cost, severity calibration, mitigation quality, model completeness, and reviewer time. Weight the results by impact. A system that finds most low-severity issues and misses a cross-tenant authorization failure is not usefully summarized by one average score.

The eval set should contain representative architectures, known findings, planted model defects, ambiguous evidence, and cases where UNKNOWN is the correct answer. Useful mutations include removing an authentication annotation, broadening an IAM policy, adding a cross-tenant flow, changing a data classification, introducing a privileged dependency, or creating drift between infrastructure evidence and the model.

Run repeated trials, retain the full traces, and maintain both capability tests and regression tests. Use deterministic graders where possible, model-based graders where necessary, and human reviewers to calibrate them. Anthropic’s agent-evaluation guidance describes tasks, trials, graders, outcomes, traces, and a stable harness, and recommends exactly that mix of grading methods.^[8]

Every prompt, skill, agent definition, model version, or orchestration change should run against the suite before it reaches the release path.

Federate ownership without fragmenting the program

A central product-security team usually cannot author and maintain every model. It becomes a queue. Full decentralization also fails when every team uses a different schema, threat language, and quality threshold.

Ownership can be split as follows:

Product teams own the accuracy of their feature models and update them with releases.
Domain teams own specialist assets, skills, validators, and eval cases for privacy, identity, cloud, data, GRC, and other areas.
Product security or a center of practice owns the canonical schema, shared threat taxonomy, orchestration, common checks, release policy, and program reporting.

Product-security engineers stay in the loop for high-risk changes, ambiguous findings, and risk decisions. Routine model maintenance and first-pass analysis no longer have to wait for a central workshop.

Keep control over model selection and agent patterns

AI-based threat-modeling SaaS products can provide useful ingestion, workflow, and reporting features. The diligence question is how much control they leave with the security organization.

A routine completeness check may need a small, fast model and deterministic rules. A new multi-tenant authorization design, privileged control plane, or complex identity chain may justify a frontier reasoning model, higher reasoning effort, parallel specialists, debate, a society-of-mind pattern, a generator-critic pair, or several validation passes. The right pattern depends on the target and the cost of a miss.

Before putting a managed service at the center of the program, determine whether you can control or inspect:

Model provider, model version, reasoning effort, and routing by target complexity.
Agent definitions, prompts, skills, tool permissions, and specialist composition.
Deterministic validators, evidence sources, and rules for accepting or rejecting findings.
Critic and judge configuration, validation loops, and human approval points.
Eval suites, trial results, traces, regression history, and behavior after model upgrades.
Evidence provenance, data retention, data residency, and export of the canonical model.

Those choices affect coverage, reproducibility, and assurance. The security organization should be able to test and govern them rather than accepting a fixed black-box workflow.

A workable standard

High-level models hide risk inside boxes. Isolated low-level models trap risk inside feature reviews. Application-only models miss identity and cloud paths. Stale models describe systems that are no longer running.

The program needs granular, linked models; a global query layer; several views over the same source; updates during feature review and release; deterministic checks; specialist agents; validation loops; and an eval suite. Product and domain teams maintain their parts. Product security owns the shared structure and the decisions that require security judgment.

The result is product-wide threat analysis without flattening the detail needed to find real attack paths.

References

^[1] AWS Security Blog, “How to approach threat modeling.” Feature-level modeling, reuse of component models, distributed ownership, and integration with delivery workflows. Accessed July 17, 2026.

^[2] Andon Labs, “Blueprint-Bench 2.” Spatial-reasoning benchmark and July 2026 leaderboard. Accessed July 17, 2026.

^[3] Claude Code Docs, “Subagents in the SDK.” AgentDefinition, fresh contexts, specialist prompts, tools, skills, model selection, and reasoning effort. Accessed July 17, 2026.

^[4] Li et al., “SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks,” arXiv:2602.12670v4. Across 87 tasks and 18 model-and-harness configurations, curated skills increased average pass rate by 16.6 percentage points, with substantial configuration-level variance. Accessed July 17, 2026.

^[5] Claude Platform Docs, “Reduce hallucinations.” Allowing uncertainty, grounding claims, using citations, and retracting unsupported claims. Accessed July 17, 2026.

^[6] Anthropic, “Building Effective AI Agents.” Routing, parallelization, voting, orchestrator-worker, and evaluator-optimizer patterns. Accessed July 17, 2026.

^[7] Zheng et al., “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.” Agreement with human preferences in benchmark settings and documented position, verbosity, and self-enhancement biases. Accessed July 17, 2026.

^[8] Anthropic, “Demystifying evals for AI agents.” Tasks, trials, graders, traces, capability and regression suites, deterministic grading, model grading, and human calibration. Accessed July 17, 2026.