From Prompt Injection to Policy

By David Vohradsky, Founder and CEO, MyRISK. Written for the CyberCon 2026 Think Tank.

01The request

A large Australian organisation with federated IT: tens of thousands of users, an extended R&D ecosystem, a decentralised technology estate. Its cyber function was not immature — it held a policy with delegated authority to the CISO and a working standards set covering risk, data security, IAM, asset management, secure by design, vulnerability management, logging, incident and vendor risk.

AI arrived the way AI usually arrives: from several directions at once, mostly through products already bought. A tenanted copilot in the productivity suite. Retrieval-augmented assistants in both the business and IT. Staff building their own GPTs, technologists pulling models and datasets from public repositories as daily routine, business users experimenting with what the vendors were now calling agents.

The question that reached the cyber team was bounded and reasonable.

“We are adopting AI. Can you write us an AI security policy?”

That sentence contains an invalid assumption: that AI security is a discipline that can be written down separately, and that the writing is the work. Dismantling it took most of a three month engagement.

Six authorities, six different kinds of object

Before a policy could be written, a control set was needed. There was no shortage of guidance — there was an oversupply, and none of it written as an enterprise control catalogue.

One describes techniques. One describes vulnerabilities. One describes lifecycle activities. One describes governance outcomes. One describes national threat posture. One separates threats from mitigations. These cannot be compared one-for-one, and a framework that treats them as six comparable catalogues inherits all six structures at once.

One sequencing decision made the difference between a usable framework and an unusable one: each source was normalised on its own terms before comparison. Normalising means reducing each item to the security outcome it is actually asking for, in neutral language, so that a requirement is compared against a requirement rather than a heading against a heading.

Read with the MITRE list open, OWASP LLM06 on excessive agency looks covered by an agent-permissions mitigation. Decomposed first, it contains at least six separable requirements — minimise available tools, minimise functions within each tool, avoid open-ended functions, execute in the user’s context, require human approval for consequential actions, and enforce complete mediation in the downstream system rather than relying on the model to restrict itself. The last of those is an application-security control that would have vanished on a first-pass match. Secure output handling behaved the same way: OWASP requires deterministic handling of model output before it reaches an interpreter — HTML encoding, SQL parameterisation, command-injection prevention — which a broadly related validation mitigation does not deliver.

A mapping demonstrates overlap. It does not prove control equivalence.

02Where it went wrong

The framework was extensible, which turned out to be the problem. Every authoritative publication could plausibly add ten or twenty controls, and publications were arriving every few weeks.

The same control, six times. Least privilege for AI agents appeared as privileged-agent permissions, single-user agent permissions, tool permissions, backend access controls, runtime policy enforcement and zero-trust model execution — six named controls from three sources expressing one policy requirement. Supply-chain integrity fragmented across SBOM, AIBOM, dataset provenance, artefact verification, code signing and point-of-retrieval validation.

The MEASURE sinkhole. Almost every security control mapped cleanly to NIST MEASURE, which blurred implementing a control with evaluating whether it works. The correction was to attach preventative treatments to MANAGE and their testing and telemetry obligations to MEASURE.

Consolidation destroyed precision. Grouping overlapping controls looked cleaner immediately. A verification pass against source requirements showed what had evaporated inside broad wording: short-lived agent credentials, safe model-loading flags, two-person approval for the highest-risk actions, parameterised queries, Content Credentials, embedding-inversion protections, hallucination logging, white-box and black-box adversarial testing. An implementer reading the consolidated framework would have built none of them.

Broad controls hid gaps. “Monitor AI systems” overlaps a great deal of guidance while failing to say that monitoring must look for repeated probing, unusual query distributions, adversarial-input patterns, model-extraction behaviour or deterioration of the mitigations themselves.

Partial coverage is more dangerous than absent coverage, because absent coverage appears on a gap register and partial coverage does not.

Meanwhile the original instinct — a standalone AI Security Standard — was failing its own test. AI API security was still API security. AI credentials were still identity and secrets management. Agent permissions were still access control. AI incidents were still security incidents. A separate standard would have duplicated half the existing framework and split ownership of every control in it.

03The low point: the mapping, not the controls

Semantic consolidation at this scale is not hand work. Six sources, several hundred extracted requirements, each needing comparison against every plausibly related requirement in a growing catalogue, inside three months. That workload gets delegated to a language model, and in this engagement it was. The decision was correct and would be made again. What was wrong was the status given to the output.

The AI-assisted mapping was treated as a record. It was a proposal.

The instance that exposed it: a requirement in the high-risk action control group — that the highest-risk agent actions must require dual control — had been allocated to identity and access management and characterised as a requirement about granting or changing access. A short rationale was attached, and it read well.

It was wrong. Dual control means two authorised parties participate in, or independently approve, a sensitive action. It is adjacent to access control; it is not access provisioning. Allocated that way it would have acquired the wrong owner, the wrong assessor and the wrong mechanism — and the actual requirement, that an AI workflow must not complete a high-consequence transaction alone, would have been owned by nobody.

What made this a low point rather than an erratum is how it surfaced. Not in review — reviewers read the mapping artefact, and the artefact was internally coherent. It surfaced only when someone returned to the normative source wording for an unrelated reason. The policy draft, the clause allocation and the gap register had already accepted it.

Why AI-assisted mapping fails this way

• It compares compressions, not requirements. The model forms a paraphrase of each requirement and compares paraphrases. Paraphrase is lossy exactly where it matters: qualifiers, scope conditions, and the difference between an action and the authority to perform it.
• Plausibility is the optimisation target. A wrong mapping arrives with a confident rationale. Human mis-mapping looks like a mistake; model mis-mapping looks like a judgement.
• Terms of art are the trap. Dual control, segregation of duties, four-eyes, maker-checker and access approval occupy adjacent semantic space with different security outcomes. So do integrity, validation, provenance, privilege, grounding and attestation.
• It fails consistently, not randomly. A tired analyst produces scattered errors that sampling finds. A model applies the same misreading to every structurally similar requirement — systematic, correlated, and invisible to a sample check that assumes independence.
• Negative findings are least reliable. A “covered” finding retires risk and closes a line on the register. A “not covered” finding creates work and gets challenged. The claim with the greatest consequence receives the least scrutiny.

What to do about it

• Quote, never paraphrase, in the mapping artefact. Every row carries the verbatim normative wording.
• Assign stable control identifiers at extraction. Identifiers survive paraphrase; titles do not.
• Scrutinise “covered” harder than “gap”. Invert the natural review instinct.
• Re-derive a sample independently from the source text alone, with no sight of the first mapping. The disagreement rate is the trust metric for the whole artefact — and because errors are correlated, use a separate pass rather than asking the same process to check itself.
• Target human verification at terms of art, authority relationships and coverage claims. Verifying every row is unaffordable; verifying those three classes is not.
• Record the rationale so it can be attacked. A mapping without a stated reason can only be accepted, never reviewed.

Use AI to propose the mapping. Do not let it own the record.

04Three ideas that made it tractable

A threat is not a control

Prompt injection, model inversion and data poisoning are risk events. An assessor cannot ask whether an organisation has implemented prompt injection. The framework must describe what the organisation does to prevent, detect, contain or recover from the event.

A mitigation is not necessarily a standalone control

Differential privacy, adversarial training, watermarking, model ensembles, homomorphic encryption, federated learning and randomised smoothing are valid mitigations. They are not seven policy controls. They belong beneath broader objectives, applied where risk and architecture justify them. Together these two ideas produce the hierarchy the rest of the framework hangs from.

RAG, copilots and agents are not three sizes of the same thing

Prompt injection behaves differently in each architecture. In a RAG assistant it influences the information returned. In a copilot it influences recommendations and generated work, with the human still the actor. In a tool-enabled agent, the same technique influences an action, executed against real systems under inherited authority.

The risk moves from “what can the model say?” to “what can the system cause to happen?”

That is a change of category, not a change of severity, and it reframes the control model. The conventional picture is a line — user, model, output. The agentic picture is a chain in which every hop is a trust boundary, and none of them is enforceable by the model itself:

user → agent → model → context/memory → tool → API/data source → other agents → action

So “least privilege for AI” is not one control. Privilege must be considered separately for the user, the agent identity, the tool, the connector, the retrieval source and the downstream service. Architecture determines applicability: a simple copilot does not need every agent control, which is why the framework needs a common core with architecture-specific overlays rather than one undifferentiated “GenAI risk” bucket.

Put the determinism at the boundaries

AI should be probabilistic in reasoning, and deterministic at the point where security boundaries and material transactions are enforced.

This is an engineering statement, not an aspiration, with implementations at four layers:

• Code layer — schema-first validation. Every model output crosses a typed boundary: Pydantic in Python, Zod in TypeScript, JSON Schema in strict mode for tool contracts, with constrained decoding tightening the same boundary upstream. The security property is not accuracy. It is that failure is forced to be a rejection rather than a misinterpretation.
• Tool layer — mediation outside the model. An allow-listed tool registry, explicit scopes, parameter validation, destination restrictions, and authorisation checked against the acting identity at the point of call. The agent may request anything; the tool layer decides.
• Transaction layer — assume the reasoning was wrong. Idempotency keys, replay protection, value limits, approval gates, circuit breakers, rollback where reversal is possible and compensation where it is not.
• Business layer — a deterministic record of a probabilistic decision. Capturing, at the time, the inputs, retrieved sources, model and prompt version, rationale, approving identity and resulting action. The reasoning need not be reproducible for the decision to be reconstructable — and reconstructability is what a regulator, auditor or complainant actually asks for. This is the problem MyRISK Trace is built to solve.

The general form is the same at every layer, and it is precisely the discipline arrived at for the mapping process in Section 03: let the model propose, and make something else decide, record and constrain.

05Rebuilding

The control-convergence rule

Source controls were not discarded; their role changed. They became cross-references and detailed requirements beneath roughly fifteen policy-level control domains, each carrying its objective, its generic risk, an organisation-specific risk scenario in plain language, and full provenance.

A new source should normally create new mappings and new requirements — not new controls. A new control is the exception, justified only by a materially distinct security objective.

Every extracted requirement is classified covered (the source becomes another provenance reference), partially covered (the control statement is enriched) or missing (a new control, or a recorded decision not to create one). The framework gets stronger without getting larger. Note the dependency on Section 03: the rule is only safe if the covered classification can be trusted.

Map in both directions

A gap analysis assumes the external standard is more complete. Reverse mapping showed otherwise: the internal framework treated agentic workflows, runtime policy enforcement and application-layer output controls more explicitly than the draft ISO standard did. This is a statement about timing, not superiority. ISO 27090 is strong on the classical adversarial-ML problems while the practical enterprise problem has moved to delegated authority.

Standards provide anchors. Threat-informed engineering has to fill the temporal gap between standards.

Tested against three publications that arrived mid-project, the rule produced three different outcomes. ISO/IEC DIS 27090 was mostly a precision overlay — sharper language on lifecycle attack-surface mapping, mitigation-interaction testing and model memorisation, with zero trust, DLP and prompt injection already present. ASD Frontier Model Guidance added operating-posture controls: threat-horizon monitoring, accelerated patch posture, AI-assisted defensive vulnerability discovery, vendor assurance. ACSC agentic AI guidance was the only one that justified genuinely new controls: cryptographic agent identity and runtime attestation, agent separation of duties with delegation expiry, graduated autonomy with rollback, goal-drift monitoring, tool-description integrity.

Architecture with more than one view

No single diagram represents enterprise AI governance adequately. Four views were separated: the operating model (who governs), the lifecycle model (when governance occurs), the control architecture (what must be controlled) and the runtime architecture (where controls are enforced during execution). Within the control architecture, not every layer is vertical.

The purpose of architecture is not to find one box for every control. It is to make control coverage and control boundaries visible.

Agent identity, against a moving target

An interactive copilot assists a user inside an authenticated session. A persistent agent authenticates separately and continues acting after the session ends. A digital worker represents a function that outlives the staff associated with it. Treating all three as service accounts produced poor accountability.

This was the one area requiring realignment mid-engagement. The original design carried three types: user-controlled AI, a Delegated Identity acting on behalf of a named user within an approved scope, and an Agent Identity performing an organisational role. Microsoft Entra Agent ID subsequently settled on a different cut — agent identities distinguished by who provisions and who sponsors them, giving a Microsoft-provisioned and an organisation-provisioned Agent Identity, plus an optional Agent’s User Account pairing one-to-one and immutably with an agent identity where mailbox or presence functions are required. The framework was realigned, because an identity control that does not match the identity platform is unimplementable. The delegated-identity concept survives as a scoping and sponsorship requirement rather than a separate type.

Where a control depends on a vendor construct that is still moving, keep the control objective stable and let the taxonomy be an implementation mapping.

The durable objective — every agent has an accountable owner, an approved scope, reviewable authority and a defined fate when its sponsor leaves — did not change when the vendor model did. Written as the taxonomy, the standard would have needed reapproval within a quarter. Two further principles followed: an agent-to-agent handoff must not create an implicit expansion of authority, and risk assessment follows capability, not product identity. The latter was validated immediately by Model Context Protocol access appearing in SaaS products: a tool that previously generated text can suddenly reach external systems, records and business processes while the asset inventory shows no change at all.

06The return

The original request was an AI security policy. The framework was now good enough to write one. It was not written.

AI controls belong in the control domain that already owns the underlying risk. A separate AI requirement remains only where the control addresses a genuinely AI-specific risk that cannot be expressed through an existing domain.

Mapping at control-family level proved too coarse — agent authority alone crossed IAM, secure by design, logging and incident management. So the rule became: map requirements, not control headings, and validate each against the existing numbered clauses rather than the headings.

AI requirement → existing numbered clause → gap, if any → proposed new clause

Each control was then assigned a single primary home on the criterion of primary security intent — a control that belongs everywhere belongs nowhere. Controls with no convincing home were deliberately left unmapped rather than absorbed into a plausible-sounding general clause, because that produces false assurance. Clauses were written for an IT professional with three to five years of experience: the conventional security requirement first, AI terminology as examples, unfamiliar terms in a controlled glossary. Numbering was made assessable — three digits is an enforceable clause, never a heading.

R&D freedom was preserved through scope rather than exemption. Testing deliberately vulnerable models, investigating prompt injection and evaluating models of uncertain provenance are legitimate development activities. Controls were therefore scoped with qualifying language — where justified by risk, for medium and high risk AI capabilities, where used for operational or consequential purposes.

The control question is not whether a system is unsafe. It is whether the organisation has started relying on it.

What was actually approved

The engagement ended with a memorandum rather than a framework. Four existing standards were amended and approved, effective immediately.

Note what is not in that table. There is no AI Security Standard, no parallel governance regime, and no separate AI assessment process for system owners to learn. Prompt injection, retrieval poisoning, agent authority propagation and model supply-chain compromise entered the control environment as amendments to four documents that already existed.

The sequence, if you are starting on Monday

threats → risk scenarios → external guidance → control consolidation → control architecture → governance architecture → policy and standards → assurance

Beneath it sit six artefacts, each with one job that should not be conflated with any other: a Charter that mobilises the programme; a Framework that defines the governance and control architecture; a Policy that establishes mandatory requirements; Standards and patterns that define implementation; a Control library holding detailed requirements, verbatim source wording and provenance; and Evidence and assurance demonstrating that controls operate. If the architecture cannot be explained simply, it has not been separated cleanly enough.

Finally, the framework must learn at roughly the rate of the technology it governs. Annual review is not that rate. Update on triggers: a major model capability release, new government guidance, new attack evidence, a material vendor change, a new agent tool integration, a new data class entering retrieval, or an increase in autonomy.

Conclusion

Moving from prompt injection to policy requires more than adding AI terminology to existing security standards. The key task is abstraction: understand the technical risk deeply enough to identify the required security outcome, place that outcome in the existing control architecture, translate it into a durable requirement, assign clear ownership, and scope it proportionately.

The result is not an AI security policy sitting beside cyber security. It is a cyber security framework that has evolved sufficiently to govern AI.

Questions for the Think Tank

Is a standalone AI security standard ever the right answer? How much of your control mapping was done by a model, and how would you know if it was wrong? Which of these controls can realistically be decentralised? How do you assess an agent that changes capability faster than your assessment cycle? Where is the honest line on human approval before it becomes a rubber stamp? And are we writing assurance requirements for things almost nobody can currently test?

Based on a three month engagement to build an AI cyber control framework for a large organisation with federated IT and translate it into enforceable policy. Draws on a consolidated control framework mapped to MITRE ATLAS, OWASP LLM and GenAI guidance, the NIST AI RMF, ASD and ACSC guidance and ISO/IEC DIS 27090, and on the Q3 2026 cyber security standards amendment. Individual officers are referred to by role. ISO/IEC DIS 27090 is a draft international standard and is used as an alignment reference, not a certification baseline. A longer version of this paper, with full roadblock and resolution tables, is available on request.