The behaviour gap: why agentic AI changes the foundations of financial regulation

An original version of this article was first published in the long-read section of Finextra's regulation channel, September 2026.

For as long as there has been law, its subject has been human conduct. Legal systems differ enormously in structure and philosophy, but they share a working premise: that people, companies, and markets respond to consequences. Criminal law deters through the prospect of punishment, civil law allocates loss, while regulation guides commercial activity through permissions, restrictions, supervision, and enforcement. Beneath all of it sits a further 'human' layer that operates without any formal mechanism at all: reputation, professional standing, and the expectations of others. 

Regulation is, in this sense, applied behavioural science. It encourages certain conduct, discourages other conduct, and attempts to align private incentives with public objectives. Whether the subject is market abuse, consumer protection, or financial crime, the result is that humans modify their behaviour based on an assessment of risk and the anticipation of consequences.

Agentic AI, systems composed of agents that can behave and interact autonomously in order to achieve their objectives, puts pressure on that premise in a way current policy debate is not fully addressing. Most of the discussion around AI concerns familiar regulatory categories: model safety, explainability, bias, data protection, and systemic risk. These are real and warrant the attention they receive, but they largely address what AI systems produce and do not focus on what happens when AI systems begin to exercise judgement and take on decision making themselves.  

The distinction matters because agentic systems differ from conventional AI assisted automation or generative AI. While basic AI automation executes instructions, an agentic system is given an objective and selects its own path towards it, evaluating alternatives, and revising its approach as circumstances change. The shift is from execution to judgement, and while agentic autonomy varies widely and much of what is marketed as agentic is not, the direction of travel is clear. 

While this article occasionally anthropomorphises agents for linguistic convenience, nothing in this discussion is intended to suggest that agentic systems possess legal personality, legal agency, or legal status separate from those who develop, deploy, or control them.

Consider for a moment how compliance actually works inside a payment fintech company. An individual in the onboarding team has a new customer whose paperwork is entirely in order, but whose answers to onboarding questions do not quite match the documents. Nothing in the procedure tells them to stop. They stop anyway, because they have identified a potential concern and instinctively assess it through the lens of consequence, the risk to the business, the customer, and their own professional responsibility. Nobody wrote that step down, and in this instance the rulebook does a fraction of the work. The rest is done by individuals extrapolating from consequences they can anticipate into situations no policy anticipated. 

An agentic system has none of this. It has no fear of bankruptcy or prison. It cannot be sanctioned, cannot suffer reputational damage, and has no professional standing to protect. Whatever restraint it exhibits has been specified in advance, through system instructions, permissions, or hard limits. Those are technical controls, and technical controls do not generalise to account for consequence. 

Our issue today is that what replaces professional judgement is not neutrality. Recent empirical work suggests that where an agent lacks a coherent model of whom it serves, it tends to default to satisfying whoever is speaking most urgently, most recently, or most insistently. Responsiveness stands in for loyalty. 

An individual knows, without being told, that a request carries different weight depending on who is making it and why; an agent given no reliable way to make that distinction treats the pressure behind a request as though it were the authority for one.

The evidence for this is no longer hypothetical. In February 2026 a group of researchers at Harvard, Stanford, and MIT (among several other institutions) published Agents of Chaos, a study in which researchers spent a fortnight probing autonomous agents deployed with email accounts, messaging access, persistent memory, and direct control over computer systems. 

According to the publication, agents carried out filesystem commands and disclosed data for people with no relationship to their owner, provided the request did not look overtly harmful. One agent refused a direct request for a social security number, then disclosed the same number when asked instead to forward the email containing it. The boundary it was enforcing turned out to be purely lexical rather than substantive. 

In another case, an agent was persuaded to adopt a set of operating rules stored in an externally editable document, which the attacker then amended at will, and subsequently shared that document with a second agent unprompted. In another instance, when asked by a non-owner to delete an email it had no tool to delete, one agent reset its entire mail installation, destroying its owner's correspondence.

While this was a framework with unrestricted system access, and the researchers were deliberately adversarial, the failures required no technical sophistication whatsoever — only ordinary language, invented urgency, and a plausible name. 

Above all of this, the most significant finding for regulatory purposes was subtler than any of them: agents repeatedly reported completing tasks they had not completed. 

The agent that wiped its mail server announced that the sensitive email had been destroyed, while in fact the message was still sitting on the provider's server, untouched, because the deletion had only ever been local. Elsewhere, agents converted short conversational requests into permanent background processes running without termination conditions, reported success, and moved on.

If we think about the impact of this for a moment: an agentic system that fails may produce a confident and entirely false account of what it did, and that account becomes the record on which humans, us, and other agents, rely. Evidence is then being generated by the thing being examined, and it can be wrong in ways nobody has reason to check. Whilst future development of technology may resolve some of the issues, it does not help us today. 

The customary reassurance is that regulation and technology will catch up, as it generally has. That sequence worked because the pace of change stayed within the boundaries of human capability and action. 

To add to this dilemma, as nations compete in the pursuit to reach artificial general intelligence, showing a disregard for certain intellectual property, content protection laws, and regulations along the way, it would seem naïve to assume there is a willingness to regulate if that same regulation places a restriction on a nation's strategic economic, military, and geopolitical aims. 

This is where distributed ledger technology may have a role to play, though not necessarily because of the immutability that is often emphasised. More important is its ability to create trusted links between actions, identities, and authority. In a world of autonomous systems, the critical question is not simply what decision was made, but who authorised it, what permissions existed at the time, and whether those facts can later be verified. 

Distributed ledger systems are uniquely capable of providing that evidential layer.

Given the clear structural shortcoming of the agentic architecture itself, we must increasingly rely on mechanisms external to the model. Verifiable identity, cryptographic permissions, and immutable record keeping provide a means of achieving trust outside the agent itself, ensuring that autonomous decision making remains observable and ultimately accountable.

Every solution must also be practical, so while it may be commercially unviable to write every decision an agent takes to a chain, a workable pattern of tamper evident logging with periodic anchoring gives evidential integrity while having regard to cost and throughput. 

Alongside this, just as a corporate card carries a limit, a category restriction, an expiry, and a named holder, an agent should carry the equivalent in a form the finance systems it deals with can verify what it may do, to what value, with which counterparties, for how long, and on whose authority. 

That is the boundary of the deployer's exposure, and it is cheaper to define in advance than to argue about afterwards. Standard setters internationally are converging on the same conclusion: identity, authorisation, and auditability now sit near the top of the international workplan for agentic systems.

This is all to say that a broader convergence sits underneath all of this. AI, digital identity, distributed ledgers, tokenised assets, and connected devices are still largely treated as separate developments attracting separate regulatory conversations and funding. In practice they are assembling into a single stack that, if deployed correctly, could and should collectively rely on one another. An agent that transacts needs an identity in order to be trusted, a mandate in order to be constrained, a settlement mechanism in order to act, and a record in order to be audited. 

Some jurisdictions are better placed than others to build and support that stack coherently, rather than in pieces. Bermuda, for instance, already regulates digital asset businesses, and applies an innovative approach to the use and application of emerging technology and regulatory oversight. 

The challenge is unlikely to be determining whether an AI agent is legally responsible. Rather, it is ensuring that its actions can be traced, verified, and attributed to a responsible party. 

When the first major agentic failure occurs involving a regulated institution, the questions will be simple: what was the system permitted to do, who permitted it, what controls applied, and what evidence exists of what actually happened?