All posts

The regulatory trajectory for enterprise AI agents

Compliance work has a lead time, so an operator deploying agents needs the direction and timing of the rules, not just today's text. Here is the trajectory, and why auditability designed in early is cheaper than auditability retrofitted.

Javier Leguina

Javier Leguina

Co-founder & CTO of flowscope, previously a founding engineer at ModelML (YC W24).

· Operating agents in production

An operator deciding whether to put an agent into production this year is deciding against a set of dated obligations, not only against today's published text. The clearest dates are in Europe, and the most important one moved in June. Under the EU AI Act, Regulation (EU) 2024/1689, the prohibitions on certain uses took effect in February 2025 and the obligations for general-purpose AI models in August 2025. The substantive requirements for high-risk systems, covering risk management, technical documentation, human oversight, and post-market monitoring, were originally scheduled for August 2, 2026; the digital omnibus amendment, which cleared its final vote in June 2026, moved them to December 2, 2027 for standalone high-risk systems and August 2, 2028 for AI embedded in regulated products. Penalties for the most serious breaches reach the higher of 35 million euros or seven percent of global annual turnover. The amendment moved the date and left the required artifacts unchanged. The timeline matters more than the text because compliance work has a lead time, and the documentation an agent's classification requires cannot be assembled in the weeks before a deadline.

The Act sorts by function, not by vendor

The reason a horizontal market is relevant here is the Act's classification scheme. The European Commission does not regulate "AI agents" as a category. It regulates systems by the function they perform and the consequences of getting that function wrong. A system used to evaluate creditworthiness, to filter job applicants, or to determine eligibility for an essential service falls into the high-risk tier and carries the full documentation, oversight, and monitoring obligations. A system that drafts an internal summary, routes a ticket to the right queue, or reconciles two ledgers does not, in itself, meet the high-risk definition.

A single vendor shipping a single underlying technology therefore faces different obligations depending on where it deploys. An agent that posts journal entries during the month-end close sits outside the high-risk list. The same model family, pointed at a credit decision or a hiring screen, sits inside it. So the regulatory question for an operator is never "is this product compliant" but "what does this agent do, and what does that function require." The obligation is determined by the use, not by the software.

NIST and ISO are the de facto standard in the US

There is no federal counterpart to the EU Act, so US enterprises have converged on two voluntary instruments that procurement teams and counterparties increasingly treat as the expected baseline. The first is the NIST AI Risk Management Framework, published in 2023 by the National Institute of Standards and Technology. It is not law and carries no penalty, but US agencies and regulators reference it, and its four functions, govern, map, measure, and manage, have become the common vocabulary buyers use to ask a vendor how it controls a system. The second is ISO/IEC 42001:2023, the standard for an artificial intelligence management system. Unlike the NIST framework, ISO/IEC 42001 is certifiable, which means an organization can be audited against it and hold a certificate, the same way ISO/IEC 27001 functions for information security. A certificate is a procurement asset, because it lets a buyer's security team accept an independently audited result instead of running its own assessment.

The European high-risk requirements and the American voluntary standards require the same artifacts. Each one expects a documented record of what the system does, what data trained it and feeds it, how a human can intervene, and how the deployer watches it after launch. A firm that produces those artifacts for one regime has done most of the work for the other.

The US state patchwork is the moving part

The fastest-moving piece is at the state level, and it is the hardest to plan against because it changes session to session. Colorado's AI Act, SB 24-205, enacted in 2024, was the most developed example of a state imposing duties on developers and deployers of high-risk systems, with its own definitions of consequential decisions in areas such as employment, lending, and housing. Two sessions later the state repealed and reenacted it as SB 26-189, signed in May 2026, which swaps the duty-of-care and risk-management scheme for documentation and notice obligations around automated decision-making technology. More states introduced bills in the 2025 and 2026 legislative sessions. The right posture for an operator is to read the relevant statutes directly, because the definitions of "high-risk" and "consequential decision" differ from state to state and from the European text, and a secondary tracker will not tell you whether your specific use is covered. The pattern that holds across all of them matches the Act's: duties attach to the decision the system influences, regardless of the label on the software.

Why this is cheaper to design in than to retrofit

Every one of these regimes wants the same underlying set of records: what the agent did, which inputs it acted on, where a person reviewed or could have reversed it, and a log that survives long enough to answer a question months later. An agent that produces those records as a byproduct of running is auditable by construction. An agent that runs as an opaque process and is asked to become auditable before a deadline requires reconstruction, and reconstruction is expensive because the evidence was not captured when the action happened. We design the controls for autonomous actions so the agent records its own audit trail at the moment it acts, and we monitor the agent after it ships so the post-market record the Act asks for exists whether or not a regulator ever requests it. The same logging that lets a controller trust an agent's journal entries is the logging a documentation obligation later demands.

The function-based classification also works in the operator's favor here. Because obligations attach to the use, an agent deployed first on an internal, lower-stakes function builds the audit discipline before it is ever pointed at a regulated decision. The capture and oversight design does not change when the stakes rise. Only the level of scrutiny does.

A reasonable counter, answered

A reasonable counter is that building full auditability into every agent over-engineers the many that will never touch a regulated decision, and that an operator should match the control effort to the actual classification rather than treating an internal reconciliation agent like a credit-scoring engine. There is something to this. A ledger-matching agent does not need the conformity assessment a high-risk system does, and pretending otherwise wastes effort. But the expensive artifacts are not the conformity paperwork, which is genuinely use-specific. They are the action logs and the human-in-the-loop seams, and those cost almost nothing to capture at the moment of action and a great deal to reconstruct after the fact. The agent that records what it did and where a person can intervene reflects the observation-and-action discipline flowscope already builds, regardless of tier. The formal assessment is the part that genuinely waits for the classification; the logging and the oversight seams are the part you cannot add later.

This post is informational, not legal advice. The dates and penalties above come from the Regulation as amended and the European Commission; the standards from NIST and ISO; the state duties from the statutes themselves. The decision an operator can act on now, ahead of any deadline, is the one that costs little early and a great deal late: build the agent so it produces the record of its own actions from the start.