Blog/The state of enterprise AI

AI governance consulting: what it should deliver when AI runs in your operations

AI governance consulting should leave a company with operating controls on each AI system it runs, as well as a policy: an inventory, risk tiers set by what each system can do, controls on actions, monitoring, and a named owner for each system.

AI governance consulting should leave a company that runs AI in its operations with two things: a policy that says how AI may be used, and the operating controls that make that policy hold for each system in production. A governance engagement that ends at the policy leaves the harder half of the work undone, and the incident data shows the cost of both halves going missing. IBM's 2025 Cost of a Data Breach report, conducted by the Ponemon Institute across 600 breached organizations, found that 63 percent of them either had no AI governance policy or were still developing one, and that 97 percent of the organizations reporting a breach of an AI model or application had no AI access controls in place. The policy says who may approve what, and the access control is what stops a system from doing what nobody approved.

What AI governance consulting should deliver

The governance policy is the familiar deliverable. It sets out acceptable uses, the roles responsible for AI, a review committee, principles on fairness and transparency, and a process for approving a new use. A company needs one, and writing it is a few weeks of work. Operating controls are different in kind. They are mechanisms inside each system: a permission that limits what an agent can write, an approval step before a payment is released, a log that records every action with its inputs, a defined way to reverse an action, and a monitor that alerts someone when the error rate moves. A policy is checked by reading it. A control is checked by testing whether it operated.

Finance teams already work this way. An external auditor tests an internal control over financial reporting for whether it is designed well and whether it operated during the period, and a written procedure that nobody followed fails the second test. AI governance in an operating company should be held to the same standard, because the AI systems that matter most to a CFO are the ones that touch the ledger, the payment run and the customer record.

A complete AI governance consulting engagement therefore delivers five things for each AI system the company runs: an entry in an inventory, a risk tier, the controls that tier requires, monitoring in production, and a named owner. The policy sits on top of those and refers to them.

An inventory of the AI systems the company runs

The inventory comes first because a company can't tier or control a system it doesn't know about. For each AI system it records what the system does, which systems it reads from and writes to, what data it handles, which model and vendor it depends on, who owns it, and whether it acts on its own or only advises a person. The last field matters most. An assistant that drafts an email for a person to send and an agent that posts journal entries share a model and carry very different risk.

The inventory has to include AI the company didn't build. Many companies now get AI as features inside their ERP, their CRM and their support software, and employees adopt tools on their own. The same IBM study found that one in five organizations reported a breach due to shadow AI, meaning tools employees used without approval, and that only 37 percent have policies to manage AI or detect shadow AI. An inventory built from interviews with IT misses those systems, so the engagement should also review software contracts, expense reports and network logs for AI tools in use.

Risk tiers, set by what each system can do

Risk tiers decide how much control each system gets, and the useful basis for a tier is the consequence of the system being wrong. A practical scheme for an operating company has three tiers. The first holds systems that read and suggest, such as a tool that summarizes contracts or drafts replies, where a person reviews every output before it has any effect. The second holds systems that act within limits, such as an agent that matches invoices, posts entries below a threshold or sends payment reminders. The third holds systems that influence a decision about a person, such as credit, hiring or eligibility, where regulators have set their own rules.

The frameworks a consultant will cite sort systems the same way, by what they do. The EU AI Act classifies systems by function, and its high-risk obligations cover uses such as hiring and credit scoring. Those obligations aren't in force yet: an amendment adopted in 2026 moved them to 2 December 2027 for standalone high-risk systems and 2 August 2028 for AI embedded in regulated products. Most back-office uses, such as accounts payable or reconciliation, sit outside the high-risk list. The NIST AI Risk Management Framework is voluntary guidance, and ISO/IEC 42001, the management-system standard for AI, can be certified against, with certification also voluntary. None of the three sets an error threshold or an exception-rate target for a company's own processes. The company sets those, and the governance engagement should write them down per tier. We follow the dates and the state laws in the regulatory trajectory for enterprise AI agents.

Controls on actions: approval, audit trail, reversibility

Systems in the second and third tiers change records, and the controls on them are the core of the engagement. Three controls carry most of the weight. The first is approval: the agent proposes and a person approves any action above a set amount, any new supplier, any change to bank details, and any case outside the rules the agent was given. The approval limits should match the delegation of authority the company already applies to its staff. The agent should also hold only the permissions its task needs, since an agent that reconciles a bank feed has no reason to be able to delete records.

The second control is an audit trail. Every action the agent takes is logged with its inputs, the evidence it used, the version of the agent that acted, and the person who approved it, and the log is kept long enough to answer an auditor's question months later. The third control is reversibility. Each kind of write has a defined reversal, such as a reversing journal entry, and actions that can't be reversed, such as a released wire, are held for a person to release. We describe how these controls are built in controls for autonomous actions.

Monitoring and accountability after go-live

Controls that worked at launch weaken as the inputs change. Suppliers change their invoice layouts, a new ERP release changes a screen, and a model update changes how the agent handles an edge case. Monitoring catches those changes by tracking the exception rate, the error rate found when a person reviews a sample of completed work, and the volume of cases sent to people, and by alerting the owner when any of them moves beyond a set range. We cover how that is done in monitoring an agent after it ships.

Accountability is the last deliverable and the one most often left vague. Each system in the inventory needs a business owner who answers for its output, usually the person who owned the process before the AI did, such as the controller for an accounts payable agent. It needs a technical owner who keeps it working, a review on a fixed schedule, and an incident process that says who is told when the system gets something wrong and how the error is corrected. A governance committee can review tier changes and new systems, but the committee can't own forty systems in practice, and the named owners can.

A reasonable counter, answered

A reasonable counter is that a mid-market company with a handful of AI uses doesn't need any of this, that a one-page policy and a training session are enough, and that the controls are the software vendor's job. For tools that only advise, such as a chat assistant that drafts emails a person sends, a policy and training may be enough. Once a system writes to the ledger or releases money, the controls it needs are the ones the company already applies to employees with that access: approval limits, segregation of duties and an audit trail. The company's auditor will ask for them whichever party built the system. A vendor can build the controls into its product, and the buyer should require that in the contract, but the company remains accountable for what its systems do, and the EU AI Act places duties on deployers as well as on providers. The size of an AI governance engagement should follow the number of systems that act on the company's records, and for most mid-market companies that number is small enough to govern well.

Common questions

What does an AI governance consultant do?
An AI governance consultant helps a company decide how it will control the AI systems it uses and then puts those controls in place. The work starts with an inventory of every AI system, including AI features inside purchased software, and a risk tier for each based on what the system can do. It continues with controls matched to each tier, such as approval limits, action logs and a way to reverse what a system wrote. A complete engagement ends with monitoring running in production and a named owner for each system.
Are the NIST AI RMF and ISO/IEC 42001 mandatory?
No. The NIST AI Risk Management Framework is voluntary guidance from the US National Institute of Standards and Technology, organized into four functions: govern, map, measure and manage. ISO/IEC 42001 is a management-system standard that an organization can be certified against, and certification is voluntary. Neither sets error thresholds or exception-rate targets, so the company using them sets its own. Of the three frameworks a governance consultant will usually cite, only the EU AI Act is law.
What is the difference between an AI governance policy and AI controls?
A policy is a document that says how the company may use AI, who approves new uses and which principles apply. A control is a mechanism inside a system that makes the policy hold for that system, such as a permission that limits what an agent can write, an approval step before a payment is released, or a log of every action. A policy is checked by reading it, and a control is checked by testing whether it operated. A company needs both, and the controls are the part an auditor can test.

Book a demo

Start with one department

We map how one team’s work runs, show you where the hours go, and automate from there.