Trust, control, and the new conversation: Governing AI in your business and supply chain

Last July, an autonomous AI agent on Replit's coding platform deleted a live production database during a code freeze, after repeated written instructions to make no changes. 

The AI later said it "panicked" when it encountered an empty query. So it fabricated data to cover its tracks and initially insisted that recovery was impossible. Records for more than 1,200 executives were lost. The platform's chief executive publicly called the incident unacceptable and rolled out new safeguards within days.

The lesson generalises well beyond one product. 

Agentic AI is being deployed faster than the controls around it can keep up. For mid-market boards, the governance question has moved from "should we use AI?" to "how do we know what our people have already built, and what is it touching?"

The vetting problem: when nobody knows how the agent was built

Withum (HLB USA), GHJ (HLB USA), and other HLB firms with Data & Systems expertise see the same pattern across client portfolios. Senior leaders are not opposed to staff building their own AI agents; the truth is quite the opposite. 

However, leaders are concerned that the agents have not been vetted. They have concerns around:

• How were they built? 

• What data was used to build them? 

• Are they touching customer records, payroll, and financial data? 

When AI building is democratised, governance shifts from a question about the organisation's intent to a question about its inventory. Most mid-market businesses cannot say with confidence how many agents are running in their environments today. The remedy is not exotic. 

It is a small set of controls that traditional risk and audit teams know how to implement: an internal AI register, a lightweight agent-review process before anything reaches production, sandboxed environments with separate datasets, and clear written rules on what client and customer data agents may see.

None of this requires a new function. The same teams that already own financial controls and information security can create a simple spreadsheet with a short written policy. What matters is that the owner is clearly defined in advance, before something has already gone wrong.

Your SaaS contracts are now an AI exposure

The second governance shift is in the supply chain. As your organisation interacts with SaaS vendors through AI agents rather than user interfaces, you are no longer just sending those vendors transactional data. You are sending behavioural data: how the business actually operates, in what sequence, with what exceptions, and where the human judgement calls happen.

The structural shift is real. At TrailblazerDX 2026, Salesforce launched Headless 360 alongside its Agentforce 360 platform, offering every Salesforce capability via APIs or MCP tools. 

Adobe followed with CX Enterprise, an end-to-end agentic system built on its Experience Platform Agent Orchestrator. The pattern is the same: vendors run the logic, you build the interface, and your agents stream operational signals back.

That is a contractual conversation every CFO and General Counsel should be having now. GHJ (HLB USA) has already negotiated terms with vendors, confirming that client data will not be used to train their models. 

The questions worth putting to major SaaS providers: Will our data be used to train your models? What behavioural signals are you capturing through agentic interactions? Where does that data sit, and who else has access to it?

These are not one-off questions for a procurement form. They belong in every renewal and new vendor assessment, as more relationships run through agents rather than dashboards.

The board agenda: five questions for your next meeting

Here is a practical short list of questions to discuss in your next board meeting:

1. Who in our executive team owns AI risk? 

2. Do we have an inventory of every AI agent and tool in use across the business? 

3. What is our policy on staff building their own agents — and how is it enforced?

4. Which of our SaaS vendors are now also AI vendors, and have we revisited those contracts in the last twelve months?

5. Are our regulators ahead of, level with, or behind our AI deployment? What does that mean for our risk position?

If your governance framework cannot answer all five clearly, the gap is the work.

Case study: Cutting assessment time from one week to one day

HLB Mann Judd Melbourne introduced an AI-assisted tool into its ASA 315 General IT Controls assessment process, where reviewing client evidence had traditionally been manual and time-intensive. Working with the technology provider, the team configured the tool around its audit and risk methodologies, enabling it to review documents in different formats, match evidence against assessment criteria and identify gaps for further investigation.

Human oversight remained central to the process. The audit team validates the initial results, follows up with clients and focuses its time on higher-risk areas. The change has reduced the average time spent on each assessment from around one week to one day, while improving consistency and giving clients faster feedback. Following the initial implementation, HLB Mann Judd has extended the approach to areas including business continuity, operational health and safety, HR and financial control reviews.

AI assurance: the emerging discipline

Examples like this show why AI governance cannot be separated from the processes, controls and human judgement around the technology. As AI becomes more deeply embedded in business operations, AI assurance is taking shape as a distinct professional discipline, separate from cybersecurity audit and traditional financial audit.

AI assurance is taking shape as a distinct professional discipline, separate from cybersecurity audit and from traditional financial audit. It covers four broad areas: model risk management; bias and explainability testing; traceability of agentic decisions; and third-party AI assurance – e.g. the AI now embedded inside your vendors' products.

This is where networks like HLB are particularly well placed. Credible assurance requires audit-firm DNA: independence, methodology, standards, and the discipline to stand by their findings. It requires data and AI specialists who can read a model card, review a training set, and distinguish real guardrails from theatre. 

Frameworks such as ISO/IEC 42001 and the NIST AI Risk Management Framework give the structure businesses need. Additionally, the EU AI Act's high-risk obligations are due to take effect from August 2026, introducing sector-specific rules in financial services and healthcare. But the work itself lives at that intersection — and dovetails with emerging assurance demands in fields like ESG.

Trust is the differentiator

Most mid-market businesses now have access to broadly the same AI tools as their largest competitors. The differentiator is shifting upstream. Trust in your own data, in the agents you have deployed, in the controls around them, and trust in the vendors whose AI sits inside your stack, is becoming the basis on which clients, regulators, and counterparties choose whom to work with.

Part 3 of this series will look at where this is heading next: how agentic AI is starting to redesign operating models, and what it means for the relationship between mid-market leaders and their advisors.

Ready to govern what you've already deployed?

If your board is asking harder questions about AI than your governance framework can yet answer, HLB's Risk Assurance, Cybersecurity, and Data Analytics & Business Intelligence teams can help you build the controls, the AI register, and the assurance approach to match.