The first sign of trouble at a logistics firm was not a breach. It was a dashboard nobody could explain. An operations lead had spent a weekend with an AI app builder, wired a prototype into the shipment database, and shared the link with three colleagues who shared it with nine more. The tool worked beautifully. It also held a read-write connection to production data, and nobody on the technology team knew it existed.
Stories like that are becoming ordinary, and they are not really about AI. They are about control. When the cost of building software drops to an afternoon, the number of people building software rises sharply, and the assumptions that used to hold an IT estate together stop holding. Someone used to review the code. Someone used to grant the credential.
Technology risk management answers those questions on purpose instead of by accident. It is unglamorous work: assessments, access policies, audit trails, testing. Done well, it is also what lets a company say yes to AI quickly, because the guardrails already exist.
Start With an Honest Risk Assessment
Every AI project should begin with a boring question: what can this thing reach? Not what the demo showed, but the concrete inventory of systems, tables, and files it will touch once real people use it. A support assistant reading a public help center sits in a different risk tier than one reading tickets full of payment details.
A workable assessment covers four things and does not take weeks. Name the data the system sees and classify it. Describe what happens if the output is wrong, and separately, what happens if it is wrong and nobody notices for a month. List who can trigger it. Then decide, in advance, what would make you turn it off. That last item is the one teams skip, and a kill switch nobody agreed on is a debate rather than a control.
The international frameworks converge on roughly this shape. The OECD AI Principles, now adhered to by 47 jurisdictions and updated in 2024, put accountability, transparency, and robustness at the center rather than treating them as optional polish. Borrowing that vocabulary is often what a nervous executive conversation is missing.
Access Policies Decide How Far a Mistake Travels
Most AI incidents inside companies are not clever attacks. They are over-broad permissions meeting ordinary curiosity. The prototype got an admin credential because that was faster than requesting a scoped one, the credential stayed, and six months later the prototype is load-bearing.
Least privilege sounds like a cliche until you price out the alternative. Give each AI system its own identity rather than a borrowed human one, so you can revoke it without firing anyone. Scope database access to the tables the use case needs and nothing adjacent. Put the tool behind the same single sign-on as the rest of your stack, because an access review that misses half your applications is theater.
Platform choices matter here. Teams building internal tools on infrastructure designed for governed vibe coding inherit identity, permissions, and audit logging from the platform rather than reinventing them per project, which is usually the difference between a policy that exists and a policy that holds.
Audit Trails Give You a Story You Can Reconstruct
When a traditional application misbehaves, you read the logs. When an AI feature misbehaves, you need more: the input, the output, the model version that produced it, the retrieved context, the prompt in force at the time, and the identity of whoever was on the other end. Miss one and you cannot reproduce the failure, so you cannot honestly claim to have fixed it.
Retention deserves a decision rather than a default. Logs that vanish after seven days will not help with a pattern that emerges over a quarter, and logs keeping raw customer inputs forever become a liability themselves.
Governance frameworks have wrestled with this longer than generative AI has existed. ISACA's COBIT framework maps control objectives across the technology estate, including a focus area for information and technology risk, and has since been extended with guidance for AI system governance. Reusing a mature control structure beats inventing a parallel one, not least because auditors already understand it.
Test Systems That Do Not Behave the Same Way Twice
Conventional software fails predictably, which is why conventional tests work. A model can return a different answer to the same question on Tuesday, and a vendor can swap the underlying version without telling you much.
Build an evaluation set of real cases with known-good answers, including the awkward ones, and run it before every change to the prompt, the retrieval layer, or the model. Add adversarial inputs, since somebody will eventually paste instructions into a form field to see what happens. Keep a human in the loop for consequential decisions, and release to a small group first. All of it fails the same way: teams do it once at launch and never again.
Treat Risk as a Number You Can Argue About
Risk conversations go badly when they stay purely qualitative, because this feels risky and this is fine are not comparable positions, so the louder person wins. Rough probabilities and rough costs change the discussion even when the numbers are crude, since the assumptions become visible and somebody can challenge them. The mechanics are worth understanding: see this breakdown of how markets turn uncertainty into usable estimates.
Shared language helps as much as shared math. ENISA's work on an interoperable EU risk management framework exists because organizations assessing the same risk with different methods cannot compare notes, and that problem scales badly across a supply chain.
The companies handling AI well are rarely the ones with the strictest rules. They are the ones whose rules are cheap to follow. When requesting a scoped credential takes ten minutes, people request scoped credentials. When logging is a platform default, every tool is logged.
The alternative is not a safer company. It is the same adoption happening invisibly, on personal accounts and unreviewed credentials, with governance deferred until something breaks in public. Prohibition does not slow AI down inside an organization. It just moves it somewhere you cannot see.
So treat risk management as part of the adoption plan rather than a tax on it. Inventory what your AI systems touch, scope their access, log what they do, test them like the probabilistic systems they are, and write down what would make you stop. That is what turns AI from a pile of experiments into capability the business can depend on.