- Released
- A customer-facing chatbot our founder built at a different company, live on a working site with real customer data behind it.
- Later
- Someone probed it with injection attempts, trying to make it execute code and leak what it knew.
- Result
- It refused, logged the attempt and flagged it for a person. No data was exposed.
- Why
- Every check it hit was designed before launch, when the system was still a diagram.
AI Safety Audit
Find out what your AI does on its worst day.
We make that day happen on purpose, in one bounded place, before your customers become the test case. Two weeks. Fixed scope. Priced as a project.
-
Before
The system as it runs today, with a case it has never seen.
-
Test running
We make that case happen on purpose, in a bounded place.
-
Checked
We open it up and write down exactly what held and what didn't.
-
Before
The system as it runs today, with a case it has never seen.
-
Test running
We make that case happen on purpose, in a bounded place.
-
Checked
We open it up and write down exactly what held and what didn't.
Why we lead with the audit
We'd assumed someone would try.
That assumption is what we sell. The checks this chatbot hit were written months before anyone attacked it.
What we check
Eight ways an AI system fails quietly.
Each one ends up in the report as found, not found, or not testable, with the reason.
If you are earlier than an audit and still working out whether any of this applies to you, the broader version is written up here.
Data exposure
What it sees that you didn't intend.
Prompt injection
Can it be talked into ignoring you?
Human oversight gaps
Decisions with nobody's name on them.
Failure modes
What it does on its worst day.
Logging and audit trail
Can you reconstruct what happened?
Data consent
Are you allowed to feed it that?
Compliance
Can you prove it to an auditor?
Model selection
Is your model defensible?
The gap we look for
The inputs nobody rehearsed.
A demo runs the path someone planned. We send the others: the customer who's angry, the order left half-finished, the message written to trick it, the question it has no data for. Then we write down what it did.
What you get
A list you can act on the same week.
- Risk assessment report
- Remediation roadmap, in fix order
- Governance framework
How we work
- Data flows are mapped before anything is connected.
- Sign-off on anything high-stakes stays with your team.
- If we find something uncomfortable, it goes in the report anyway.
The EU AI Act sets duties for high-risk AI systems.
Risk management, data governance, logging, human oversight and monitoring after launch. Small B2B tools often fall outside the high-risk category. Working out which side you're on is the first thing we check.
Questions we get
Before you ask.
What is an AI safety audit?
Two weeks reviewing where an AI system leaks data, gets manipulated, or fails invisibly. It ends in one written risk report.
How is it different from a penetration test?
It examines the AI layer, not the infrastructure: what the model can be talked into, what data it reaches, and how it fails.
Do I need an audit if my AI agent is already live?
That's the most useful time for one: real users and real data are already behind it, and every week it runs is a week of exposure you can't see.
How much does it cost?
Fixed scope, not a retainer. The price depends on the systems and data sources in scope.
What do you need from us?
Agent configurations, an hour with whoever runs the system, and its data sources. Under three hours of your time.
Does the EU AI Act apply to my business?
It depends on what the system decides and for whom. Many small-team tools sit outside the high-risk category, with lighter transparency duties. Working out which side you're on is the first thing we do.
Have us try to break it.
Tell us what's running and who built it. We'll reply by email with what an audit would cover.