AI Systems Due Diligence: Notes from the Ground Up

 

A Conversation Nobody Was Having

For all the hype surrounding artificial intelligence, very few conversations were happening where it truly mattered… inside the decision rooms of acquisition teams and boardrooms. It wasn’t that people didn’t care about the risks. It was that the risk felt too complex to unpack. When I started embedding myself inside teams conducting evaluations of AI-heavy ventures, it became clear: AI systems due diligence wasn’t just missing… it hadn’t even been defined yet.

The software due diligence process had its checklists. The IP assessment had its legal frameworks. But due diligence in AI systems was slipping through the cracks. Teams kept asking the wrong questions. Not “how was the model trained?” but “is the interface intuitive?” Not “what biases might the algorithm be embedding?” but “can it scale easily to other markets?” These weren’t bad questions, just incomplete ones.

 

What We Miss When We Rush

One case stays with me. A mid-stage AI healthcare startup was being vetted by a cross-functional investment team. On paper, the tech looked solid… but no one had probed the ethics of the training data. Turns out, it was trained primarily on datasets from Western populations. The AI misdiagnosed patients from other geographies at alarming rates.

This is exactly why AI systems due diligence must exist as a standalone track. It's not a sub-bullet under technology assessment. It’s a parallel effort—one that looks at model explainability, drift risks, and transparency. You have to ask whether the developers understand the limitations of what they've built, or whether the system is a black box to them too. You also need to test how the model behaves under edge-case pressure or erratic real-world inputs.

 

Data Isn’t Just a Resource… It’s a Liability

Another field note from a fintech context: The team proudly presented an AI credit scoring model that had lowered default rates significantly. But when asked about retraining cycles or safeguards against data poisoning, they had no answer. They didn’t even have protocols for data integrity over time. The algorithm was brilliant… but brittle.

In AI systems due diligence, data pipelines and governance matter as much as the model architecture. Too often, people assume AI is static—develop once, deploy forever. But AI is dynamic. If you don’t maintain or adapt it, it will deteriorate or diverge from its original intent. Poor data hygiene, if left unchecked, becomes a silent risk multiplier.

 

The Gap Between Intuition and Insight

Something else became apparent after a few months in the field: teams were often over-reliant on gut feel. If a demo was slick or if the founders were confident, that seemed enough to get nods of approval. But AI doesn't care about your gut. It behaves exactly as it is trained. If your assumptions are flawed, your model inherits them.

This is where due diligence in AI systems acts as a calibration tool… cutting through the emotional fog and surfacing the hard truths. It grounds teams in verifiable data, reproducible evidence, and architectural clarity.

 

What Good Due Diligence Looks Like

When I see due diligence teams perform well, they’ve absorbed the mindset of a skeptic and a systems thinker at once. They want to know what the AI learns when it fails… how the feedback loops behave… how the retraining is managed across real-world deployments.

Good AI systems due diligence includes scenario testing—not just what happens when the system works, but when it doesn’t. What if the AI falsely flags 5% of legitimate transactions? What if regulators challenge the model’s opacity? What if a global data privacy law suddenly renders 40% of your training data unusable?