| Dimension | Score |
|---|---|
| Output reliability | 3/5 |
| Workflow fit | 4/5 |
| Context handling | 2/5 |
| Learning curve vs. payoff | 4/5 |
| Failure transparency | 2/5 |
| Overall | 3/5 |
What AI does well
First-pass policy comparison. I fed Claude and ChatGPT Enterprise two versions of an internal AML policy and asked for a redline summary. Both produced an accurate list of substantive changes in under a minute, a task that used to take me 20 minutes of manual diffing. The caveat: neither tool flagged a renumbered clause that changed which section a control referenced, so I still had to check cross-references by hand.
Regulatory change monitoring. Compliance.ai tracks rule updates from bodies like the SEC and FCA and pushes a daily digest. It caught a FinCEN guidance update two days before our internal legal alert did. It still needs a human to decide whether the update actually applies to our license type, since it flags by keyword, not by relevance.
Drafting first versions of policy language. Harvey and Claude both write a workable first draft of a data-retention policy or a vendor due-diligence questionnaire from a short prompt. This saves real time on boilerplate. The output still needs a compliance officer to check it against the actual regulatory text, because the model sometimes cites a rule number that does not match the clause it describes.
Summarizing long filings. ChatGPT Enterprise handled a 90-page consent order and gave me an accurate summary of the obligations and deadlines. That is a genuine win for triage. It is not a substitute for reading the document before you sign off on a remediation plan.
Where it fails
Cross-document consistency. I asked Claude to check whether a new supplier contract conflicted with our existing code of conduct. It confirmed no conflicts. There was one: the contract allowed a data-sharing arrangement our code of conduct explicitly prohibits. The tool had both documents in context and still missed the conflict, because it treated each clause locally instead of checking it against the full policy set.
Citation accuracy on evolving regulation. All four tools I tested (ChatGPT Enterprise, Claude, Harvey, Compliance.ai) occasionally cited a superseded version of a rule. On a GDPR question, Harvey referenced guidance that had been updated six months earlier. If you are not already tracking the current version, you will not catch this.
No audit trail for its own reasoning. None of these tools reliably explain why they concluded a document was compliant, only that it was. For a function where you need to show your work to a regulator, that is a real gap, not a minor inconvenience.
My take
My take (August 2026, Bernat Sampera)
AI earns a place in compliance work, but only in the first pass. It reads faster than I do and it never gets bored halfway through a 90-page filing. That is genuinely useful. What it does not do is hold the whole regulatory picture in its head the way an experienced officer does. It checks a clause against itself, not against every other policy, contract, and jurisdiction that clause touches. I compared ChatGPT Enterprise against Claude on the same test set and neither one caught the planted cross-document conflict. My verdict stands: use these tools to save time on the first pass, never on the sign-off. If your compliance program depends on catching the one clause that contradicts everything else, you still need a person reading the whole thing.
What I would build
The unsolved problem is not summarization, it is memory across documents. A compliance officer holds dozens of policies, contracts, and regulatory updates in their head at once, and knows that a change in one ripples into three others. No current tool keeps that web of relationships persistent between sessions: each query starts from the documents you paste in, not from the accumulated state of your whole policy library. A real solution would index every governing document once, keep track of which clauses depend on which others, and flag a conflict the moment a new contract or policy draft touches a clause with existing dependencies, without the officer having to remember to ask.
Verdict (August 2026, Bernat Sampera): AI cuts first-pass policy review time for compliance officers by roughly half, but in my tests it missed 2 of 5 planted conflicting-clause errors across ChatGPT Enterprise, Claude, and Harvey, so nothing ships without a human sign-off. Overall: 3/5.