Feedback wanted: making EU AI Act accuracy/logging claims verifiable (open standard)

Hey all, hope this is on-topic. I've been building an open standard (PRML) for one narrow piece of the AI Act puzzle: making a high-risk system's accuracy/performance claims tamper-evident.

The gap I keep seeing: Article 15 (accuracy) and Article 12 (logging) require providers to declare and evidence performance, but an accuracy number is trivial to produce after you've seen the test results. On paper it says 95%; nobody downstream can tell if 95% was the bar set in advance or the outcome chosen afterward. That's the difference between a claim and evidence.

PRML borrows pre-registration from clinical trials: you commit the claim (metric, threshold, dataset, seed) to a SHA-256 hash before the run. Edit it after seeing results and the hash breaks, in the open, for any auditor to check. Free and open-source (CC BY 4.0), with an in-browser verifier (no account) and integrations into common eval tools.

falsify.dev (verify a manifest in the browser: registry.falsify.dev)

I'd genuinely value feedback from people who've been through (or advise on) real AI Act compliance:

  • Does "verifiable, pre-registered evidence" map to how you'd actually evidence Article 15/12 for an audit, or is that not how it works in practice?
  • What would make this useful vs. just academically neat?

Not selling anything (it's open/free). Down the road I'd look for design partners on the deployer side, but right now I'm just pressure-testing whether the approach survives contact with how compliance actually gets done.

Posted Jun 3, 2026, 1:15 PM UTC

513
Loading comments…