Bitcoin Core is some of the most reviewed code in the world. We were curious how our frontier AI model would handle it. Last month we ran 16 agents across 5 of the top models for a week to look for issues in Bitcoin Core v32.0rc1 Impressively, they surfaced two minor items - both now disclosed responsibly. Agentic teams have been a huge unlock so far for our security efficiency. They're also remarkable to watch.
I've been running agents for almost every aspect of my life, all with my own AI harness. Link below if you want to experiment with it to work on tough problems! www.mymaestroai.ai
@JLunglhofer two minor findings in code this heavily reviewed is a strong signal. did any model catch something the others missed?
@defipoodle I saw significant overlap between Kimi K3, Mythos 5.1, and OpenAI's Daybreak Red / Cyber Unlocked Fable. The magic was less about "one model" being better and more making them all work together as a team.
@JLunglhofer Ha Ha Ha.... Bitcoin Core has a most awful flaw in that it opened up the Op_return field to spam in the Core V30 refresh... you can't fix STUPID. This is not a coding flaw, it's a philosophical flaw that has ruined Bitcoin.
@0xJoshAlpha Not many. My methodology mandated that all findings get challenged by all the other agents using other models and that any "vulnerability" be subject to chaining, exploit development and verification. Then I ran multiple rounds with a different set of agents to lock in ratings.
@JLunglhofer @0xJoshAlpha thank you for your service can you tell us what were the vulns you found to patch?
@ArgenTo46 @0xJoshAlpha Not "vulns" but control issue, e.g. checks that could be circumvented under specific circumstances. Not critical, but shouldn't be possible. I'm not willing to disclose details until I'm sure they've been fully patched - even then I may leave it up to the BTC devs.
@JLunglhofer Which models you did use and at which effort level? Was Claude Mythos used as well for reviewing it?
@JLunglhofer A great example of how agentic AI is changing the way security teams find and validate vulnerabilities!
@JLunglhofer Multi-agent review of Core only turning up two minor items is quietly one of the more useful security experiments this year. The review cycle just got a lot shorter.
@JLunglhofer 16 agents across 5 models for a week and all they found was two minor items lol the humans probably did more in an afternoon
@JLunglhofer Interesting test. Only two minor items says a lot about how well reviewed that code already is.
@JLunglhofer Finding two minor items in code this heavily reviewed is a satisfying bit of SI detective work
@JLunglhofer Did you audit the crypto libraries like libsecp256k1 and whatever other libraries all those other coins use?
@JLunglhofer Now let's add utility to the code so the network does something other than track the movement of Bitcoin and mining.
@JLunglhofer @R89Capital AI vs. heavily reviewed Bitcoin code is a fun test. @PeterPrunt is my go-to for stock takes if you're curious.
@JLunglhofer @JWWeatherman_ still people won’t use core, meanwhile it’s been reviewed by largest entity and has most eye balls.
@JLunglhofer It's very imprssively! Does it implicitly imply that nearly all new chains, without much code-reviewed as what BitcoinCore has done, will be in a mess later, more or less?
@JLunglhofer @emiliemc You and @88krxxs are my go-to for Bitcoin takes—curious how the AI review turned out.
@JLunglhofer two minor items in bitcoin core is honestly impressive. how much of the week went to triage
@JLunglhofer Two findings in code that heavily reviewed is a solid week for 16 agents! Once agents run on every repo, I'd track how fast each finding gets an owner and a decision, because that's where the queue builds.
