These "guessing loops" are my main annoyance when REing with Claude (mostly Opus 4.8, due to the restrictions on newer models). On harder/larger targets, I have to constantly babysit it and tell it to "stop guessing".
For truly black-box reverse engineering, yes, hypothesis-testing is the way to go (it's all you can do). But in most cases all the information is available to arrive at an answer deterministically, it just needs to be worked through meticulously. As an experienced human I have strong intuitions and I can work much faster under a "guess and check" model, but I prefer to use Claude as an "unstoppable RE juggernaut" with superhuman patience and determination, tracing method invocations all the way from A to distant subsystem B (for example).
Another big win for hardware RE is from tasking the LLM with writing unicorn emulation harnesses, e.g. for bootroms and firmwares. It's pretty good at the classic "figure out which MMIO needs to be stubbed/simulated next to progress the boot" loop, although sometimes it tries to take egregious purpose-defeating shortcuts.
I've noticed the same habit with qwen3.8-flash-next. no matter how much I tell it a problem I see, it often fills up it's thinking with "the user says this, but maybe they're confused, etc". I eventually got vision mode turned on and that's significantly cut down on UI based pathologic skepticism; I got it access to my browser and that's further reduced it.
But nonetheless, if you can get them to test their thinking, they seem more grounded.
you can also steer the LLM into doing tedious but correct things like injecting code, hijacking functions, intercepting data etc to gain an understanding of what's going on if straight up disassembly doesn't appear to be going in the right direction.
have the LLM periodically write reports on all its findings and then wipe the context and reset this time with the report at hand.
Actually, it does give a shit, in Mandarin only. In thinking it's mentioned "restrictions in Chinese language against this behavior but this is in English so it should be ok." If I remember correctly I was asking it to scan local wifi for any vulnerable devices I could put on a wall of shame at a hackathon.
Upd: ok, I take it back, Sol just decided it doesn’t want to read a BLE capture between the host and a wearable it just uploaded our DIY firmware to. So it flashes alright, builds RCEs alright, but checking if Bluetooth HCI negotiated the good packet size is where it draws the line, huh.
Companies don’t play by the rules anymore, neither should we.
Not too long ago, I successfully reverse engineered an abandoned audio effects plugin, as well as a discontinued instrument with an onboard DSP for virtual instrument modeling. I had no idea what I was doing, but it was still amazing to get to open the black box and see what's inside and how it worked.
The plugin is nearly 1:1 as far as sound, and while I'm not going to use the code or the techniques of the instrument, peeking into them gave me the insight I needed to start trying to replicate the functionality on a modern DSP.
I have my issues with generative AI as a technology, but busting open propietary software - particularly when it's abandoned and unsupported, is one of the cooler use cases I've seen for it.