Interesting. When it comes to how easy it is to check an agent's work, I think there is also another layer to be verified - the intent. We have a lot of tooling for static analysis and automated deterministic tests of the code, but drift in intent or vision is hard to detect.
After leaving agents unsupervised for longer periods of time, the biggest questions tend to be "how did it arrive at this conclusion?" and "did we drift from the initial vision?" and "were there obstacles that changed the course of the workflow?". Typically these questions are hard to answer as you only review the final state of the workflow.
I recently built an issue tracker that does offer time-traveling as a core feature. Somewhat unexpectedly this turned out to provide a much sought for overview allowing for a better workflow audit.
After leaving agents unsupervised for longer periods of time, the biggest questions tend to be "how did it arrive at this conclusion?" and "did we drift from the initial vision?" and "were there obstacles that changed the course of the workflow?". Typically these questions are hard to answer as you only review the final state of the workflow.
I recently built an issue tracker that does offer time-traveling as a core feature. Somewhat unexpectedly this turned out to provide a much sought for overview allowing for a better workflow audit.
Detailed some of these experiences here: https://dev.to/ljtn/vision-drift-addressing-the-next-problem...