A variation on that theme that I would like to try is forking a worker per user (or per session), which would be sandboxed (e.g. with Capsicum in FreeBSD) and only have access to that user's credentials. A separate process would route requests to these per-user workers and provide them with the appropriate credentials. This could then be combined with PostgreSQL's row-level security, with each user having their own user in PostgreSQL, and the per-user worker only having access to that particular user's database credentials.
I feel like this should significantly limit the consequences of both SQL injection vulnerabilities and RCE vulnerabilities. It wouldn't solve stored XSS, I suppose, but I believe that is handled by using a template language for HTML (one that encodes parameters automatically on output). It also probably wouldn't scale to a planet-scale mass-market web application, but there are lots of apps that don't need that kind of scale but do need to be secure.
Maybe this sort of thing has already been done and written about (by humans)?
We've gone from talking about giving agency to AI (agents) to us having some agency over them.
How the turn tables!
Although I personally give more weightage t to "Creator of xyz" part than the "pre AI" part.