> Belfort today released the "so far" CIFAR demo, an encrypted implementation of ResNet-20, a popular model for image classification. It outperforms recent SOTA by 3x with a total latency of less than 200ms
Not many details on how they've done this, so I'm a bit skeptical. Fast HE is a holy grail.
> Belfort's image classification is built on top of its upcoming GPU library, Cyclops. It comes with several optimizations that make Cyclops extremely fast on Encrypted AI workloads.
Looks like a lead up to an upcoming library release
https://i.postimg.cc/90WGjk8t/results.png
What's the use case for this?
"The 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks."
I.E., gaining any sort of insight a transaction of protected information is what the protections were in place for.
So is FHE more about skirting regulations and privacy laws? Or, is it a new frontier of an untapped data source that has some red tape around it?
To me, something was simply not encrypted properly if you are able to draw conclusions/learn insights/detect anything about the data. It's in conflict with the idea of what secure encryption means to me.
The server providing the FHE-based service does not gain any sort of insight. This is a key point: only the client can see the output of the computation.
Where can I find out more about how this could be possible?
> Image classification without the server seeing the image.
The value is in being able to get the prediction from the model without ever giving unencrypted data to somebody you don’t trust.
You could have an LLM generate code for you without ever giving the operator your proprietary source code.
Here, you literally download client_secret.bin from their server, so they have control over the keys and evaluators. So two things. First, the per user key flow would be several minutes for per user keys, the evaluator bundle would be in the 100s MB to GB realm. Second, there's no way for us to tell the difference between them really doing FHE or decrypting with the key. To be clear, not evidence it's fake, just not total proof it's real. Really hope it's real, been a field I've been following for awhile.
Key management is a critical part of the story, as j2kun has pointed out before. In this demo, we're intentionally caching the same key material for every visitor so we can showcase the actual FHE computation without making everyone wait through client-side key generation and upload. Even as a one-time cost, having each user generate their own keys in the browser and transfer them to the server introduces noticeable setup time, which we felt would get in the way of the experience for a demo. We actually implemented it that way at first; setup wasn't a matter of minutes, but we changed course after worrying about the experience for visitors on mobile data.
This demo is focused on demonstrating the computation itself, not a production-grade key management flow. Supporting per-user keys and a more realistic trust model is definitely on our roadmap for future demos.
So it seems kind of reasonable to me that if you want to showcase your fast inference for its own sake, you can sidestep the remaining engineering problems around key management by giving the user pre-computed key material corresponding to hot-loaded evaluation keys.