Vibecoding an inference chip
58 points by ankitvgupta 3 hours ago | 19 comments

cpldcpu 2 hours ago
I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

It seems implementing transformers is the new "hello world" of AI assisted coding.

reply
datadrivenangel 2 hours ago
Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
reply
risyachka 2 hours ago
Fpga is software-driven, model can verify its output easily.

Zero surprise it did it

reply
datadrivenangel 2 hours ago
it only cost them $10,000 in Claude API usage. :)
reply
hmate9 2 hours ago
You get that much usage in a monthly $200 subscription.
reply
dvh 2 hours ago
So who's paying the cost?
reply
stronglikedan 26 minutes ago
Everyone else that doesn't use the entirety of the plan they pay for.
reply
nyc_data_geek 22 minutes ago
Retail and society as ANT shares get pulled into index and wealth funds
reply
ralph84 33 minutes ago
API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.
reply
surgical_fire 26 minutes ago
People believe in some wild stuff.
reply
exe34 2 hours ago
Corporate customers
reply
dofm 2 hours ago
And a kitten.
reply
codekansas 2 hours ago
This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

reply
chalkycrimp 2 hours ago
Good chance I'm not the only one thinking this: share the code
reply
lxe 12 minutes ago
Repetitive "make it faster" prompting is essentially what all these "RSI" claims are
reply
spwa4 26 minutes ago
I must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

(unfortunately for a decent model it will also be price competitive with a big nvidia card)

reply
saejox 2 hours ago
i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.
reply
Archit3ch 34 minutes ago
Now do silicon. ;)
reply
IshKebab 28 minutes ago
reply