SIMD-accelerated computer vision on a $2 microcontroller
138 points by zdw 23 hours ago | 41 comments

jwr 4 hours ago
BTW, I find it pretty amazing that you can buy complete ESP32-S3-Zero boards on AliExpress for $5.33. That's for a little board with the chip, a ceramic antenna, a USB connector and an LED.

ESP32-C3-Zero are $2.80 each.

These prices are absolutely ridiculous for what you get. Also, the ease of use of these boards is incredible. Just connect them via USB to your computer and off you go.

reply
twisteriffic 2 hours ago
Anecdotal, but believable, warning for those in the US

https://www.reddit.com/r/GlobalEntry/comments/1x4c062/us_ord...

reply
monocasa 5 minutes ago
The whole point of most of those esp32s being in that metal can module is that they're already tested and have FCC certs. That is, the intentional transmitter is already certified, making intgrators' lives a lot easier.

I bet there's either something the poster isn't telling us, or CBP made a bad call.

reply
anigbrowl 31 minutes ago
I am a bit skeptical. IT caught my attention because I recently ordered some parts, but then I recalled that all my existing gear from that supplier already has FCC certification. And indeed so does the base technology:

https://documentation.espressif.com/en/documentList?t=Certif...

It could be there's more to the story of what the person ordered, or it could be that CBP are misapplying the rules (quite likely under this administration), but it seems like some very overstated assumptions as written.

reply
htrp 3 hours ago
what do you do with them?
reply
nsarrazin 29 minutes ago
Personally it made it very easy to build custom sensors/actuators for my Home Assistant setup. I have a folder with a bunch of esphome configs and just point Claude Code at it and tell it to add new entries for the stuff that I need, it's really good at that it can even write drivers for specific sensors autonomously. Granted you still need to do the wiring/soldering by hand (for now!)

Just around the home I have:

- a board that captures the audio output from my vinyl player and streams it on my LAN (so I can play my records on speakers across the house)

- one esp32 in each of my DIY speakers around the house driving the DAC+amplifier

- smart plant sensors that measure soil humidity, temperature, etc.

- a camera inside of a bird house, solar powered

- a 8x8 lidar ToF distance measurement board for detecting sleep phase

reply
jwr 2 hours ago
I am currently working on a Bluetooth Mesh home network allowing me to control lights, gather sensor data from various places, etc (think interfacing with a home alarm system for presence detection).

Also, a home air quality monitor, both with a display and reporting to my mesh network.

Or you could build a "plane radar" like this one :-) https://github.com/ironicbadger/ESP32-Plane-Radar

reply
ssl-3 2 hours ago
Cheap MCU dev kits in general, for me?

A somewhat-elaborate fan speed controller for a wonderful old pro audio power amp that also needed a good bit of other rehab

A window fan controller, for summer use, that only turns on the fan when it's cooler outside than it is inside -- so it only cools the indoors, seeks never to heat it

A sensor for automating the color temperature of my indoor lighting based on outdoor conditions

A macro pad that ultimately works kind of like a Streamdeck, but without the built-in display features (that was a download-and-print project)

Other random stuff, like an ESPHome widget that talked to a window aircon via infrared commands (mostly just a download-and-run project)

They're just cheap little programmable low[ish]-power computers on a board, with some flash storage and RAM, and a fair bit of low-level IO. Often with wireless network connectivity as a built-in. They do stuff. :)

If the code is tight-enough, they can do a remarkable amount of that stuff -- in terms of raw compute, they present something near-ish to the Pentium era of personal computing for a few dollars.

For me, they make sense for simple projects. When something like a disused laptop or a Raspberry Pi-like device [and the accoutrements, like a whole-ass operating system] seems like overkill, then: It's probably overkill.

With a dev board, I can breadboard a circuit and maybe even leave it that way forever, or get a custom PCB drawn up with KiCad and made inexpensively by JLC PCB or Osh Park or whatever, or just solder wires and parts to the dev board like a caveman.

All methods work, and the cost of screwing it up is usually very low.

After that, the thing gets powered up with one of the featureless USB wall warts that we all have accumulated too many of and it just sits there and does its job.

(And honestly, these Pi Pico and random ESP32 dev boards are also overkill for what I do with them. Most of what I've done could be accomplished with an MCU chip that costs pennies instead of dollars, but my quantities are low-enough that I just don't care much about a few dollars for occasional one-off projects.)

reply
KZerda 2 hours ago
Personally, I just finished up a project with one to poll outdoor and indoor weather sensors and notify the household over discord when it's warm enough outside to probably need air conditioning, when it's cooled off enough to probably not need AC, when the AC goes on, and when the AC goes off. It also parses and displays and notifies information from the National Weather Service when there are relevant alerts in my area.
reply
teaearlgraycold 2 hours ago
Similar class of chip, but I use the rp2350 to perform computer vision with LiDAR to detect when people leave dishes in a sink and beep angrily at them.
reply
javawizard 2 hours ago
Wait until you get into developing your own PCBs and discover the BK7238 for $0.81 :)
reply
pjmlp 5 hours ago
> For its price, the ESP32-S3 is a powerhouse of a microcontroller.

Indeed, don't forget it is more powerful than computers like this one,

https://en.wikipedia.org/wiki/Amstrad_PC1512

reply
asimeqi 4 hours ago
That was my exact first computer with the CGA graphics, GEM Desktop, Locomotive Basic etc. All the software came in 4 360KB diskettes. The diskettes had each it's own beautiful color. There were 3 sample Basic programs that seemed to me quite advanced. One sample was a document management system. Another was a drawing program where you could draw with a mouse. Didn't know that GEM Desktop was made by Digital Reasearch. It was a very good windowing system. I believe it was based on MS-DOS. There was also a CP/M OS. So much software in only 1.2MB.
reply
pjmlp 4 hours ago
Exactly, thanks for sharing.
reply
sedatk 3 hours ago
I dearly remember the keyboard's warm red Caps Lock light and my sessions of running Print Master and writing BASIC programs on it.
reply
MeteorMarc 4 hours ago
We will see the day your toothbrush has more compute than the original IBM PC.
reply
zimpenfish 4 hours ago
Pretty sure we have, e.g. [0], "TI's MSPM0Cx MCUs featuring Arm® Cortex®-M0+ core with 24 MHz CPU, 32-bit structure, 4 to 16 KB Flash provide cost-effective designs for electric toothbrush and shaver applications." - compared to the 5150[1], it's 4x the bus width, 5x the CPU clock, albeit with fewer RAMs.

[0] https://www.ti.com/lit/ab/slaaeg0/slaaeg0.pdf

[1] https://en.wikipedia.org/wiki/IBM_Personal_Computer

reply
glouwbug 4 hours ago
At 8mhz? I'd imagine it rivals at least the Pentium III
reply
pavlov 4 hours ago
Well, probably not, but maybe a plain Pentium?

The Pentium III was a superscalar core that scaled up to 1.4 GHz. This microcontroller is max 240 MHz and does less work per cycle.

reply
pjmlp 4 hours ago
Probably, I was use this example because I feel youngers can't imagine how much we could do with such MS-DOS class computer, even within its limitations.

So naturally an ESP32 can do so much more.

reply
camel-cdr 58 minutes ago
The CH32V407/CH32V467 is a bit more expensive, was $2.4 per unit two month ago, but supports the embedded subset of the RISC-V vector extension.

So you basically get a full AVX-512/SVE level SIMD ISA, minus FP support and with 64-bit SIMD width. But you still have LMUL, which will be great here to maximize utilization.

reply
lukan 2 hours ago
The engineering seems impressive, but how useful is the result? What can you detect exactly with it in real time?

Detecting corners, but to what resolution/detail, a picture with a result would have been helpful to understand the capability.

edit: missed this link, this is what this is about

https://en.wikipedia.org/wiki/Features_from_accelerated_segm...

reply
peesem 5 hours ago
nit: 5.1MP/s to 11.2MP/s is not an increase of 220%, it's an increase of 120%. it's 2.2x faster, but percentage increase starts at 0%, not 100%. 100% faster is double, 200% faster is triple.
reply
seritools 5 hours ago
nit²: 2.2x as fast, or 2.2 times the speed; not 2.2x faster. same issue you pointed out.

using faster with x/times is just begging for misunderstandings

reply
HPsquared 3 hours ago
2.2x the speed is 1.2x faster.
reply
paaloeye 5 hours ago
ESP32-S3 SIMDs are super difficult to use unfortunately
reply
PowerElectronix 4 hours ago
Practice makes perfection
reply
paaloeye 4 hours ago
true, but nothing is perfect
reply
dsign 3 hours ago
I find it easier than x86 SIMD[^1], but the lack of support for vector operations for 32 bits makes me very grumpy.

[^1] Though, to be honest, x86 has great automatic SIMD tools so I never had to try too hard.

reply
meindnoch 5 hours ago
Just LLM it
reply
ballmerpoint 5 hours ago
IME, they’re still pretty bad at getting specific implementation details of SIMD correct. The best I’ve done so far is to harness them to an actual piece of hardware and have them iterate until they see performance improvements but 1) Goodhart’s law, they tend to overfit the benchmark suite and 2) the resulting SIMD often becomes illegible.
reply
TGower 4 hours ago
For a lot of SIMD applications, overfitting isn't really a concern. In my use case I need to calculate the sum of absolute difference between two vectors for autocorrelation, there's no real possibility for overfitting, either it gets the correct answer or it doesn't. LLMs are good at reading through the CPU manual and trying different optimizations.

The resulting SIMD being illegible isn't an issue in my eyes, assembly isn't a language known for it's readability in general. If you want to really understand what the assembly is doing, ask your LLM to whip up a visualization and explanation tailored to your input data where you can see how each operation leads you to the final result.

reply
pezgordo 3 hours ago
Have you tried including the instructions documentation to the project?
reply
ballmerpoint 2 hours ago
Yes, of course. It doesn’t seem to recall the specific details very well, or confuses them with x86 or ARM instructions that do similar things. And yes, my AGENTS.md says ONLY USE INSTRUCTIONS IN blah.txt.

I dunno, clearly I need an LLM psychiatrist.

reply
pezgordo 43 minutes ago
Haha, maybe try different agent/harness which would the closest thing to that.
reply
paaloeye 4 hours ago
reply
adolph 2 hours ago
Interesting. Since this was implemented on the S3, which uses the ISA XTensa LX instead of the RISC-V of most newer ESP32 series chips, I looked into how Espressif is handling SIMD on the latter ISA. Apparently it is only present in limited fashion as a processor extension on the P4 and S31 [0]. The S31 datasheet claims SIMD [1] and the P4 datasheet [2] files them as "Processor Instruction Extensions."

0. https://bitbanksoftware.blogspot.com/2026/04/esp32-p4-simd-e...

1. https://documentation.espressif.com/esp32-s31_datasheet_en.p...

2. https://documentation.espressif.com/esp32-p4_technical_refer...

reply
varispeed 3 hours ago
Why is it still called SIMD-accelerated, if these are now just normal instructions. You just have to take a little bit more effort into restructuring your algorithms to make use of them.
reply
anthomtb 58 minutes ago
How does using instructions that operate on multiple data elements, which make your software 3 times faster, not fit the definition of SIMD-accelerated?
reply
varispeed 16 minutes ago
Because these are normal instructions.

If you do multiplication using mul instead of loop and add, is it acceleration too?

reply