My first gut reaction to this was: Have you tried using a vision-language model?
I used to have an elaborate Home Assistant setup with tons of zigbee sensors here and there telling me if front door is not closed properly, if the rice cooker is left on (via current monitoring), if the garage door is left open (with a magnetic contact sensor), if window is left open, if the soil is dry, etc. I had to keep changing batteries of all these things and manage flaky Zigbee connections in the past.
More recently, I've found the absolute most reliable, ~zero-maintainence solution to be an LLM. Home Assistant already has an LLM plugin, you can wire it up to a local LLM if you want, and you can just feed it an image and ask "Is this garage door open or closed" and I found that to be much better for my sanity than changing batteries every 6 months and dealing with flaky sensors sending me false alerts.
I even found it useful for "Did the cat vomit on the floor" (if so notify me and pause the cleaning robot schedule) and "Do any plants look like they need attention"
For TFA's use case, if you already have a security camera facing front, you can just feed it that image and ask "Are the garbage bins out"
Also, if you're worried about cameras sending home images to the cloud, there are non-cloud options. (I use Unifi cameras and Home Assistant gets a RTSP stream over LAN.)
I have an automation that will grab an image from that camera and send it to the Anthropic API with a prompt describing where the bins are, and the color of each bin. The prompt comes back with JSON indicating which bins it can see. If it can see e.g. the yard waste bin, and it's time to take the yard waste bin to the curb, it will send me a notification.
It also reminds me when it's time to return them to their "resting" position after pickup- unless I already have.
Home Assistant orchestrates this all. I also have it hooked up to a "Bindicator"- a white 3D printed bin with a color-changing LED inside, controlled by an ESP8266 using ESPHome. When it's time to move the bin it lights up the color of the bin that needs to be moved. If I haven't moved the bins back after a day it turns orange to remind me.
Claude configured the automations for me via MCP but it's not particularly complicated. It's pretty reliable, though sometimes just because of where the doorbell camera is it can't clearly see the bins. If it can't make a ruling on the bin location it just falls back to notifying me so I will sometimes get a reminder I don't need.
- how often do you have to change your cameras' batteries, or are you accepting ugly wiring for powering from mains?
- how many cameras do you have installed to cover your home?
- IP cameras are flaky, too
- constantly sending images of one's private space to a cloud LLM is Orwellian
- are you polling your LLM minutewise "Did the cat vomit on the floor"?
Meh (a) I want security and cat-monitoring cameras anyway (b) I'm willing to take the effort to install cameras neatly since they're capable of serving as 100 different sensors
> constantly sending images of one's private space to a cloud LLM is Orwellian
I agree. I have a local server with a GPU. The security side of things records to a local NVR. Nothing goes to the cloud.
> IP cameras are flaky, too
haven't had nearly as many problems here
> are you polling your LLM minutewise "Did the cat vomit on the floor"?
Kind of, but it only re-polls if the camera detected motion, and then every 15 minutes as a catch-all.
I’ve not played with vision models much. I did some experiments with OpenCV and Processing many years ago, but times have changed.
I think you just wrote me next weekend project ;)
why are so few hacking their cognitive abilities (basic short-term memory)?
I don't have a problem with short or long term memory, in certain contexts. I can hold entire problem definitions and codebases in my head while working, but I often set down my keys on the way to pick up my water bottle and get to my car only to have no idea where they are. I can remember details of specifications, language features, video game mechanics, etc. for years, but often can't remember someone's name.
There's a whole cottage industry of social media telling me this is ADHD but I don't know if that's real or just for clicks, and if I can improve these issues.
Many, having unfortunately been all the way at the worst-version of that – and figured out the way back.
No guarantees I can help, obviously, but this rhymes with the types of intermittency I dealt with (if you start going into rooms and forgetting what you went in for, that really sucks and is not “normal” - despite being common.)
Give a way to reach you? If you’re motivated, you can probably optimize.
My biggest barrier to doing something like thins in my home automation setups is that I already have an extensive schedule of things that need periodic disposable battery changes (like CR2477 cells in zigbee humidity/temperature sensors), and I'm really hesitant to add even more into my house. In addition to things like battery-backed smoke detectors in every bedroom, etc, which need periodic 9V replacements on a schedule.
In fact I've started looking at going the opposite direction, where I can run a cat5e cable from my utility room, down and across the ceiling of my basement, and come up in a location where I can install an 802.3af to arbitrary DC voltage DC-DC converter (3.3, 5VDC, 12VDC out, or similar) as a PoE splitter, to make some of these things not dependent on batteries. Same idea as how you can find 9V or AA battery to direct DC wiring "dummy battery" battery eliminators now. Obviously such an approach wouldn't work for garbage bins but that's how much I am trying to eliminate having dozens of batteries to replace, as they're already all over the place with window and door sensors.
Just as aside, lithium-battery powered smoke detectors are a godsend. The battery lifespan lasts the recommended life span of the detector, i.e. around 10 years. No battery replacements needed, although you still ought to test them once in a while.
The BLE tags I previously used lasted about 2 years of a CR2045. These are much larger CRs, and I have tuned it right down on the report timing when stationary, so I am anticipating more than 2 years.
https://www.amazon.com/Splitter-Compliant-Surveillance-5-5x2...?
You can also find variants of them that are microUSB or USB-C power out. Finding ones that are natively 3.3V or 5VDC out is more difficult and rare, it seems the most viable option if I do this is to buy a very tiny DC-DC converter that's 12VDC in and then 3.3 or 5VDC out, and glue/stick it to the poe splitter, perhaps in an enclosure of some sort. Then wires out to the battery dummy eliminator.
The thing above is a 'dumb' device so any monitoring of the power you would have to do from your 802.3af PoE switch, entirely depending on what model/capabilities your switch has for port status, wattage, etc.
edit: Google "USB AA battery eliminator" and you'll find products which have a USB-A male input, expect standard 5V USB input, and output either 1.5V, 3.0V or 4.5V or 6V (1S, 2S or 3S or 4S AA battery voltage). You could then plug one of those into the many 802.3af to USB-A female (5V2A) poe splitters which exist on the market. Example: https://www.amazon.com/Battery-Eliminator-Supply-Adapter-Rep...
Years ago I worked as a consultant on a projects which is quite similar to this, but larger scale and involving moving machinery. The task was basically to come up with sensors that would register when and which machines would be at what place in a structure. In the end we came up with a solution which only involved cameras and object detection...this was before AlexNet etc., so fun times with classical machine learning.
Trash Train: I put my trash cans on rails so they move automatically [video] - https://news.ycombinator.com/item?id=34216415 - Jan 2023 (87 comments)
I like the tech, and would like to do something with the boards. We only have 1 bin, and as others point out, I could just ask HA if the bin is out on the street on Friday morning (already have cameras around the house).
I've thought of a garage door opener that senses when the car is home. I would still like to do something public though, just not sure what.
I have a transit info app at bus stops, so maybe a bus aware arrival board, but GPS solves that problem better, and the UWB devices would just get stolen anyways.
Super cool work! I had no idea UWB tags had gotten so cheap, or that CR2077 was a size.
CR2477 is high enough capacity that it's used in a number of zigbee or zwave800 based products that have no wired interface.
https://www.google.com/search?client=firefox-b-d&q=cr2477+ba...
https://aichiplink.com/blog/CR-Battery-Sizes-and-Specificati...
Interestingly I’ve used Zigbee for well over a decade, tried easily over a hundred different devices, built others homes, and I’d never encountered this battery size until this project!
Will fix the typo later
So those tags are going to have to be a little cheaper before I’ll use them. :-)
Luckily these are only £5 each https://www.alibaba.com/x/1lBJtin?ck=pdp
The majority of cost is the wrover and the programming jig (which I think the manufacturer built to order, it’s very… home brew
Then I can overengineer the fensor circuit! Ultralow power, bin recognition, alarms, fox nerfs, auto-lids and so on.
Back in 2019 I worked for a company (in San Diego) that added UWB-based positioning to the 7 train line in NYC.
The code I wrote ran on what was basically a combined Raspberry Pi 3 compute module connected to a UWB device installed into the train's bonnet which would ping UWB devices set along the track side to determine the position of the train.
In most situations I could get a unique position based on only two beacons, but I had the benefit of the trains being on one of a limited number of fixed tracks which severely limited where they could practically be in space... well, assuming there hadn't been a train derailment.
The system didn't work in a 2D plane, though, our surveyed track data and track side UWB positions were all 3D encoded as ECEF coordinates.
In some situations depending upon track layout you needed more than 2 beacons to determine the correct position if you were just sampling single points in isolation, but since you're working with a continuous stream of changed positions as the train ran along the track at predictable speeds, you could pretty easily figure out which of multiple possibilities it had to be based on what came before even if you had potential multi-positional results for the current sample.
Of course, we would use as many of the track side beacons as we could see to strengthen confidence in the positioning, and the beacons were placed so we could generally see a good 4-6 of them most of the time.
Snow was the biggest challenge for the system, in a heavy thick snow the UWB signals bounce all over the place and you get severe multipath effects that can make it pretty difficult to determine which is the true line of sight signal for your time of flight calculations.
Fun job, I had to get certified to be able to walk the NYC train tracks because we installed a lot of the track side beacons ourselves, and I spent a lot of late nights doing live testing runs on trains that were empty other than a couple of people from our company and some MTA workers assigned to our test trains.
My solution is similarly overengineered: a vision system combined with an olfactory system feeds into a biological neural network, which monitors the bins and determines when intervention is required.
The neural network is annoyingly power-hungry, occasionally unreliable, and has yet to support OTA firmware updates. I also have a Home Assistant as a backup for when the neural network fails. It provides a gentle nudge that perhaps the bins need taking out.
Shocking, I know.
This gave me a chuckle. It seems almost every where in the world this is true. Ofcourse it is quiet outside around that time, amplifying the effect, but I am always amazed why loud things always need to happen so early. Maybe I should become a morning person, but prefer dark, silent nights for concentration.
The other half is buildings being spaced 7m apart, because that's the minimum allowable at this height. The echo is enormous.
That being said it's still more tolerable than the people absolutely punching it through our 30km/h street with their modified exhausts and windows rolled down - as if the point for them was to lose hearing as fast as possible.
In Toronto, Canada (and Ontario generally?) we use single-stream recycling:
* https://en.wikipedia.org/wiki/Single-stream_recycling
So we have organic waste (food scraps, etc; green bin), recycling (blue), and everything else ("garbage"; black). Organics/green gets picked up every week, blue and black alternate (always on the same day: so one week green+blue, next week green+black).
For hazardous stuff you take it to a depot or wait for a city community event nearby:
* https://www.toronto.ca/services-payments/recycling-organics-...
This past year the provincial government made it so that municipalities do not have to handle recycling anymore, but it is now the responsibility of the manufacturers ("producer responsibility model")
* https://en.wikipedia.org/wiki/Recycling_in_Canada#Ontario
* https://www.circularmaterials.ca/what-is-epr/
Is there a reason why all these separate streams are used where you live?
I don't think it's power hungry at all! Benchmark it's energy use against SOTA, you will be surprised.