This spiel is hilarious in the context of the product this company (https://juno-labs.com/) is pushing – an always on, always listening AI device that inserts itself into your and your family’s private lives.

“Oh but they only run on local hardware…”

Okay, but that doesn't mean every aspect of our lives needs to be recorded and analyzed by an AI.

Are you okay with private and intimate conversations and moments (including of underage family members) being saved for replaying later?

Have all your guests consented to this?

What happens when someone breaks in and steals the box?

What if the government wants to take a look at the data in there and serves a warrant?

What if a large company comes knocking and makes an acquistion offer? Will all the privacy guarantees still stand in face of the $$$ ?

  - Minimize how long we have "raw data" in memory
  - Tune the memory extraction to be very discriminating and err on the side of forgetting (https://juno-labs.com/blogs/building-memory-for-an-always-on-ai-that-listens-to-your-kitchen)
  - Encrypt storage with hardware protected keys (we're building on top of the Nvidia Jetson SOM)

  sudo rm -rfv /

  > Having idle thoughts in the car of things you need to do and being able to just say them out loud and know important topics won't be forgotten about.

I agree with the core premise that the big AI companies are fundamentally driven towards advertising revenue and other antagonistic but profit-generating functionality.

Also agree with paxys that the social implications here are deep and troubling. Having ambient AI in a home, even if it's caged to the home, has tricky privacy problems.

I really like the explorations of this space done in Black Mirror's The Entire History of You[1] and Ted Chiang's The Truth of Fact short story[2].

My bet is that the home and other private spaces almost completely yield to computer surveillance, despite the obvious problems. We've already seen this happen with social media and home surveillance cameras.

Just as in Chiang's story spaces were 'invaded' by writing, AI will fill the world and those opting out will occupy the same marginal positions as those occupied by dumb phone users and people without home cameras or televisions.

Interesting times ahead.

1. https://en.wikipedia.org/wiki/The_Entire_History_of_You 2. https://en.wikipedia.org/wiki/The_Truth_of_Fact,_the_Truth_o...

> The always-on future is inevitable

Not if you use open source. Not if you pay for services contractually will not mine your data. Not if you support start-ups that commit to privacy and the banning of ads.

I said on another thread recently that we need to kill Android, that we need a new Mobile Linux that gives us total control over what our devices do, our software does. Not controlled by a corporation. Not with some bizarre "store" that floods us with millions of malware-ridden apps, yet bans perfectly valid ones. We have to take control of our own destiny, not keep handing it over to someone else for convenience's sake. And it doesn't end at mobile. We need to find, and support, the companies that are actually ethical. And we need to stop using services that are conveniently free.

Vote with your dollars.

The product that’s being implicitly advertised here is supposed to ship at the end of this year and there doesn’t even appear to be a real photo of the thing and if that’s an indicator of the quality of the product then I must assume that it is poor and the people responsible also apparently do not have the money to hire a capable web designer and I’m sorry if this is harsh or unnecessary but I never thought I would miss the generic Bootstrap or Tailwind or whatever bougie framework other companies use because boy the layout here does not elicit great expectations for their product either and I’m worried that if it ever does ship that nefarious parties will intercept all the private communications of its unfortunate owners and in an ironic sort of way their devices will become the first sort of reverse ad agent that does not transmit advertisements but receives advertisements in the form of the raw interests of their clients fed to said nefarious parties and then laundered through more traditional channels.

A man-in-the-middle-of-the-middle-man.

> They’re building a pocket-sized, screenless device with built-in cameras and microphones — “contextually aware,” designed to replace your phone.

"Contextually aware" means "complete surveillance".

Too many people speak of ads, not enough people speak about the normalization of the global surveillance machine, with Big Brother waiting around the corner.

Instead, MY FELLOW HUMANS are, or will be, programmed to accept and want their own little "Big Brother's little brother" in their pocket, because it's usefull and or makes them feel safe and happy.

The explicit ads angle is only half the story. Even without paid placements, these models already have implicit recommendations baked in.

We ran queries across ChatGPT, Claude, and Perplexity asking for product recommendations in ~30 B2B categories. The overlap between what each model recommends is surprisingly low -- around 40% agreement on the top 5 picks for any given category. And the correlation with Google search rankings? About 0.08.

So we already have a world where which CRM or analytics tool gets recommended depends on which model someone happens to ask, and nobody -- not the models, not the brands, not the users -- has any transparency into why. That's arguably more dangerous than explicit ads, because at least with ads you know you're being sold to.

This strikes me as a pretty weak rationalization for "safe" always-on assistants. Even if the model runs locally, there’s still a serious privacy issue: Unwitting victims of something recording everything they said.

Friends at your house who value their privacy probably won’t feel great knowing you’ve potentially got a transcript of things they said just because they were in the room. Sure, it's still better than also sending everything up to OpenAI, but that doesn’t make it harmless or less creepy.

Unless you’ve got super-reliable speaker diarization and can truly ensure only opted-in voices are processed, it’s hard to see how any always-listening setup ever sits well with people who value their privacy.

Maybe I'm just getting old, but I don't understand the appeal of the always-on AI assistant at all. Even leaving privacy/security issues aside, and even if it gets super smart and capable, it feels like it would have a distancing effect from my own life, and undermine my own agency in shaping it.

I'm not against AI in general, and some assistant-like functionality that functions on demand to search my digital footprint and handle necessary but annoying administrative tasks seems useful. But it feels like at some point it becomes a solution looking for a problem, and to squeeze out the last ounce of context-aware automation and efficiency you would have to outsource parts of your core mental model and situational awareness of your life. Imagine being over-scheduled like an executive who's assistant manages their calendar, but it's not a human it's a computer, and instead of it being for the purpose of maximizing the leverage of your attention as a captain of industry, it's just to maintain velocity on a personal rat race of your own making with no especially wide impact, even on your own psyche.

Just when you've asked if there are eggs the doorbell rings, the neighbor stands there in disbelief, it told me to bring you eggs? Give him the half bottle vodka, it's going to expire soon and his son will make a surprise visit tonight. An argument arises and it participates by encouraging both parties with extra talking points.

But this was only the beginning, after gathering a few TB worth of micro expressions it starts to complete sentences so successfully the conversation gradually dies out.

After a few days of silence... Narrator mode activated....

The article is forgetting about Anthropic which currently has the best agentic programmer and was the backbone for the recent OpenClaw assistants.

First it's ads, then it's political agenda. We've seen this inconspicuous transition happen with social media and it will happen even more inconspicuously with LLMs.

> The most helpful AI will also be the most intimate technology ever built. It will hear everything. See everything

Big Brother is watching you. Who knew it would be AI ...

The author is quite right. It will be an advertisement scam. I wonder whether people will accept that though. Anyone remembers ublock origin? Google killed it on chrome. People are not going to forget that. (It still works fine on Firefox but Google bribed Firefox into submission; all that Google ad money made Firefox weak.)

Recently I had to use google search again. I was baffled at how useless it became - not just from the raw results but the whole UI - first few entries are links to useless youtube videos (also owned by Google). I don't have time to watch a video; I want the text info and extract it quickly. Using AI "summaries" is also useless - Google is just trying to waste my time compared to the "good old days". After those initial videos to youtube, I get about 6 results, three of which are to some companies writing articles so people visit their boring website. Then I get "other people searched for candy" and other useless links. I never understood why I would care what OTHER people search for when I want to search for something. Is this now group-search? Group-think 1984? And then after that, I get some more videos at youtube.

Google is clearly building a watered-down private variant of the web. Same problem with AMP pages. Google is annoying us - and has become a huge problem. (I am writing this on thorium right now, which is also chrome-based; Firefox does not allow me to play videos with audio as I don't have or use pulseaudio whereas the chrome-based browser does not care and my audio works fine - that shows you the level of incompetency at Mozilla. They don't WANT to compete against Google anymore. And did not want since decades. Ladybird unfortunately also is not going to change anything; after I critisized one of their decisions, they banned me. Well, that's a great way to try to build up an alternative when you deal with criticism via censorship - all before leaving alpha or beta already. Now imagine the amount of censorship you will get once millions of people WERE to use it ... something is fundamentally wrong with the whole modern web, and corporations have a lot to do with this; to a lesser extent also people but of course not all of them)

We're getting closer to a world where every company is an ad company, period. It seems like there are more and more ads touting a dwindling number of actual products.

- new startup idea: sound proof boxes for all your electronic devices

- put them inside the soundproof box and they cannot hear anything outside

- the box even shows the amount of time for which the device has not been able to snoop on you daily

Is anthropic using ads ? Is mistral using ads ? Is déepseek using ads ?

Google, meta, and amazon, sure, of course.

It's interesting that the "every company" part is only open ai... They're now part of the "bad guys spying on you to display ads." At least it's a viable business model, maybe they can recoup capex and yearly losses in a couple decades instead of a couple centuries.

I think local inference is great for many things - but this stance seems to conflate that you can’t have privacy with server side inference, and you can’t have nefariousness with client side inference. A device that does 100% client side inference can still phone home unless it’s disconnected from internet. Most people will want internet-connected agents right? And server side inference can be private if engineered correctly (strong zero retention guarantees, maybe even homomorphic encryption)

Contextual irony aside, this is a big reason why the proposal of leveraging AI agents for workflow processing in lieu of using them to develop fixed software to perform the same functions has always struck me as weird, and of late come across as completely nonsensical.

If you're paying someone else to run the inference for these models, or even to build these models, then you're ultimately relying on their specific preferences for which tools, brands, products, companies, and integrations they prefer, not necessarily what you need or want. If and when they deprecate the model your agentic workflow is built on, you now have to rebuild and re-validate it on whatever the new model is. Even if you go out of your way to run things entirely locally with expensive inference kit and a full security harness to keep things in check, you could spend a lot less just having it vomit up some slopcode that one of your human specialists can validate and massage into perpetual functionality before walling it off on a VM or container somewhere for the next twenty years.

The more you're outsourcing workflows wholesale to these bots, the more you're making yourself vulnerable to the business objectives of whoever hosts and builds those bots. If you're just using it as a slop machine to get you the software you want and that IT can support indefinitely, then you're going to be much better off in the long run.

The concern is real but the local solution is not ready. The author does not seem to think about that from the perspective of an "average consumer". I have been running my personal AI assistant on a consumer-grade computer, for almost an year now. It can do only one in thousand of the tasks that cloud models can do and that too at a much slow pace. Local ai assistant on consumer grade hardware is at least a few year away, and "always-on" is much further than that IMO.

> Every company building your AI assistant is now an ad company

Apple? [1]

[1] https://www.apple.com/apple-intelligence/

> There needs to be a business model based on selling the hardware and software, not the data the hardware collects. An architecture where the company that makes the device literally cannot access the data it processes, because there is no connection to access it through.

Genuine Q: Is this business model still feasible? Its hard to imagine anyone other than apple sustaining a business off of hardware; they have the power to spit out full hardware refreshes every year. How do you keep a team of devs alive on the seemingly one-and-done cash influx of first-time-buyers?

One point I see less discussed, not related to the post, is "We never trained people to pay for software. If there existed proper global payment mechanism for software companies, the whole trajectory would look different. People are ok to pay 5$ for a coffee but not for software which makes their lives easier."

This was the inevitable endpoint of the current AI unit economics. When inference costs are this high and open-source models are compressing SaaS margins to zero, companies can't survive on standard subscription models. They have to subsidize the compute by monetizing the user's context window. The real liability isn't just ads; it's what happens when autonomous agents start making financial decisions influenced by sponsored retrieval data.

Advertising and AI colliding is gonna be horrible, but their post is also just an ad itself

I really dislike the preorder page. The fact that it's a deposit is in a different color that fades into the background, and it refers to it as a "price" multiple times. I don't know if it was intentionally deceptive, but it made me dislike this company.

Enlightenment is man's emergence from his self-imposed immaturity. This is the age of enlightenment in reverse, the age of immaturity.

Perhaps I'm not totally clear on how this particular device works, but it doesn't seem like it has no ability to connect to the Internet.

Honestly, I'd say privacy is just as much about economics as it is technical architecture. If you've taken outside funding from institutional venture capitalists, it's only a matter of time before you're asked to make even more money™, and you may issue a quiet, boring change to your terms and conditions that you hope no one will read... Suddenly, you're removing mentions of your company's old "Don't Be Evil" slogan.

It's interesting to me that there seems to be an implicit line being drawn around what's acceptable and what's not between video and audio.

If there's a camera in an AI device (like Meta Ray Ban glasses) then there's a light when it's on, and they are going out of their way to engineer it to be tamper resistant.

But audio - this seems to be on the other side of the line. Passively listening ambient audio is being treated as something that doesn't need active consent, flashing lights or other privacy preserving measures. And it's true, it's fundamentally different, because I have to make a proactive choice to speak, but I can't avoid being visible. So you can construct a logical argument for it.

I'm curious how this will really go down as these become pervasively available. Microphones are pretty easy to embed almost invisibly into wearables. A lot of them already have them. They don't use a lot of power, it won't be too hard to just have them always on. If we settle on this as the line, what's it going to mean that everything you say, everywhere will be presumed recorded? Is that OK?

The level of trust I have in a promise made by any existing AI company that such a device would never phone home: 0.

The always-on future is absolutely not inevitable but I get that people have a lot of money riding on convincing people it is

This isn't a technology issue. Regulation is the only sane way to address the issue.

For once,we (as the technologists) have a free translator to laymen speak via the frontier LLMs, which can be an opportunity to educate the masses as to the exact world on the horizon.

Ads in AI should be banned right now. We need to learn from mistakes of the internet (crypto, facebook) and aggressively regulate early and often before this gets too institutionalized to remove.

Does anyone know how this device will filter out other voices like TV talking and stuff like that?

Always on is incompatible with data protection rights, such as the GDPR in Europe.

Who would buy OpenAI's spy device? I think a lot of public discourse and backlash about the greedy, anticompetitive, and exploitative practices of the silicon valley elite have gone mainstream and will hopefully course correct the industry in time.

We need an ai adblocker !!

I mean why is it so difficult for such companies to understand the core thing: irrespective of whether the data related to our daily lives gets processed on their servers or ours, we DON'T want it stored beyond a few minutes at max.

Even if these folks are giving away this device for 100% free, I'll still not keep it inside my house.

I’ve moved to Opencode, and I don’t see myself ever leaving it (if there were no alternatives ie AI glasses etc)

This is just an ad for maximally intrusive “AI”. We’re quite inured by now to all the dystopian nightmares, so this barely even registers.

How long web search had been objective, nice, and helpful - 10 years? Now things are happening faster so there is max 5 years in total of AI prompt pretending that they want to help.

I guess it goes to show that real value is in the broader market to a certain extent, if they can’t just sell people the power they and up just earning a commission for helping someone else sell a product.

I mean google was always an ad company and search engine. SOOOO hasn't changed much.

> The always-on future is inevitable

Well the consumers will decide. Some people will find it very useful, but some others will not necessarily like this... Considering how many times I heard people yelling "OK GOOGLE" for "the gate" to open, I'm not sure a continuous flow of heavily contextualized human conversation will necessarily be easier to decipher?

I know guys, AI is magic and will solve everything, but I wouldn't be surprised if it ordered me eggs and butter when I mentioned out loud I was out of it but actually happy about this because I was just about to go on vacations. My surprise when I'm back: melted butter and rotten eggs at my door...