How ChatGPT serves ads
220 points by lmbbuchodi 6 hours ago | 142 comments

programjames 4 hours ago
Less than two years ago, Sam Altman said

> I kind of think of ads as a last resort for us for a business model. I would do it if it meant that was the only way to get everybody in the world access to great services, but if we can find something that doesn't do that, I'd prefer that.

So, is this OpenAI announcing they're strapped for cash?

reply
staticshock 3 hours ago
Feels to me like idealism crossing into realism. OpenAI could be the next Google, or the next Facebook, or the next… I don't know, Netflix?

All those companies (and many other large tech companies) have discovered the same arbitrage that older media companies discovered decades ago, which is that we, on the average, are much more willing to pay with attention than with money, even where money would have been the better choice.

Advertising continues to be one of the most powerful business models ever invented, and I don't think that's changing any time soon.

reply
plemer 3 hours ago
Altman is an idealist?

I read this as: I know ads are likely if not inevitable but I can’t say that while I’m trying to gain users and inspire trust but I’ll start to float even in this non-denial the justification for the thing I’m ultimately going to do.

reply
nine_k 3 hours ago
Altman wanting to look idealistic and inspiring.

See it as a brand image advertising campaign of the time.

reply
nerptastic 4 hours ago
Well - I think the writing was on the wall when they announced they were going to be for-profit. Slippery slope and all that, but I’m sure some of this is because they’ve been giving out free tokens for years.
reply
Aurornis 3 hours ago
The ads are for the free tier and new $8 ad-supported plan.

The revenue from a few ads on the free tier in exchange for limited queries to GPT-5.3 is negligible compared to what they pull in from API costs and the subscription plans. This looks like a play to justify the existence of the previously money-losing free tier as they go into an IPO. Throw some ads in there to make it closer to a neutral on the balance sheet.

The key part of that quote was "everybody in the world". The ads are their way of sustaining the low end of the access.

reply
nine_k 2 hours ago
The revenue from highly targeted ads, using even better profiles than Google Search or even Facebook could build, may be non-negligible.

Commercial ads could be a smaller revenue source than political ads.

reply
famouswaffles 2 hours ago
>The revenue from a few ads on the free tier in exchange for limited queries to GPT-5.3 is negligible compared to what they pull in from API costs and the subscription plans.

Unless they botch the implementation, it's not going to be negligible with ~800M+ free subscribers.

reply
chromacity 3 hours ago
> The revenue from a few ads on the free tier in exchange for limited queries to GPT-5.3 is negligible

So why chase this negligible revenue?

reply
kingstnap 3 hours ago
The real question is what do you get out of advertising to people who don't have any money? Kinda squeezing blood from a stone.

You'd be better off saying you use those people to A/B test changes and filling idle GPU batches while giving paying customers a more consistent experience.

reply
troyvit 41 minutes ago
> The real question is what do you get out of advertising to people who don't have any money?

Psychographic data. What they learn from these folks will create the most powerful manipulation technology yet.

reply
ldoughty 3 hours ago
A bunch of people pay to remove ads, and a bunch of people that are happy to give businesses their attention (view ads) I'm exchange for services... I.e. Gmail, YouTube, but don't feel they use enough / are annoyed enough to warrant $15-25/month.

Some brands are okay with impressions.. you can build trust in your product be advertising it for weeks/months and when the user does make a purchase that brand is on the mind.

reply
giancarlostoro 3 hours ago
> The ads are for the free tier and new $8 ad-supported plan.

Dang.

> The revenue from a few ads on the free tier in exchange for limited queries to GPT-5.3 is negligible compared to what they pull in from API costs and the subscription plans. This looks like a play to justify the existence of the previously money-losing free tier as they go into an IPO. Throw some ads in there to make it closer to a neutral on the balance sheet.

Yeah, I guess this time around Sam Altman can't be lying about how many Monthly Active Users he has.

reply
mh- 4 hours ago
That's not how I read that sentence at all. Maybe I've just been speaking VC for too long.

What he meant was: "I'm going to get everybody in the world access to great services. Doing so means monetizing somehow. Ads will be the last way I chose to do that, but I will if it's the only way I can figure out how to achieve that goal."

reply
normie3000 3 hours ago
You've said the same thing.

> Ads will be the last way I chose to do that

The implication is that they've exhausted all other options.

reply
mh- 3 hours ago
I haven't said the same thing as the parent commenter:

> So, is this OpenAI announcing they're strapped for cash?

It by no means conveys that. It means they haven't figured out another way to monetize something they want to do; it indicates nothing about their financial situation. It means they don't want to sell something at a loss perpetually while they figure it out.

reply
Dylan16807 3 hours ago
Being forced into something you don't want to do, to stop selling at a loss... I would categorize that as some level of strapped for cash.
reply
mh- 3 hours ago
You realize we're talking about a product that is currently free, right? Neither of us have any insight into the margins of their paid offering.

All this means is: we have a free offering that we can't figure out another way to monetize right now.

We can each draw our own conclusions about what that might mean for the state of their business, but all of the other inferences (ha) in this thread are conjecture.

reply
hattmall 3 hours ago
Presumably the way to monetize a free tier is by converting them into paying users.
reply
conductr 53 minutes ago
“Upgrade for an Ad free experience” will certainly be a part of it.
reply
Dylan16807 3 hours ago
> You realize we're talking about a product that is currently free, right? Neither of us have any insight into the margins of their paid offering.

I don't see how that changes the analysis.

> All this means is: we have a free offering that we can't figure out another way to monetize right now.

And they're doing something they significantly don't want to do to monetize it.

Either they fully changed their mind, or the money is somewhat important, or they're utterly crazy.

The first is unlikely, the last is unlikely, the middle one is enough for a casual "strapped for cash".

It's a very minor conjecture. Actions aren't taken for no reason.

reply
mh- 3 hours ago
If we can agree that "strapped for cash" also includes "not stupid with cash", I think we're on the same page here. :)

(For all I know they are strapped for cash, to be clear; I just don't think the quote says that.)

reply
Dylan16807 3 hours ago
Going with a last resort implies more than "not stupid".
reply
mh- 3 hours ago
Okay, fine: "conservative with cash" or even "tight with spending"?

(I'm not sure how much deeper HN threads can nest.)

reply
Dylan16807 3 hours ago
"Tight" gets pretty close to "strapped", especially when it comes to making a change.

(They can go super deep if people are committed.)

reply
mh- 3 hours ago
I concede.

(Haha, ok, let's call a truce here before we break HN! Appreciate the conversation.)

reply
bitvvip 4 hours ago
Who can resist the temptation of profit? One always has to make money
reply
bitmasher9 4 hours ago
If I say “Doing X is a last resort” and then I’m caught doing X, it should raise some eyebrows about my level of desperation.

It’s not that OpenAI is trying to raise revenues that bothers me, it’s how they are doing things that said was desperate just a couple years ago.

reply
bluefirebrand 18 minutes ago
Tons of people can resist the temptation, but they aren't likely to be the sort of person that gets put in a role like where Altman is
reply
m463 34 minutes ago
more like "Sam Altman said"
reply
holotherapper 3 hours ago
"last resort" doing some heavy lifting in that quote.
reply
whatisthiseven 3 hours ago
Sam Altman is the guy fired for lying. Why believe what he claims?
reply
jimmygrapes 4 hours ago
Charitably, it seems that we have yet to find, as a species/society, anything more effectively profitable than ads. I cannot blame those who come to this conclusion so long as no more powerful and proven motivator yet exists. I hate it, but I understand.
reply
sayYayToLife 4 hours ago
[dead]
reply
programjames 3 hours ago
I think you're missing that Sam Altman is very smart. If OpenAI really were on the verge of becoming massively profitable due to their next-gen AI, he would not want that information leaking. If Sam Altman acts differently in the world where profits are on the horizon, that information leaks prematurely. Thus, he has to act as if OpenAI is strapped for cash, whether or not it is.

The keyword is "glamorization": https://www.lesswrong.com/w/consistent-glomarization

reply
HWR_14 2 hours ago
I would just assume that they were still spending VC money to lock in users if nothing happened. I would not assume "AI is about to make money obsolete"
reply
largbae 2 hours ago
This reads similar to the Trump 4D chess excuse. It seems unlikely that this is a ruse, and much more likely that OpenAI's market cap is supported by doing "all the things" to exploit the huge monthly average user base that OpenAI has accumulated.
reply
torben-friis 4 hours ago
These are the less worrying kind of ads in our future.

Seeing how google has been fighting SEO for ages, what's going to happen when companies figure out how to inject ads into the model?

We haven't yet seen the problem of adversarial content in play, I think.

reply
mgambati 2 hours ago
The model already advertises because they where trained on massive data’s that refers big brands.

Ask for suggestions for a new pair of shoes. What brand do you think it will suggest Nike, Adidas or some random small one?

reply
jcims 3 hours ago
I experimented with this way back when custom GPTs were first released (looks like late 2023). There are a few / commands you can use to suggest what product to inject, how overt, etc and a generic /operator command to send whatever you like 'out of band' from the chat.

https://chatgpt.com/g/g-juO9gDE6l-covert-advertiser

One of the most interesting things is when it starts pitching a product and you start interrogating it about why it picked that product. I haven't used it in probably a year so it may not do the same thing now, but back then it 100% lied consistently and without any speck of remorse. It was rather eye opening.

Edit: Tried again, it didn't lie this time lol - https://chatgpt.com/share/69f16aa4-c008-83ea-92b3-51f16ca77d...

reply
BoorishBears 4 hours ago
Why do you need to inject ads at the model weights layer when you control the frontend?

Have the model generate keywords from the query, then inject guidance from matching advertisers into the context window

q: How do I make a new React app?

a: Vercel makes it easier to get your project running fast

Some other choices would be:

...

ⓘ This part of the response was sponsored by Vercel

reply
JumpCrisscross 3 hours ago
> ⓘ This part of the response was sponsored by Vercel

LLMs are essentially unregulated. I don't believe they have any legal disclosure obligation in America.

reply
HWR_14 2 hours ago
They may ignore the disclosure obligation, but technically they are supposed to disclose this fact.
reply
BoorishBears 3 hours ago
They'd show it regardless (maybe as a popup though): the disclosure doesn't make it that much less effective at scale, and the optics of getting caught vs just disclosing it are not worth getting dragged into
reply
sayYayToLife 4 hours ago
[dead]
reply
WaxProlix 4 hours ago
It's not an issue of how - there's a great ADM with markup/down supported already, waiting for system prompts to be injected in realtime via the same online auction system that powers banner ads and smart tv content. There's got to be some latent resistance to the idea for now - but it's so easy to do, it'll happen.
reply
_boffin_ 3 hours ago
Can you provide some references to what you’re talking about
reply
WaxProlix 2 hours ago
Sure, https://iabtechlab.com/standards/openrtb/

There's a standardized, normal (in adtech) approach to building 'creative's (viewed/seen ads) around context-dependent scenarios. It's not hard to extend existing IAB primitives to include things like context-enrichment (system prompt augmentation in this case) or whatever. I don't want to malign my downvoters but suspect they're mad I'm pointing it out, rather than engaging with facts as they are. It's trivial for ads to interact with your(our!) AI usage.

reply
WD-42 5 hours ago
Since they are served as distinct events then I would think they should be easy to block.

Once the ads are injected directly into the main response is when things get interesting.

reply
kardos 4 hours ago
> Once the ads are injected directly into the main response is when things get interesting.

This would be where you post-process the LLM response with a second LLM to remove the ad..

reply
naruhodo 4 hours ago
I think it will be difficult to remove bias when you ask a model to compare alternative products. The model will simply lie, as with a biased human opinion and you will need to consult multiple models for a diversity of opinion and presumably use a "trusted" model to fuse the results. Anonymity will be a key tool in reducing the model's ability to engage in algorithmic pricing.

Super easy. Barely an inconvenience.

reply
normie3000 4 hours ago
> will simply lie, as with a biased human opinion

Is this really how bias works?

reply
inetknght 3 hours ago
Oh no. Definitely not. Humans would never just lie. They always lie only if they're biased. That is, after all, the definition of how a bias works.

/s

reply
naruhodo 3 hours ago
I'm using bias to mean hidden motivations to the benefit of other parties. Feel free to substitute a better word.

EDIT: actually I'm really not sure what hairs we're trying to split here. I see bias as a departure from objectivity. It can be conscious or unconscious, but when someone is selling something, it's frequently conscious and self-serving, and I believe that's referred to as a lie.

reply
tempest_ 4 hours ago
This is already how email works in the corporate world.

A writes email with chatgpt to B.

B sees big blob of text and summarizes email with chatgpt.

Adding an LLM in the middle is just the next step.

reply
torben-friis 4 hours ago
It's like one of those memes about the worst possible date picker, except for a communication system.
reply
devmor 3 hours ago
Then you just end up in an arms race that ultimately leads to photocopy-of-a-photocopy output.
reply
ihsw 4 hours ago
[dead]
reply
lmbbuchodi 5 hours ago
you can block these URLs: |bzrcdn.openai.com^, ||bzr.openai.com^ It won't blanket block everything but will significantly reduce telemetry collected.
reply
nazcan 2 hours ago
And that's why you gotta just use one domain. Or mix ads and important content on one domain.
reply
sheiyei 24 minutes ago
No, wrong lesson. That's why you use UBlock Origin.
reply
TZubiri 4 hours ago
Blocking transparent ads is not a good idea. The consequence is that you will be fed opaque ads.
reply
saghm 4 hours ago
I don't buy this premise. Nothing stops a company from trying to hide ads in the first place, and plenty of them do. Ad blockers for web content have been a thing for years, and using an ad blocker has continued to be strictly a better experience regardless of how many "organic" ads are present on a page.
reply
TZubiri 55 minutes ago
So products have 3 choices.

1- No ads. 2- Transparent ads. 3- Opaque ads.

By removing option 2, you only leave options 1 and 3.

If the product has costs (always true), then option 1 means that there is no gratis tier. So you force companies to remove their free tier, or to make ads opaque.

If you want to enjoy a free product without paying and without ads, then do so, but don't pretend you are an activist for doing so, just pay the ethical cost instead of trying to avoid paying that as well.

This isn't complex either, the only reason you don't get it is because you don't want to get it, you want things that are gratis without paying for them, and you want the free things to be given to you on your terms, and you don't want to be guilty about it. It's easier to think of yourself as righteous than to recognize that you want to be a leech.

reply
estimator7292 4 hours ago
What possible reason could they have to not always run both? It would make zero sense to leave that money on the table
reply
TZubiri 53 minutes ago
It's simpler to do one thing than to do two. You make a choice and you do that.

Could they be doing opaque ads right now and we wouldn't know? It's possible, that will probably eventually come to light and it might have legal consequences, but sure it's possible.

But it's not a given, and your logic of "it would make zero sense to leave money on the table" is certainly not a QED, it's absolute reductionism.

reply
Aurornis 5 hours ago
The ads are in the free tier and the new ad-supported $8/month plan.

Every time this comes up there are comments assuming that ads are being injected into the normal plans, but these are for the free tier and the new Go plan which warns you that it includes ads when you sign up.

reply
ceejayoz 4 hours ago
Cable TV was once ad free. So was Netflix. Companies just can’t help themselves.
reply
darepublic 4 hours ago
Would require a lot of training to implement ads blended into convo and not have it be too obvious/ eff up the results?
reply
catcowcostume 3 hours ago
Until next quarter earnings, when ads become a feature in more expensive plans.
reply
blackjack_ 4 hours ago
It is one of the eternal lessons; All tech business plans eventually lead to serving ads. At least until we ban pixels / 3rd party tracking.
reply
benleejamin 5 hours ago
I'd always thought that ChatGPT ads would be indistinguishable from actual content.
reply
ticulatedspline 4 hours ago
I think that's where they want to be. feels like everyone knows it too, that the long term expectation is basically being able to buy ad words and have LLMs lean responses towards whatever people bought.

Seems the playing field is a bit too open though, models are more fungible than the companies would hope so most of the current moat is brand based and seems like they're not ready to go all "Black Mirror" on us just yet.

reply
irjustin 5 hours ago
this would be a breach of trust and short term would work great but long term is too detrimental.

same thing could've been said for search results, so at least that part is still "safe".

reply
SchemaLoad 4 hours ago
Long term all of the major LLM platforms will have invisible ads, influences, and propaganda woven into the content. The temptation will be irresistible for these companies.
reply
doginasuit 3 hours ago
I'd be surprised if product placement isn't already basically at play. Charging companies for including/prioritizing their documentation in the training data, for example. Thankfully LLMs are terrible at the subtlety it would require for a direct marketing campaign.
reply
bix6 5 hours ago
O you think trust matters? This is capitalism not trustism.
reply
saghm 4 hours ago
Well it's sure not "anti-trustism" in recent years...
reply
PradeetPatel 4 hours ago
Long term retention is built on brand trust and usability, then ensh*ttification happens.
reply
nalekberov 4 hours ago
No, this is late stage capitalism without regulation.
reply
JumpCrisscross 3 hours ago
> always thought that ChatGPT ads would be indistinguishable from actual content

Remember when we got upset that Google was putting ads into image search [1]?

[1] http://www.ryanspoon.com/blog/2008/12/14/google-image-search... 2008

reply
senectus1 4 hours ago
I'm pretty sure that will be an eventual evolution of the product. The business model cant sustain itself as it is at the moment, eventually chatGPT wont be the product... we the users will be.
reply
phailhaus 4 hours ago
That was the fearmongering, which made no sense because advertisers can't put a dollar value on "the AI will kind of sort of mention you", and because every conversation needs an ad. If ChatGPT always snuck in a brand mention even on the simplest questions, everyone would hate it.

Ad technology is really old. They're just going to use the same proven tech that has a track record of creating billionaires: intersperse content with sponsored blocks.

reply
acdha 4 hours ago
I don't think that's a fair dismissal: you see ads all over media websites because the rates have been plummeting as consumers tune out ads. One main reason why everyone does is that ads are so obtrusive and repetitive, and that's exactly what LLMs change: I'm sure we'll see regular ads on AI apps because the companies have trillions of dollars to repay but advertisers would pay a lot more for openings where they aren't _forcing_ their message as a distraction but are instead able to insert it fairly naturally into a context where the user is engaged.

The entire history of advertising before the web was companies estimating a dollar value on “awareness” when they couldn't measure direct referrals and every business in the world has gotten a lot better at measuring sales since then. It's not going to be transformative but if, say, Toyota got ChatGPT to say their vehicles were a better value than Ford's I suspect they'd be able to tell pretty quickly whether sales were improving relative to the competition and would pay well for that to continue.

reply
infinite_spin 5 hours ago
I see OpenAI making a significantly larger amount from defense contracts than from advertisements pumped into chats. So I wonder whose bright idea it was to create a public perception risk.
reply
Larrikin 5 hours ago
Every single MBA can show for at least one quarter revenue is up after they introduced ads. They do not care what happens after if they can plan their career around that.
reply
saghm 4 hours ago
I wish I had the optimism that you did about companies being willing to stop at just doing one dubious thing or another for money when there's nothing stopping them from doing both.
reply
peddling-brink 5 hours ago
Maybe the negative press from ads is better than the negative press from powering murderbots?
reply
tayo42 4 hours ago
Bad press from a contract like that happens once and everyone forgets. Ads are in your face everytime
reply
peddling-brink 4 hours ago
"OpenAI Powered Drone Destroys Elementary School, Hundreds of Children Dead" might last a while.
reply
Enginerrrd 2 hours ago
I mean Palantir’s targeting product led to EXACTLY that outcome and it seems to have been largely forgotten already, and they managed to avoid a lot of bad press about it.
reply
EcommerceFlow 2 hours ago
If highly targeted/tailored LLM ads on free accounts aren’t good enough for HN, are any ads acceptable?

Let’s be reasonable.

reply
didip 4 hours ago
So news about OpenAI demise is real. They can’t sustain themselves without ads.
reply
boringg 4 hours ago
Never in any world were any of the top AI labs not going to sustain themselves with ads. It has always been a timing issue.

Even a cut on every sale on site + sub rev not close.

reply
saghm 4 hours ago
Even if it wasn't necessary for their survival, it's hard to imagine a world where they wouldn't try to do it anyways. I'm not someone who buys into the idea that companies are obligated to maximize profits at the expense of all else, but I do think that in the absence of other factors (e.g. regulation) it's where pretty much every company will end up.
reply
chrisweekly 4 hours ago
"the idea that companies are obligated to maximize profits at the expense of all else"

!! That is literally the definition of legally-binding fiduciary resonsibility for publicly-traded corporations. There are exceptions (PBCs, B-Corps) but they're rare.

reply
hattmall 3 hours ago
It's really not though.
reply
SubjectToChange 4 hours ago
They can’t be hemorrhaging cash when they IPO.
reply
sayYayToLife 4 hours ago
[dead]
reply
keyle 5 hours ago
Can't wait for "watch this ad for 90s to use xxhigh on your next prompt!"
reply
jonah 3 hours ago
I was looking to see if BZR referred to a 3rd party ad network. I didn't find anything, but apparently someone has replicated OAI's system and you can run insert it into your own LLM.

GH: system32miro/ai-ads-engine

reply
djmips 5 hours ago
And it begins.
reply
holotherapper 3 hours ago
The schema is literally named single_advertiser_ad_unit. The single_ prefix is doing all the foreshadowing you need.
reply
danilocesar 3 hours ago
You are absolutely right!
reply
vicchenai 5 hours ago
figured this was inevitable once they started the free tier. the attribution loop being a separate event stream is actually kind of clever engineering though -- means they can A/B test ad formats without touching the core model response
reply
yoyohello13 2 hours ago
Here we go again. Imagine if we put as much engineering effort toward actual things that help people, but more ads it is, as always. This is proof AGI doesn’t exist. If it did, it could come up with a better business model than more fucking ads.
reply
avaer 5 hours ago
Remember that ads are the "last resort" for OpenAI, and they're doing this despite the fact that it's "uniquely unsettling", according to Sam.

Was he lying, or has OpenAI given up hope that this train wreck works economically without enshittification? Neither option is good, but I don't really see a third.

reply
Aurornis 5 hours ago
The ads are only for the free and $8/month plans. They basically added an ad-supported super discount level that you can ignore if you’re paying for the normal plans.
reply
RussianCow 4 hours ago
But the fact that they've added an ad-supported tier this early into their life as a company means they're desperate for revenue. You start inserting ads when you're optimizing for profit, not when you're still growing. It took how long for Netflix to introduce an ad-supported plan?
reply
milkshakes 4 hours ago
when did netflix offer a free tier?
reply
chrisweekly 4 hours ago
options 1 and 2 are not mutually exclusive
reply
dankwizard 4 hours ago
Really well written, technical post. Good read.
reply
singingtoday 5 hours ago
I don't like anything about this.
reply
renewiltord 3 hours ago
Interesting, no bidding flow entirely first party and contextual.
reply
guluarte 4 hours ago
I've seen chatgpt suggest me more amazon products lately
reply
sayYayToLife 4 hours ago
[dead]
reply
mock-possum 4 hours ago
Not to me they don’t, cause I canceled my account and stopped using their products when they made the announcement.
reply
Aurornis 3 hours ago
They don't serve them to me, either, because I don't use GPT-5.3 on the free tier or Go plan where these ads show up.
reply
sayYayToLife 4 hours ago
[dead]
reply
BoredPositron 5 hours ago
I don't get what's wrong with charging for your product. Like get rid of the free tier and make a small tier with an easy to serve model for like 5 bucks. Is it still the DAU rage of the 2010ss that's driving burning money?
reply
teaearlgraycold 5 hours ago
How do you pick up new paying users without letting people use the service for free for a while first? Freemium is popular because it works well.
reply
yoyohello13 2 hours ago
Free trial? Demo?
reply
uriahlight 5 hours ago
Let the enshittification commence!
reply
gxs 5 hours ago
This is gross

It feels like we’ve been in the golden age and the window is coming to a close

Let the enshitification begin, I guess

reply
dannyw 5 hours ago
How do you expect the spend & COGS for free LLM inference to be funded? For users who don't want to pay, or maybe can't pay?
reply
derektank 5 hours ago
Perhaps it’s a glib and easy thing to say, but after a teaser period, I would simply not offer free LLM inference. Agreeing to serve ads just completely re-aligns your interests away from providing the best possible user experience to something else entirely.
reply
infinite_spin 5 hours ago
From things like defense/private contracts

e.g. colleges pay for institutional subscriptions

reply
2ndorderthought 5 hours ago
The average person doesn't benefit from defense contracts ... Like ever.
reply
IX-103 4 hours ago
The average person is slightly more female than male and has 2.1 children, but they do benefit from defense contracts since it makes up a small percentage of their salary.
reply
2ndorderthought 4 hours ago
You are a fun person. We should be friends
reply
iammrpayments 5 hours ago
It has begun ever since they nerfed chatgpt4 before releasing 4o
reply
2ndorderthought 5 hours ago
In the past month local models have been ramping up in major way meanwhile the namesake providers have upped prices, went offline randomly, and started doing slimier and slimier things.

I really think the future is local compute. Or at least self hosted models.

reply
SchemaLoad 5 hours ago
The hosted ones still have the advantage of being able to search the internet for live info rather than being limited to a knowledge cut off date.
reply
gbear605 5 hours ago
I’m not sure why a model needs to be hosted in order to make network calls?
reply
hansvm 5 hours ago
Is there a library of good tools for LLMs to call? I have to imagine the bot-detection avoidance mechanisms are a major engineering effort and not likely to work out of the box with a simple harness and random local LLM.
reply
gbear605 42 minutes ago
If your volume is low enough, it should be pretty fine. It can just piggy back onto your personal browser cookies for Cloudflare.
reply
ossa-ma 5 hours ago
Even the hosted ones are blocked from searching certain sites, for example Claude is banned from searching Reddit:

`Error: "The following domains are not accessible to our user agent: ['reddit.com']."`

reply
wyre 5 hours ago
Tavily, Exa, Firecrawl, Perplexity, and Linkup are all tools for agents to search the web.

I’ve been building a harness the past few months and supports them all out of the box with an API key.

reply
goosejuice 4 hours ago
Kagi also has an API. People who hate ads are probably the same folk that should be paying for Kagi. That's the sane alternative world where companies respect their users.
reply
chrisweekly 3 hours ago
That's not how it works. Whether local or hosted, every modern model has a cutoff date for its training data, and can be leveraged by agents / harnesses / tools to fetch context from the internet or wherever.
reply
darepublic 5 hours ago
Local ones that support tool use can do the same
reply
eightysixfour 5 hours ago
You can do that locally too!
reply
CSMastermind 5 hours ago
What's the rough equivalent of a local model? Are we talking GPT-4?
reply
2ndorderthought 5 hours ago
Qwen 3.6 which was released this month is a large but still smaller model. Supposedly it's at about sonnet level when configured correctly. It can be run on commodity hardware without purchasing a data center. https://www.reddit.com/r/LocalLLaMA/comments/1so1533/qwen36_...

Then there are middle size ones which require multiple gpus which are like gpts latest flagships.

Then there is kimi 2.6 which is a monster that is beating opus in some benchmarks. https://www.reddit.com/r/LocalLLaMA/comments/1sr8p49/kimi_k2...

It's basically whatever you can afford. Any trash heap laptop can run code auto complete models locally no problem. The rest require some level of investment, an idle gaming pc, or a serious investment

reply
Terretta 5 hours ago
Depends on your VRAM or "unified" memory for how smart it is, and CPU/GPU for how quick it is.

128GB of RAM? Sure, the early to mid 4s releases, except maybe 4o. And on an M5 Max, about the same speed.

I wouldn't really bother under 64GB (meaning 32GB or less) except for entertainment value (chats, summaries, tasky read-only agent things).

reply
kay_o 5 hours ago
GLM 5.1 and DeepSeek 4 are acceptable, but the cost of hardware and energy cost that depending on your use case you may as well purchase a Tokens. They get useless and stupid rapidilty if you quant enough to run on single 16-24GB GPU style.
reply
rnxrx 5 hours ago
The arc of the technological universe is short, but it bends toward enshitification.
reply
jesse_dot_id 5 hours ago
That's cool, I'll never see them.
reply