Show HN: Claude-thermos – keeps your Claude session warm for you
35 points by s0ck_r4w 4 hours ago | 24 comments

unholiness 3 hours ago
I've directly inspected calls for pro/Max plans and as of today they have 1hr cache expiries. This has definitely degraded to 5 min in the past but that's the behavior today.

If you're paying API rates, you can choose 5m or 1hr yourself (and pay different rates).

Keeping a 1hr cache warm could still be useful, sure, but outside that, I don't see much use of this today.

reply
s0ck_r4w 2 hours ago
When using API keys the default is still 5 minutes. Setting 1 hour for the entire session is actually a lot more wasteful considering the 2x rate that it comes with
reply
jonas21 3 hours ago
Is the 5-minute expiration correct? I thought it was more like ~1 hour.
reply
s0ck_r4w 3 hours ago
You can set the retention globally (whole session) to 1 hour which will actually make it more expensive. The default is 5 minutes.

*UPD:* actually it appears the default is authentication-dependent. API key gets 5 minutes, subscriptions - 1 hour.

reply
alukin 3 hours ago
Feels like will be shut down real quick
reply
gogobio 3 hours ago
Yes, this wastes cycle. They dump your cache and deallocate the VMs so that others can use it. This will result in tokens wasted and downtime for others.
reply
supern0va 53 minutes ago
Can you expand on this? I can't imagine this is holding up an entire machine or materially impacting capacity. Isn't it common to have a tiered cache and to ultimately evict (or move to another tier) if there's an active inference request and no available capacity elsewhere?
reply
Lalabadie 46 minutes ago
Yes, however if a tool like this commoditizes a way to stay "higher" in the cache hierarchy to avoid getting dropped, then for the same total capacity, Anthropic have to become stricter about retention.
reply
supern0va 13 minutes ago
It's unfortunate that there isn't some better way to signal that there's a high chance that a turn is coming (ie, due to active sub-agents). If CC could send a ping saying "I don't need a turn, but please keep this cached at some tier because there will be a turn soon." that would probably help with cache efficiency. Instead, it seems like there's just the shotgun one-hour solution.
reply
l1n 3 hours ago
Why? Seems fine if people want to burn more cache reads.
reply
agluszak 3 hours ago
Because it makes Anthropic earn less money
reply
munk-a 3 hours ago
> Because it makes Anthropic burn more money

Fixed that for you.

reply
cadamsdotcom 3 hours ago
"Earn"?

Cache duration is arbitrary. What it actually does (if used en masse) is decrease the amount of oversubscription their infra can handle..

reply
janderson215 52 minutes ago
Earn is synonymous with profit, and this hurts their profits. Introducing latency can benefit much more people and actually make usage more efficient. Reducing latency can sieze up a functioning system.

Lately, I’ve been thinking about how this related to fractional banking. If you were to eliminate fractional banking introduced in the US by Hamilton, you would destroy a lot of current prosperity.

reply
cosmotic 2 hours ago
How will this not lead to tragedy of the commons?
reply
davesque 2 hours ago
Exactly. I really wish people wouldn't use this. If this becomes popular, anthropic will just modify their cache policy to be much less fair. It's not like they have infinite cache.
reply
ATMLOTTOBEER 3 hours ago
Glad this exists. It will force anthropic to fix their flawed cache mechanism.
reply
cortesoft 3 hours ago
How should they fix it?
reply
addaon 53 minutes ago
There are a lot of workflows where cache is likely to be consumed exactly once (e.g. anything chat-like where a single thread is extended; if a prefix cache hit is found, and a set of new longer prefixes is inserted in the cache, it is unlikely that the original shorter prefix will be consumed in later turns). I could see allowing each session key a small number (1?) of 24-hour cache entries, where inserting a new cache entry (perhaps the maximum-length prefix, perhaps an API-tagged specified-length prefix) consumes that slot and either boots the rest of the items from cache, or demotes them to 5 minutes or something. Basically: workflow awareness, not uniform handling.
reply
s0ck_r4w 2 hours ago
Make the cache TTL more adaptive. Have more tiers than just 5m and 1h. Long-running subagents become increasingly ubiquitous. There's no reason for Anthropic to not do a better job for this use case.
reply
razodactyl 3 hours ago
Don't we pay for cache input though?
reply
broodbucket 2 hours ago
You can send and receive 1 byte and refresh the cache. Keeping the cache alive isn't free, but it's close.
reply
cortesoft 3 hours ago
The idea is this keeps it in cache so you don’t have to pay to re-input.
reply
j45 3 hours ago
This is a good idea if it could manage it in an acceptable way.

For example, there might be something I intended to complete in one sitting, but took two sittings in the same day unexpectedly. Maybe it could just be a few cache delays per day or something, tagged in advance somehow.

reply
szin 3 hours ago
[dead]
reply