Meta's GB{2,3}00 Cluster
33 points by stokedmartin 2 days ago | 10 comments
sgvwthry 24 minutes ago
Posting this two days before earnings seems to indicate that there is going to be some bad capex news.
replyzer00eyz 6 minutes ago
Ya dont say.
replyNegative Free Cash Flow are the words every one on HN needs to learn.
Every thing before now, you will look back upon as the happy time. When every year storage, ram and compute grew and the prices went down. Your cloud provider made so much money they paid cash for hardware.
Now they borrow at 8% and net margins on ram/storage were around $0.10/GB ... today they sit at over a dollar. (SEE micron's s10's if you want to check the math on that).
Forget circular financing, its a circular firing squad because your going to have to figure out where the hell your going to run any of your gear.
4x to 10x for ram +8% for financing +30percent AWS margins. Good luck everyone its gonna get rough out there.
alberth 53 minutes ago
Actual title: ”Inside Prometheus: Meta’s Journey to a Gigawatt-Scale Supercluster”
replyma2kx 15 minutes ago
Imagine your job is just replacing millions of Dollars worth of hardware every day. I mean there must be a reason the hold 8 GPUs per rack spare.
reply
nft add set inet filter meta_crawl '{ type ipv4_addr; flags interval; }'
nft add element inet filter meta_crawl '{ 57.141.0.0/24 }'
nft add chain inet filter input '{ type filter hook input priority 0; }'
nft add rule inet filter input ip saddr @meta_crawl tcp dport {80,443} reject with tcp reset
I think that's not incompetence, I think that's by design.