PingCap TiDB Leaks
158 points by swyx 22 hours ago | 22 comments

xKingfisher 15 hours ago
Coming this close on the heels of the security audit by NCC [1] is pretty embarassing for all parties.

[1]https://www.pingcap.com/press-release/tidb-ncc-group-securit...

reply
hintymad 3 hours ago
> TiDB, the open-source distributed SQL database powered by PingCAP, today published the results of a rigorous source-code security assessment by NCC Group, one of the world’s leading cybersecurity consultancies.

It looks like you can assess the hell out of your source code, but your system can still leak if you don't secure your operations.

reply
kobieps 5 hours ago
From personal experience having worked with multiple "enterprise-grade" pentest vendors (including NCC), I think the entire pentest industry is overdue for a big shake-up.
reply
dherls 17 hours ago
If true, this is next level incompetence from PingCap. Apparently the attackers set up a replication pipeline for a month?!

> We even stood up a replication feed of their own control-plane database using credentials committed in their own repositories — GCP service-account keys and an Azure service principal — and let their infrastructure deliver it to us. Every six hours, for the last twenty-two days of the campaign, from 8 September to 30 September, that pipeline faithfully re-exported every table of the operational heart of TiDB Cloud — tenants, users, API keys, cluster credentials, changefeeds, migrations — and pushed it out.

reply
arpinum 21 hours ago
AWS private rates gives PingCAP ~47% off list price. Wonder which services got the biggest discounts? Unsurprisingly for a db company the cross-az traffic is 15% of their bill.
reply
gooseai 21 hours ago
[dead]
reply
nemothekid 17 hours ago
Man I recently executed a migration from Aurora to Planetscale. I seriously considered PingCap but decided against it because I thought they cloud pricing was too complex. I wonder if my personal information is in the leak.
reply
wannabe44 15 hours ago
> Summary: "The root password of the cluster can be reset unconditionally"

But what matters is that it was all memory safe and had no data races.

reply
boorbat 8 hours ago
You realize it's written in Go right?
reply
wannabe44 7 hours ago
Not even data race safe then. NGMI.
reply
dangoodmanUT 18 hours ago
what is happening today

and how are people not talking about this more

reply
dilyevsky 15 hours ago
only the beginning. next year or two are going to be wild on this front so buckle up...
reply
jedisct1 11 hours ago
It's written in Rust, so it's safe.
reply
boorbat 8 hours ago
https://github.com/pingcap/tidb reports as 95% Go with no Rust at all?
reply
squirrellous 8 hours ago
That confused me too since I remember reading they are Rust proponents. Apparently their storage engine TiKV is written in Rust, and TiDB is the database built on top.
reply
rvz 22 hours ago
First the Google Cloud outage [0], then the CA private key leak [1]...

...and now this...

> Today we are making the PingCAP breach public. We possess the complete TiDB Cloud control plane, its billing and finance estate, its GCP storage estate and its AWS cost and billing estate, its Jira, its CRM, its analytics, its source code, and a four-year log of every connection every customer ever made to its product, from the IP addresses of their application servers.

This is an entire leak of ALL PingCap TiDB's customers AND data.

All of this in a single day. It is over.

[0] https://news.ycombinator.com/item?id=50163961

[1] https://news.ycombinator.com/item?id=50166579

reply
gregwebs 18 hours ago
I am reading this as a leak of every possible piece of metadata but not an actual leak of customer data. They have SQL logs which do contain customer data and there is a mention of some snapshots/imports.
reply
hdjdnxjzhzhs 17 hours ago
How the fuck is 10.6TB metadata?
reply
fock 5 hours ago
(POV when storing 1PB in Oracle) when I saw the 10 TB in another post, I was like "wait, isn't this sold to scale out, we have 100 times the data here".
reply
gregwebs 16 hours ago
TiDB is for large scale data. You generally wouldn't use it unless you have at least 1TB of data. 10TB is a single small customer database for them.
reply
binsquare 17 hours ago
Easily at scale

Size is not a good indicator for content quality

reply
qingcharles 17 hours ago
Don't forget the PHP megabomb dropping in a few hours too.
reply
gooseai 21 hours ago
[dead]
reply