MiniLake Is a Local Databricks API Emulator with SQL, Delta Lake and Spark
61 points by kermatt 3 days ago | 10 comments

gavinray 3 hours ago
If this actually works, this is such a huge boon. Let me share why:

I work on connectors to various datasources, mostly SQL DB's

The most useful tests are the integration tests, sending actual queries to the source

For most DB's we have a giant Docker Compose which runs a local instance

But things like Snowflake, Databricks, Redshift, Athena suck to test because we need an external hosted instance

I would kill to be able to run test versions of all of these locally

reply
skeeter2020 28 minutes ago
Why are you writing tests on the actual provider? IME these types of tests are slow, brittle and don't give you much value. Even if you get a true positive, what can you do about it?
reply
efromvt 3 hours ago
[dead]
reply
lysecret 35 minutes ago
Want to hijack this someone made experience recently with https://ducklake.select/ ? We need to setup a warehouse soon and this looks enticing.
reply
akbarian 3 hours ago
Getting tired of the standard AI design, all websites look the same now
reply
smashed 2 hours ago
I would say go straight to their GitHub but there again, same thing.

With a list of "Known gaps" and implemented engines listed with status like "Real", and of course the Quick start first, as Claude loves to do.

It's not "wrong", it just shows low effort. But then again it's just something someone needed and shared it for free so let's not complain too much.

All the products and saas launching with obvious AI landing pages, that the authors did not even care to trim down, that's another story.

reply
pjmlp 3 hours ago
Before it was all those common template sites.
reply
ngruhn 4 hours ago
You should call it "pond".
reply
isoprophlex 4 hours ago
Great for CI i guess; or is this meant as a cost savings measure where you can swap out your databricks for self-hosted this?
reply
dominotw 58 minutes ago
Wtf is databricks now anyways? Looks like their ceo is hawking cheap chinese models and some silly harness called ominiagent. Anyone use ominiagent? I thought they were building their own models at somepoint? what happened to that.

Also looks like they did one full round of english alphabet and are on series L second time around. I feel like we might have some a tech theranos on our hands here.

reply
NDizzle 30 minutes ago
Databricks is a furnace. It burns money.
reply
quietsegfault 39 minutes ago
[flagged]
reply
lanmu 2 hours ago
[flagged]
reply
PIYUSHMSONI98 3 hours ago
[flagged]
reply