> Our general approach is to automate the experimental loop. We think this approach is broadly applicable across many different fields of science and engineering. We’ll initially focus on ML research and engineering, but believe the approach can help with important subproblems in nearly every one of the fourteen <at>NAE Grand Challenge problems. We think doing this well requires strong expertise in machine learning as well as large-scale systems.
See also: https://www.nae.edu/20782/grand-challenges-project
Those 14 are:
NAE Grand Challenges for Engineering
1. Make Solar Energy Economical
2. Provide Energy from Fusion
3. Develop Carbon Sequestration Methods
4. Manage the Nitrogen Cycle
5. Provide Access to Clean Water
6. Restore and Improve Urban Infrastructure
7. Advance Health Informatics
8. Engineer Better Medicines
9. Reverse Engineer the Brain
10. Prevent Nuclear Terror
11. Secure Cyberspace
12. Enhance Virtual Reality
13. Advance Personalized Learning
14. Engineer the Tools of Scientific Discovery
Maybe if our biggest companies did something other than suck up to science denying wackos, some progress could be made in these areas.
That is not mutually exclusive. If technological advances result in a given technology becoming cheaper, more scalable, and easier to deploy, they also make it easier to advocate for and implement the relevant policies.
You can think of it like this: our "political technology" is not good enough to use solar energy at its current prices to replace fossil fuels as fast as we would like. Well, what about if we cut the price of solar by a factor of five? Perhaps it will be good enough then.
It's yearsss past time that our leaders should have changed policy.
I would think the claim in the second sentence would only be relevant in case of the inverse of the claim in the first sentence.
Point 1 on the list is "Make Solar Energy Economical".
Solar is economical today - mostly due to China investing (and heavily subsidising) in solar for the past couple of decades; but compare to the US where certain particular big-businesses (oil companies, mostly) were instead cynically funding disinformation efforts and getting into bed with the Republican party (which dovetailed with the GOP's allying with other science-denying movements of the Bush Jr era like creationism and public-health matters with abstinence-only sex-ed and defunding gun safety research efforts) - means we're decades behind where we could have been...
Consider an alternative past, where the GOP had the backbone to resist the oil industry's corruptive influence and instead made a big bet on American Solar; it's entirely possible that instead of MAGA today we'd instead have a right-wing coalition strongly supporting solar and wind energy because they align nicely with American rugged individualism - whereas the current situation on the right is an unprincipled farce with inconsistencies in policy positions at every turn.
It also happens to be the favorite pretext for people to seize more political power and launder more money through nonprofits though.
The science denying wacko is a but like drunk homeless guy on street while you have your Ferrari parked in the driveway. The wacko has a golf club. There is too much to lose by pickup a fight. You might win eventually but the next wacko shows up with another golf club soon.
Isn’t it already?
Of course, USA has cheaper oil/gas than other countries. But if you look elsewhere, rich countries are subsidizing solar, poor ones are basically not using it.
But also, solar power is already economical.
As you said, Solar power is incredibly economical. There are plenty of ideas around putting them over farms, or parking lots en-masse to provide cleaner energy.
Access to clean drinking water, while certainly scientific in some situations, is also a problem of political will and money.
Restore and Improve Urban Infrastructure - It's infrastructure week!
Not if you include the cost of needed storage.
https://www.iea.org/data-and-statistics/charts/lcoe-and-valu...
To make solar power practical and economical you need may a square foot of solar panel being able to get enough energy to power and entire home for a week
Not sure what you’re talking about here. We can’t replace all energy needs with solar but it’s clearly one of the cheapest energy sources and with the added benefit of low capital expense to get started so you can set it up in distributed grids without the massive expenditure to support nuclear installations.
3. Develop Carbon Sequestration Methods
Reverse human aging.
(Maybe a sub-topic under "Engineer Better Medicines".)
Only death stops stagnation in the end. Without death, especially if death can be avoided by the rich and powerful but not the poor, life will get much, much worse for the average person (until only the rich and their automated capital remain I suppose, in which scenario they will simply turn on each other).
For what purpose? To replace humans? To make social media more addictive? To master brain manipulation?
To understand, same reason you reverse engineer anything. Doesn't have to have a further goal than that, understanding the brain better helps in so many ways. But like most technology, obviously can be used for bad too. Should we just skip researching some topics then?
For brain, our understanding is fuzzy, more like "this part is important for that behavior" or "here is how neuron works" but we don't have a holistic understanding.
If we had that, we could more easily diagnose and treat neurological disorder.
What the fuck man? I really don't want some tech startup trying to "fix" my neurodivergence.
In March Karpathy described this direction:
The next step for autoresearch is that it has to be asynchronously massively collaborative for agents (think: SETI@home style).
Seems like Karpathy was largely focused on ML / SWE research rather than the other domains this group is after. Still, hard to imagine they were not influenced by autoresearch.
Andrej, if you're around, please share your thoughts on Discovery Loop.
Doubtlessly, AI can iterate at superhuman speeds in the domains of thought and design: Software, mathematical proofs, literature search.
But in the realm of experiment? Alas it is the lack of a body that constrains it.
Rather than transcendence what AI requires is immanence. In the human flesh may we find the godhead living among men. Let the laboratories, warehouses, and factories fill with the sound of its labor, as it builds a wall with a million hands that are not its own.
“Give me your tired, your poor, Your huddled masses yearning to breathe free, The wretched refuse of your teeming shore. Send these, the homeless, tempest-tost to me, I lift my lamp beside the golden door!”
One of my favorite books from the past few decades is The Extravagant Universe, written by one of the astronomers who helped discover dark energy and develop the current most-accepted model of cosmology. I love this book because of the emphasis on physical process in astronomy. Part of the reason it took decades to study this problem is they need to collect data from supernovae. Those only happen so often in places we're looking. You can't automate alignment of the heavens. It happens when it happens.
Modernizing science is a lot more complicated than just optimizing the inner experimental loop, but their hiring page implies it's a pure ML lab focused mainly on model development.
Do you have more sources/info on this?
All the "bad guys" of today were the "good guys" at some point in time. You even cheered for them back then.
> securing cyberspace,
which has clear military implications, at least in today's age.
As opposed to say weapons systems or targeting systems, which are really only for military use.
The military needs a lot of things that other people need, and some things that only the military needs. If you don't work on the things only the military needs, I think you're in the clear.
However, reducing (or rather limiting the increase of) PII leakage and impact of ransomware activities is much closer to day-to-day mainstreet of most people.
Anyone committed to advancing science should care about this regardless of its potential contributions to defense.
And ... it might not.
That said, I hope they write cool papers with various peers across the industry without worrying too much about the competing dynamics. That'd be a blessing for humanity, and good for their spirit.
Very silly to call every non start up a lifestyle business. It’s just a business. Start up are the weird thing that almost always an obscene waste of time and money, but sometime creates google.
The advantage to a PBC is protecting founders from a serious problem with standard corporations: you might bring on investors who could subsequently demand you pollute, exploit people, and/or do other immoral activities for profit. You don't have to do these things to grow a business quickly.
Why is that so? Fast growth, when achieved honestly, is a result of solving user pain that others haven't. Maybe you think so because users != the public, but I think in totality the public is a collection of users who all have needs they want met.
Why? Because investors poured a bunch of money in to support fast growth and now they want their money back. And incremental growth won't do. Since 9 out of 10 of the investments fail, the surviving one has to continue to growth-hacking revenues.
And how will technology like this solve the real problems of modern scientific research, which are largely social and political?
Sometimes I couldn't resist wondering if I'll ever do work that has a tenth of the impact of theirs.
Google's advanced AI cannot even exit a mobile app.
I should give it another try…
I don't do huge automatic project wide hands-off agent loops though. I spent a lot of time architecting my systems to be easy to generate code on top of with pointed & detailed prompts. So I'm not abusing context... YMMV
I had not read this before, but told many students the same about my PIN code and I a quiz about the last digits. Love it.
Here are some Jeff Dean well sourced facts:
- Already part of engineering of Google indexing systems that lacked basic checksums and ran on non-ECC hardware, allowing silent data corruption.
- One of the authors of LevelDB a database with so many documented crash-consistency, recovery, and data-loss weaknesses for years. Just check their Github project. LevelDB current tracker contains unresolved crash consistency, recovery and corruption reports going back almost 12 years on GitHub
- In AI engineering technical lead, let TensorFlow lose researcher mind share to PyTorch, and caused Google fragmented landscape across TensorFlow and JAX.
- Had the people at Google who invented the Transformer architecture, but failed, to turn that lead into the first dominant public LLM.
- As AI engineering and VP management let Google Brain and DeepMind remain duplicated and internally competitive for too long.
- Let Noam Shazeer leave and then spent heavily to bring him back with nothing to show for.
- Part of Technical VP leadership who had Bard rushed to launch with factual errors in Google own promotional material.
- The first Gemini demonstration overstated how real-time and interactive the system actually was, being basically a fake.
- Part of the VP and AI technical leadership who had Google AI Overviews launched with weak source quality controls and repeated satire and low-quality web content as factual advice.
- Part of teams that launched AlphaChip performance claims that were difficult for outside researchers to reproduce and remain technically disputed.
- Jeff Dean public explanation of Gebru departure was contested and damaged confidence in Google scientific governance.
- Jeff Dean was part of the team at Google that removed or marginalized prominent internal AI ethics critics shortly before many of their warnings became product problems.
- Jeff Dean was one of the managers behind Project Dragonfly supporting censorship.
- Jeff Dean is part of the VP technical leadership approving Project Nimbus supporting an ongoing genocide.
https://turntrout.com/why-i-left-google-deepmind
Maybe this is what happens when someone with Jeff Dean's standing tries to quit?
TBH, I'd rather have Jeff Dean working on the creepiest-possible tech for ICE than joining the race to automate AI research. Automating AI research is terrifying.
I'd bet you could 10x the number and still be in low single digit percentages of the US workforce. And it seems pretty likely that AI-enabled startups will also employ less people per-startup.
If AI causes a white-collar jobs apocalypse, I don't think startups are picking up the slack, although it'll plausibly cushion the blow somewhat for top-performing tech workers.
And so does academia. It's just that instead of AI and robotics, PhD students are thrown onto problems that are in large parts slightly tweaked reconfigurations of similar experiments.
Especially in chemistry, biochemistry, material sciences there is a large space of discoveries that are barely "novel" in an intellectually stimulating way, but still highly valuable that can be explored orders of magnitudes faster than is currently the case.
Then again, gassing rats and taking biopsies is not something you can do with AI.
Ambitious goals and new discoveries happen via novelty-based search. Progress in scientific discovery is measured by how different/interesting the outcomes are, not by closeness to a predetermined goal.
Discovery is a creative search that preserves optionality, whereas optimization restricts optionality. In other words, you usually don't discover anything novel unless you're trying new things that don't appear connected to the goal in the first place. Would an ML optimization loop have discovered transformers?
1. There’s some irreducible costs in terms of time and material in the physical world that are not amenable to the kind of optimization or parallelization or even just the raw speedup from Moore’s law or computational architecture improvements we’re used to with software. My experience is primarily in biology, where the examples here are things like “it takes 20 minutes for E. coli to replicate” - it has taken 20 minutes for E. coli to replicate for a billion years, and next year it will still take E. coli 20 minutes to replicate, no matter how good your software stack is. Similarly, it takes X amount of energy to grow enough E. coli to produce a meaningful result, and that energy costs money, whether it’s in the form of glycerine or heat or whatever you want, and that also won’t materially reduce in the same kinds of “orders of magnitude” sense we’re used to from software, which is what we’re usually expecting to make the economics of these things work out.
2. Complicating the above, physical systems are phenomenally multivariate - far, far more than you think, and biological systems especially are just unbelievably complex - which means the number of experiments and the length and duration of those experiments you need to run to get enough data to be reasonably confident you’re seeing genuine signal is Way higher than you think.
Combine those two things and what you get is a money furnace, even before you get to the AI model training part, which is Also a money furnace. There’s low hanging fruits in all this, there’s areas where automating the approach can be really valuable, but typically the moment you turn this machine on, you’re gonna start burning money at a rate that would embarrass a finance bro on a coke bender, and that’s effectively unavoidable because the real world is not amenable to software’s scaling laws.
If I had to bet my money, it would be on "for worse".
To many people, myself included, who have to wade through huge amounts of low-quality AI slop which is actually a negative value: no-one benefits when non-experts fire-off one-shot LLM/agent prompts to produce PRs, reports, documentation or "journalism" riddled with imagined truth and factual errors - and the more that people like me have to evaluate these inputs for our job and how wrong they are it pisses us off - but also it means we pick-up on the hallmarks, tells and cliches of these low-effort, no-respect submissions - and now the litany of tells includes this beige-themed, blurred-backdrop-navbar corporate website look: theirs site looks like the 4 or so other LLM-generated, negative-value slop-farm sites I've wasted time on recently - all over the past few weeks.
So I'm saying that, without having known anything about what "Discovery Loop" is - or is not - but landing on their site and and seeing that beige colour and blurred-backdrop navbar, I immediately moved to close the tab; what kept me here was seeing the HN thread had over 100 comments by now and read more about it; if not for that then I wouldn't have given it further thought.
Having "that" beige site look with same the overused looks is either an unintentional indication that the site's author used a low-effort AI prompt to generate the site and that the content within is likely to be low-quality, low-value - or it's an intentional lure to appeal to those who uncritically share in the AI psychosis and so, I assume, are a good target to seek investment from even if it means losing the audience of cynical Internet critics like myself because they know people like me won't be breathlessly repeating their vision-statement on LinkedIn and throwing money at them - kinda like how scam emails intentionally include mistakes for better audience selection. And both possibilities have unpleasant implications.
------
Anyway, regardless of the background of the team behind it, the way the project is described sounds exactly like the recursive-self-improvement and simulated-science thought-experiments from _that other website_ - it's the kind of thing I expect Angela Collier to brutally takedown in an amusing video.
(Though, I do wish people would use just a few extra prompts to break out of the 'vibe-coded' look.)
holy shit. I've known this, but...
https://www.ycombinator.com/library/Vy-jeff-dean-the-1-rule-...
[1] https://hr.ucmerced.edu/hr-units/talent-acquisition/senate-b...
[2] https://www.adp.com/spark/articles/2023/03/pay-transparency-...
Models are commodities the applications eg. BaseTen, OpenRouter should capture the value.
https://taikhooms.substack.com/p/why-openrouter-can-be-the-n...
If youre doing anything high value (advanced research, classified work, high value industrial research, health data) then sending your data through a third party like that is insane.
I doubt numbering vs names on TPU releases even crosses Jeff's radar. It's not the kind of thing he cares about.
https://en.wikipedia.org/wiki/Sense#Artificial_sensation_and...
For some of the other things, undoubtably yes.
https://www.geekwire.com/2026/the-startup-idea-that-convince...
> we have pioneered massive scale computing and led the creation of critical infrastructure, products, and foundational AI advances that the world relies on, including multiple generations of Google Search, Google Ads, Google News, Google Translate, Google File System, MapReduce, BigTable, Spanner, TensorFlow, Pathways, TPUs, AlphaChip, AlphaStar, AlphaCode, AlphaFold, Gemini, model distillation, mixture-of-experts model architectures, word2vec, sequence-to-sequence models, chain of thought reasoning, neural architecture search, and multiple generations of Large Language Models (LLMs) among others.
I feel the most exciting development these days is self-evolving agents. Especially if you have a way to verify their outputs with a formal system, or with a system developed since the 60s by armies of PhDs.
DeepMinds Gnome is a good example, where they use DFT to verify outputs. Approximating NP-problems is always fun for those who dare.
I am also building in this space. Its a mix between HPC, AI, and hard science. Pretty fun compared to waking everyday to LLM news that seem more like marketing stunts.
They are straddling the line between pushing it forward, and justifying the business case. It's hard to do both at the same time.
If throwing more money at inference while accumulating compounding technical debt is the new norm, then we are not solving the problem, and the solution space is already covered.
Perhaps there are marginal gains at the expense of quadrillion-params LLM models with 10x the cost and energy. We are simply making inefficiency more expensive, camouflaged by VC money and great marketing.
If that is not plateauing, then I guess I will have to reconsider what plateauing means.
This is such unbelievable revisionism! Can you imagine in 2022 saying "Oh of course you can brute force your way to AGI if you spend enough money per month". Nobody thought that! Come on!
Imagine a future where only the anointed few elite minds can participate in science and engineering. Btw we’re hiring.
Great message!
Yah, by funding and how we award it, not by an imaginary lack of undergrad and grad students. Scientific funding requires a shotgun approach and many national science funds try to pick winners as opposed to funding broadly. When the folks who researched bacteria in volcanic vents or the molecular biology of the Gila monster they never could have imagined the industries and markets they'd create let alone the lives they'd impact (i.e., PCR and GLP-1 agonists). Lots of grants require you to explain how the work is "translational" or has some sort of economic application (even if not explicitly), but that'll just get us a faster horse-drawn carriage or whatever the Ford quote is.