https://developer.nvidia.com/blog/nvidia-transitions-fully-t...
Nvidia does something to prevent abuse so that regular people who want the laptops can buy one. Nvidia gets knocked on.
When did "we" have a say in this? I don't get these types of comments from the "open-source as a community."
Nvidia has been doing this for decades, and there are two possible arguments:
1. They are successful despite it.
2. They are successful because of it.
There is no mythical collective "we" representing the open-source community in Nvidia's supply chain of success. It's Nvidia and its consumers.
Absolutely hilarious.
This is just incompetence. No reason Nvidia with its TRILLIONS of market cap couldn't of tested and fixed this on Linux before shipping.
Or a day one patch.
I have no doubt that'll it'll work right eventually. Long after RTX Spark laptops flop and can be purchased for half off.
Last night I pulled out my 5070TI laptop and installed CachyOS. I was running various models within about 20 minutes.
That 12GB of VRAM is a hard limitation, Unsloth can't split jobs between vram and system memory.
Man it really is exhausting to read a title that sparks my interest, just to be greeted with the most basic Claude-isms in the subheading.
I just heard a quick J. Talarico speech at some country concert. In the 10 or so sentences he spoke, he had 3 or 4 instances of doing this "Claudism" enumeration. Are you saying Texas just elected Claude for Senate?
Maybe we should stop with such superficial judgements. It's the modern equivalent of not liking the speaker's tie. (Doesn't mean the speaker's speech is good, but it doesn't mean it's bad either.)
Edit re "equivalent to not liking the speaker's tie": This is not equivalent at all, this is a written blog post and I'm put off by the way it is written, not by say the URL or their chosen font. It's more akin to a speaker dropping an inappropriate joke at the start of their talk and leaving because of that
This is a new thing with the firmware that can be used only by Windows. If this NVIDIA policy continues, then NVIDIA will indeed become a bad choice for Linux, but until now, for graphics and AI applications NVIDIA has been the best choice for Linux, by providing the best performance and reliability.
For pure computational applications, especially for those requiring good FP64 throughput, there are many years since NVIDIA has become a worse choice than AMD and more recently also than Intel, but that is due to poor hardware support, not to bad software support.
Those are very different things as far as I understand, and the whole "doesn't work well with linux" really only makes sense on the RTX Spark laptop I think? Isn't the DGX spark linux by default?
> The laptops also support unified memory configurations up to 128GB, shared between CPU and GPU, which is unusual for a laptop and central to why they’re being marketed at AI developers in the first place.
This isn't rare on laptops, pretty default on AMD with iGPU
If you really want to do DIY inference, plug it in at home and set up tailscale so you can access it on the go.
No huge battery drain or toasty hot laptop required. Lower cost laptop. Lighter laptop. Longer battery life laptop. Win win win win.
I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.
Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.
I am not saying don't use a laptop.
I am not saying you have to use cloud inference.
I am saying using a laptop for inference is dumb. If you want private inference:
- use a dgx spark or similar plugged in at home
- set up tailscale so you can connect from anywhere
- use your phone as a hotspot on the go
- work on your laptop, using tailscale to connect to your home inference server if you're on the go
This gets you everything except for the few hours you might be on an airplane.
This has been an issue on Linux since 2014, and the gtx980.
Also very curious if the NPU is visible in Linux, too.
https://www.xda-developers.com/nvidia-rtx-spark-laptops-same...
If you want to prosper with the current of nature, provide a thing easily for mutual benefit. The more you try to block a thing, the sooner your demise into uselessness.
A laptop can't run the latest big models, because memory is too expensive. This thing probably can't run models larger than Mistral Small 4. But it'd be a completely different thing to have a hardcoded Mistral Large 4, than a choice of tiny models.
I think I misunderstood what the technology could do a little bit, because I imagined these fixed matrix multiplies encoded into hardware as small-ish, as using routing space etc., not just fast.
I think what I really imagined was useful would be more easily achieved by having a big ROM with model weights connected to an ordinary GPU.
But with Nvidia people just expect they nerf all the features.
Still miles ahead compared to Apple, where linux does not even boot on newer macs!
Or maybe just not at all - shell out for OAI tokens and send us all your data - your privacy is very valuable to us and we will make sure to be the only ones to monetize it.
You could say, Nvidia is selling pickaxes and has no business gatekeeping what rocks you hit with it, but I guess it depends on their relationships with major mining companies, and ultimately what pays more.
But, on the other hand, having my DGX Spark stack at hand, I know the solution:
/btw reverse engineer the Nvidia Platform Controllers and Framework component and build a Linux version
Sovereign AI has never been more important.
They really got it right on multiple levels - for those who don't know, roller coaster tycoon was infamously written in 99% assembly and meticulously optimized by the developer to run effectively in 16 megabytes of ram.
Sovereign AI has never been more important.
Controlling AI is so much more powerful and impactful than controlling search or social algorithms.
Centralized AI a 10x on social control over everything we have seen before, and there is no way that a free and open society survives contact with centralised, high capability AI if the system is manipulated towards guiding outcomes. It’s essentially mind control, as people will offload an even greater share of their cognition to AI than they do with search and social.
Build and maintain sovereign AI while you still can.
Nothing stops you from giving Nvidia half a million for their latest and greatest AI GPUs, unless you are in China I guess, but China does very well despite the export control.
It is just classic market segmentation, the thing Nvidia and just about every company that can benefit from it has done for decades. Cloud providers are ready to pay a lot of money, consumers wanting local AI have a lower budget, and gamers even lower. But a low budget is greater than zero, and there is still money to be made on these markets. But they also don't want to compete against themselves, so they are going to find ways of crippling enterprise-class workloads on consumer hardware.
It used to be that they crippled high precision arithmetic on consumer hardware, because it was a differentiating factor, but AI doesn't care about high precision, so now, it is something else.
Who exactly?
Pay up for license to unlock full capacity, pay up for license to use encrypted models from AI moguls, paid per billion tokens processed or just operations in GPU
I wonder if this space is explored fully enough at the moment.
Aside from some implementation details, it's conceptually much easier than doing it for a SQL database.
The consumer has access to free and open models for local use.
The supplier has better control and wider market to serve running the models in data centers.
Only way I can see encrypted models being widespread is if they are bundled as part of a software package (OS/Browser/etc) and not sold as the product.
So it is something that has been done for a long time now. Still feels like unethical rent-seeking that should not be tolerated by the customer.
See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.
Yep, "Batterygate"
https://www.npr.org/2020/11/18/936268845/apple-agrees-to-pay...
It saves the car manufacturer on logistics cost. The per-unit cost of a seat heater is (much) smaller than the cost related to keeping two variants of seat in stock and to provide spare parts. Say the heating mat costs 100€ per seat including materials and installation, and can be sold to the customer for 400€ permanent, that means as long as 1 in 4 customers buys the seat heating option, the books are balanced, and all money you save on logistics is pure profit.
Adding subscription on top of that is financially even better, if there is one thing companies LOVE thanks to stonk market incentives it is monthly recurring revenue.
I think more than that, it is probably a lie they tell us that you are not paying for the hardware/equipment.
AI is a software just like operating systems and executables where open source absolutely dominates the industry. Remember when the compiler engineers optimized the shit out of llama.cpp.
Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.
It's a waiting game. Open source always wins.
Tons of stuff happenning behind closed doors today though. No more stackoverflow for the next decade, that could create a problem if all new context only exists in chatgpt conversations only openAI has access to (or whoever).
[Hanlon's Razor] This is such a contrived performance drop scenario that I find it hard to believe it was done with the purpose you're inferring.
I'd interpret this as Nvidia not caring that much about Linux desktops/laptops once again (as opposed to Windows desktops/laptops), rather than anything to do with local vs cloud.
As in, make your own inference and training hardware?
I stopped using my laptop for dev and switched to talking to claude code on my phone and as you can imagine I am faster at producing code.
100% cloud with a single entry point device to manage dedicated servers + agents