Hacker News

49 points by zdw 5 days ago | 18 comments

speedgoose 41 minutes ago

Time to first token is a very important performance metric, as I figured out using a Mac Studio M3 Ultra (that is quite slow on this aspect).

But 32GB for a TDP of 230W is perhaps not super interesting. Especially because you probably want to have more than one card. It's a lot of heat. You could use the cards for heating up a building, but heatpumps exist.

bigyabai 27 minutes ago

A lot of the TDP is reserved for running the shader units at full-power. My RTX 3070 Ti only pulls ~110w of it's 320w running CUDA inference on Gemma 26b and E4B.

Scaevolus 20 minutes ago

It's not that it's reserving power, but rather that you hit some bottleneck on a 3070 Ti before running into thermal limits-- it's likely limited by either tensor core saturation or RAM throughput. Running the workload with Nvidia's profiling tools should make the bottleneck obvious.

driverdan 29 minutes ago

From what I've read the Intel drivers are terrible and holding back using them for LLMs.

martinald 24 minutes ago

Don't think that's true. The drivers are bad (not sure terrible is fair, they have improved a lot) esp for older directx etc games. But Vulkan support is pretty good and that's all you need for LLMs really.

999900000999 26 minutes ago

Everyone has terrible drivers here aside from Nvidia.

Intel looks like they'll leave the dedicated GPU space, so it's a bit doubtful if the drivers will ever catch up.

tempest_ 48 minutes ago

I would like one for the vram but I am sure they will be unobtainable after the initial stock sells out as I assume they were produced before the RAM prices went up.

SparkyMcUnicorn 33 minutes ago

Here are some llama.cpp benchmarks for it: https://www.phoronix.com/review/intel-arc-pro-b70-linux/3

zargon 12 minutes ago

Also from phoronix, a comparison with AMD R9700 and RTX 6000 Ada (because Nvidia has not sent them a blackwell card): https://www.phoronix.com/review/intel-arc-pro-b70/2

MostlyStable 44 minutes ago

Is Intel still making GPUs? I have heard so many conflicting things about will they/won't they stay in the market.

2OEH8eoCRo0 20 minutes ago

I don't know what to believe when it comes to Intel news because they have so many haters.

dismalaf 27 minutes ago

They'll always have iGPUs so whether or not they stay in the dGPU market depends mostly on whether or not people buy them. So they might not, whole market seems to be moving to SoCs/APUs/whatever you want to call them.

XCSme 46 minutes ago

Can you use those AI cards for gaming too?

Or the makers intentionally nerf them, in order to better segment the markets/product lines?

wmf 5 minutes ago

They nerf gaming cards to make money on the pro cards. Since this is a pro card it's not nerfed.

ZiiS 33 minutes ago

The drivers often need per game optimisations these will be missing but I doubt Intel would nerf them, just rely on you not paying a lot for RAM the game won't use.

XCSme 14 minutes ago

I actually meant it in a different way. I would get it for local AI stuff, but being able to game on it would be a huge plus, otherwise I would need two different machines.

100ms 2 hours ago

These seem amazing for hobbyist, but that TDP given the perf might be an issue deploying a lot of them

zrm 57 minutes ago

Its performance is pretty unbalanced. If you're using it for the couple of things that it's good at, the TDP is competitive.