Finding a CPU Design Bug in the Xbox 360 (2018)
137 points by mariuz 5 days ago | 41 comments

chasil 7 hours ago
It is interesting that IBM dominated this generation of consoles, and was vanquished in the next.

The high failure rates of the Xbox 360 did not help.

https://en.wikipedia.org/wiki/Xbox_360_technical_problems

reply
vondur 7 hours ago
I thought the design flaws of the Xbox 360 cooling system had more to do with Microsoft than any inherent design flaw by IBM. I assumed that switching to x86 processors let Microsoft leverage their native developer tools from Windows which helped developers.
reply
chasil 7 hours ago
The main issue was revealed to be solder.

"Microsoft did not reveal the cause of the issues publicly until 2021, when a 6-part documentary on the history of Xbox was released. The Red Ring issue was caused by the cracking of solder joints inside the GPU flip chip package, connecting the GPU to the substrate interposer, as a result of thermal stress from heating up and cooling back down when the system is power cycled."

reply
timw4mail 6 hours ago
And there was the same problem with early PS3s, on Nvidia's GPU package...it was a fairly widespread problem at the time.
reply
rwmj 6 hours ago
And Apple iBook G3s too. There's a whole thing with owners reflowing the GPU: https://www.instructables.com/Fixing-the-infamous-iBook-scre...
reply
keyringlight 5 hours ago
I seem to recall baking PC nvidia GPU boards in your oven was a reasonably common out-of-warranty fix around that era.
reply
Moosdijk 4 hours ago
I had to do this with my MacBook Pro models early 2015 and late 2017.

It seems like there was a period in time when solder just wasn’t done well, it seems like.

reply
rangestransform 2 hours ago
IIRC this is to do with the phase in of RoHS and bad lead free solder
reply
hbn 6 hours ago
I don't have any solid numbers on me, but I believe early 360s failing wasn't just widespread; it was straight up most of them dying within the first couple years. It's honestly insane they more or less got away with that. And I guess also speaks to how much Microsoft was killing it in that era that people were willing to go through multiple console RMAs (which I heard was a terrible, slow, and unreliable process) to play 360 games. How far they've fallen.
reply
Aarostotle 5 hours ago
Simple answer: Halo 3.
reply
mrguyorama 2 hours ago
It was something like 25% - 50% of all first version 360s died.

Microsoft spent over a billion dollars replacing and repairing consoles to maintain the good brand name of Xbox.

https://en.wikipedia.org/wiki/Xbox_360_technical_problems

reply
rustystump 5 hours ago
Family got first gen 360. Still works to this day. We hit the jackpot with that console. It out lasted 2 wiis and a ps2
reply
kjkjadksj 5 hours ago
Whenever we lost a 360 we got a pre owned 360 from gamestop. I think they went for like $70 for one without any hdd.
reply
bombcar 2 hours ago
That was the real story, by the time they started dying you could just grab a working one for "decently cheap" if you still cared.

However, I wonder how many people got "burned" by it and swore off Xbox consoles going forward.

I know that era we got a lot more use out of the Xbox (original) and the Wii.

reply
thenthenthen 6 hours ago
Sounds like the 2012(?) Macbook Pro after the switch to leadless solder (?). I had to cook my motherboard 3 times in the oven to revive it.
reply
chasil 6 hours ago
Funny!

I've heard that flash memory can also be revived with heat, either long duration or high intensity.

https://www.extremetech.com/science/142096-self-healing-self...

reply
thenthenthen 5 hours ago
Some macbook hacks involved disabling sleepmode, running a benchmark and putting it in a pile of blankets for a few hours
reply
esaym 4 hours ago
When did the industry transition to different/lead free solders? Wonder if that was part of the issue?
reply
monocasa 3 hours ago
Yeah, it was the transition to RoHS.
reply
hrmtst93837 45 minutes ago
Sony stuck with Cell even longer and it locked them into a decade of weird ports and missed optimization tricks, so "vanquished" cuts both ways.
reply
brucedawson 26 minutes ago
It did not help that the performance of the Xbox 360 CPU (and Cell) was terrible. In-order execution without the crazy-high frequencies the CPU was designed for was a failure.
reply
cptskippy 5 hours ago
> It is interesting that IBM dominated this generation of consoles, and was vanquished in the next.

IBM's Power was the only logical option at the time.

These consoles were being designed around 2000. Intel and AMD weren't partnering on bespoke CPUs at that time. I don't even think AMD would have been considered a viable partner. Neither had viable 64 bit options and part of console marketing at the time was the ever increasing bit depths.

Prior console generations had use MIPS which wasn't keeping up with ever increasing performance expectations and players like Toshiba and Sony were looking for a higher performance CPU architecture. IBM's Power architecture was really the only option. Sony, Toshiba, and IBM partnered to develop their a new 64 bit microarchitecture called Cell.

Microsoft's first console was basically a PC and that's how everyone saw it. The 360 was an opportunity for Microsoft to show that it could compete with the big boys. It was also an opportunity to keep a toe dipped in RISC, because it had dropped support for RISC CPUs with Windows 2000.

reply
Grazester 5 hours ago
By the way, the AMD athlon 64-bit launched 2003. The PS3 launched in 2006. I had an AMD64 bit process in my laptop in 2005.

What wasn't viable?

reply
Narishma 4 hours ago
Yeah that part didn't make sense, not to mention that neither the PS3 nor the 360 were running 64-bit software. They didn't have enough memory for it to be worth it.
reply
sidewndr46 4 hours ago
you don't need memory to make 64 bit software worth it. Just 64 bit mathematics requirements. Which basically no video game console uses as from what I understand 32-bit floating point continue to be state of the art in video game simulations
reply
duped 4 hours ago
Fundamentally it's still a memory limitation, just in terms of memory latency/cache misses instead of capacity. If you double the size of your numbers you're doubling the space it takes up and all the problems that come with it.
reply
sidewndr46 3 hours ago
No it isn't. The 64-bit capabilities of modern CPUs have almost nothing to do with memory. The address space is rarely 64 bits of physical address space anyways. A "64-bit" computer doesn't actually have the ability to deal with 64 bits of memory.

If you double the size of numbers, sure it takes up twice the space. If the total size is still less that one page it isn't likely to make a big difference anyways. What really makes a difference is trying to do 64-bit mathematics with 32-bit hardware. This implies some degree of emulation with a series of instructions, whereas a 64-bit CPU could execute that in 1 instruction. That 1 instruction very likely executes in less cycles than a series of other instructions. Otherwise no one would have bothered with it

reply
comex 10 minutes ago
Typically, it doesn't have the ability to deal with a full 64 bits of memory, but it does have the ability to deal with more than 32 bits of memory, and all pointers are 64 bits long for alignment reasons.

It's possible but rare for systems to have 64-bit GPRs but a 32-bit address space. Examples I can think of include the Nintendo 64 (MIPS; apparently commercial games rarely actually used the 64-bit instructions, so the console's name was pretty much a misnomer), some Apple Watch models (standard 64-bit ARM but with a compiler ABI that made pointers 32 bits to save memory), and the ill-fated x32 ABI on Linux (same thing but on x86-64).

That said, even "32-bit" CPUs usually have some kind of support for 64-bit floats (except for tiny embedded CPUs).

reply
bombcar 2 hours ago
"Bitness" of a CPU almost always refers to memory addressing.

Now you could build a weird CPU that has "more memory" than it has addressable width (the 8086 is kind of like this with segmentation and 8/16 bit) but if your CPU is 64 bit you're likely not to use anything less than 64 bit math in general (though you can get some tricks with multiple adds of 32 bit numbers packed).

But a 32 bit CPU can do all sorts of things with larger numbers, it's just that moving them around may be more time-consuming. After all, that's basically what MMX and friends are.

reply
chasil 3 hours ago
The original 8087 implemented 80-bit operands in its stack.

It would also process binary-coded decimal integers, as well as floating point.

"The two came up with a revolutionary design with 64 bits of mantissa and 16 bits of exponent for the longest-format real number, with a stack architecture CPU and eight 80-bit stack registers, with a computationally rich instruction set."

https://en.wikipedia.org/wiki/Intel_8087

reply
monocasa 3 hours ago
Parts of the 360 did. The hypervisor ran in 64bit mode, and use multiple simultaneous mirrors of physical address space with different security properties as part of its security model.
reply
chasil 4 hours ago
I have some confidence that AMD's acquisition of ATI had a huge impact.

That allowed both a CPU and an advanced GPU to be on the same die.

They also wisely sold Global Foundries, and were able to scale with TSMC.

reply
cptskippy 3 hours ago
You have to remember that the AMD and Intel of today are very different companies than they were 20-25 years ago. AMD split off it's fab capabilities, acquired ATI, adopted TSMC as a fab, and developed a custom silicon business.

At that time AMD wasn't in the custom CPU business, AMD64 was a new unproven ISA, and x86 based CPUs of that time were notoriously hot for a console. These were also some of the reasons why Microsoft moved away from the Pentium III it had used in the original Xbox.

The PS3 was launched in 2006 but the hardware design was decided years earlier to provide a reference platform for the software.

reply
dev-ns8 58 minutes ago
Wow... A speculative branch prediction path actually get's preemptively executed despite the branch outcome? No matter if the execution has side-affects??? That's quite amazing. Are modern CPUs doing speculative execution like this and just put extra safeguards around affects or do they just prefetch / decode instructions now-a-days?
reply
brucedawson 27 minutes ago
Author here: This is not a common problem. I think I was told that Alpha had basically the same bug but it is a bug, for sure. Speculative execution causing problematic side effects is a deal killer.

Speculative execution, however, can cause less problematic side effects. For instance, a speculatively executed load or prefetch will usually actually prefetch which will pollute the cache, TLB, etc., and reveal side-band information, but that is a performance problem and perhaps a subtle security flaw, not a correctness bug like this was.

reply
cumshitpiss 38 minutes ago
[dead]
reply
jszymborski 7 hours ago
Would need "(2018)" in the title.
reply
NooneAtAll3 5 hours ago
unrelated, but recently XBox One was hacked for the first time

https://www.youtube.com/watch?v=FTFn4UZsA5U

reply
brcmthrowaway 5 hours ago
How does XBox get hacked when it uses Secure Boot?
reply
Tuna-Fish 4 hours ago
Voltage glitching. An outside attacker who has direct, extremely fine-grained control over the power supply to the chip can cause it to brown out for one instruction cycle, preventing a result of an instruction from being written.

With enough sophistication, physical access is more powerful than root access, no exceptions.

reply
alumno_007 3 hours ago
Sobre calentamiento
reply