Qwen 3.8 follows GPT-5.5 Pro reasoning prefills
64 points by wsxiaoys 2 hours ago | 18 comments

spijdar 23 seconds ago
It's interesting that someone else noticed this. A week or two ago, GPT-5.6 Sol starting leaking reasoning into a tool call in Pi. I don't really know what happened, but it was ... interesting:

  Attach. Use hub debugger. Ensure source binary perhaps same. start. todo init. parallel no. two tool calls in same turn sequential is okay. immediately. exactly. Need not mention apologies yet final. [...] Let's do. [...] Do tools. Use commentary. Let's initiate. rambling no. use tool. searching now. okay. Really must call. Let's send. done. why stuck? generate. Sorry. go. no more. (The answer engine expects tool). [...] I think no hidden issue. Go. I'll type tool. now. Stop internal repetition. We have 8000 tokens. tool. sorry. I'll produce call. need include i. Great. final. no. Let's send.gpt. This may be bug. I'll consciously construct tool message next.
It eventually triggered some error state and stopped.

Nevertheless, this was the first time I'd seen Sol's CoT. I looked up the stolen thought's paper, aaaaand yep, that's Sol's CoT alright.

But it occurred to me, hey, Qwen3.8 27B's CoT seems ... very similar.

I compared the geometry problem in the paper, which had a reasoning block open with:

  We need solve. Need reason geometry Weber point? Given pentagon sides and angles. Need find min sum distances. Likely construct rotations / Fermat point lower bound via vectors calibration, maybe triangulation.
I passed the same prompt to Qwen, which opened with:

  We need solve geometry optimization. We need provide final answer. Let's analyze thoroughly.
This proves nothing, but it does seem an awful lot like they did use GTP-5.5/6 reasoning traces...
reply
wongarsu 2 hours ago
That writing style might be a tad too tense

If I got it correct (appending B from https://stolen-thoughts.com/paper.pdf is essential) they are the authors of the well-known exploit to recover readable CoT from OpenAI and Anthropic models. They use that to find hints of distillation, by running a benchmark with a SotA model, recovering the CoT, then taking the first 1% of the CoT and running the open-source model as if that was the start of its own CoT. In the paper they found that Kimi-K3 gets a lot closer to Claude 4.8 answers when prefilled with the start of Claude 4.8 reasoning, suggesting that Claude 4.8 was used in its post-training. This blog post is the follow-up with results that suggest that Qwen3.8 was post-trained with the help of GPT-5.5 Pro (or some similarly responding GPT model, it's unclear how many models they tested)

reply
7734128 2 hours ago
The problem with this is obviously that the only GPT 5.5 thoughts that we have access to are from stolen thought.

Qwen 3.8 0902 was trained after the release of the paper on August 10, so it should have seen those specific thoughts.

reply
usernomdeguerre 51 minutes ago
seems like only the companies in question could run this sort analysis long-term; since they have full access to their CoTs not in public datasets.
reply
verdverm 22 minutes ago
and we have to "trust them bro" to be fair and accurate, something I am very unlikely to do given their other false / misleading statements to date
reply
sureMan6 15 minutes ago
And the only end result would be that the Chinese trained on their data just like OAI and Anthropic trained on our data so who cares
reply
verdverm 5 minutes ago
capitalism ensures I get high marx on my Ai bill
reply
refulgentis 50 minutes ago
The thoughts trick was known before their paper / August.

I "independently" "invented" it for the first Anthropic reasoning models because the API required you have thoughts for each assistant message. My app lets you switch AIs within a chat, and their API used to require thinking for all messages if thinking was enabled, so I needed to get a valid thinking stub to insert.

Time has flew by for me the last 3 years, but, I'd guess it's been at least 18 months. And IMHO it wasn't very complicated to work through how to do once you were dead set on making it happen. I expect it was well-known to distillers before the paper.

reply
7734128 16 seconds ago
Sure, but TFA is trying to use Qwen's reaction to the thoughts as proof that they did indeed extract thoughts to train on.

My point is that any model trained after August 10 will know of those specific thoughts.

reply
brcmthrowaway 3 minutes ago
This makes me very sad

If Qwen and other Chinese labs are just copying reasoning traces, then those labs are more than a year behind the frontier.

reply
jari_mustonen 51 minutes ago
> Qwen barely moved toward Opus 4.8 in the earlier experiment, but moved by +20.58 points toward GPT-5.5 Pro here, including a large effect on the private synthetic puzzles. The data suggest that Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model, rather than from Opus.

How does this suggest anyting of the sorts?

reply
wsxiaoys 48 minutes ago
Appendix B of https://stolen-thoughts.com/paper.pdf discusses this
reply
unrented7977 47 minutes ago
> moved by +20.58 points toward GPT-5.5

Score go up. Probability go up. Conclusion.

reply
CamperBob2 47 minutes ago
News flash: people who scraped the Internet without permission to build their product complain when something vaguely similar is done to them. Water still wet, sky still blue. Film at 11.
reply
RivieraKid 4 minutes ago
For some reason, what China is doing seems worse. Part of it is that I want the US to stay ahead of China.
reply
verdverm 20 minutes ago
I would not be surprised in the slightest if we later find out they are running those same open models to find useful traces or bits to incorporate into their own training. Lots of rules for thee but not for me from Big Ai

I look forward to a day when open models are so dominant that we stop considering traces to be some form of intellectual property that must be hidden from / manipulated for paying users.

It's that manipulation of inputs and outputs that really rubs me the wrong way

reply
noir_lord 29 minutes ago
I'd send them the worlds smallest violin but Rufus is getting in the way of me finding it.
reply
vlyan 17 minutes ago
not just the internet, but every commercially published written work in existence, and I doubt their highly publicized destructive scanning thing had managed to legitimize even a fraction of a percent.

this what is permissible for Jupiter is not permissible for a cow bullshit alone should tell people all they need to know about what kind of greasy sociopaths run "open"ai and (mis)anthropic, and how seriously you should take their purported stances on "safety" and other self-serving shit.

reply