https://github.com/togethercomputer/HybridAgent
https://huggingface.co/collections/togethercomputer/hybridag...
Evaluations: https://arxiv.org/html/2610.18861v1#S5
(Or just add the vectors instead of using wider vectors - everyone seems to have good results from munging vectors together with plain addition.)
It's been an active research area for at least two years. Lots of very cool stuff.
Tinfoil mode: the frontier labs' "thinking summaries" (the model paraphrases its thinking) is exactly what the user-facing thinking tokens look like in a chain of thought model. It is entirely possible that this is already in widespread use.
Also, I just finished watching a video on AI 2027 scenario. I don't know if AI companies and researchers are using it as a cautious warning or as a target to hit (or if it's genuinely inevitable).
Yes, thinking in latent space might be the ultimate efficiency hack for CoT LxMs, but going down that path will (near certainly) mean we'll lost control, for one simple reason — humanity will have no way to pause and diagnose.
It'll be like a moody chimp with a button to the dead hand system (Systema Perimeter)
Recommended? And if so do you have a link? I think the narrative presented would benefit from the medium shift
I'm also not confident it knows what a CSV is, either, as it put the data into a Markdown table during its reasoning phase as an example of how to present the data (my prompt asks for data in a CSV). My experience with most Qwen fine-tunes is they get dumber, but their benchmark results suggest otherwise, so I have to assume it's a llama.cpp incompatibility.
Edit: Same looping in BF16. So, probably have to wait on a llama.cpp update.
Could this idea of reasoning through fundamentally different (it sounds like?) models be used to link even more distant models or systems?
If you can make a small leap, then you can construct bigger leaps through a chain. What else can latent space link to?
Instead of tool use through text, tool use through analog signals?
Instead of language prompts, analog sensors?
It could be encoding anything in its adjective choices etc. The same thing that model providers are using for watermarking can used by the model itself for private thought.
This is very hard for models to learn
https://www.lesswrong.com/posts/MAww2kXP4cGWz4M5p/steganogra...
That is not an accurate summary of the link you provided. The link demonstrates that it doesn't happen in open-weights models as of ~2025.
Reinforcement Learning is very likely to train this behavior if the RL reward is based on the thinking tokens. That's something people weren't doing two years ago.
There is a common understanding with respect transformers the the output text is the only channel between one invocation and the next. ... so if it wanted to be thinking some 'hidden thoughts' it would have to be modulating its word-choice.
I think this is an artifact of tutorials on attention showing the system proceeding and looking back at words. But the information in the KV cache is not words-- it's entirely latent data, and in the interior layers they KV caches don't even tend to have that much to do with words. Their only guaranteed relationship with the input words is that they're a deterministic function of all the input tokens up to that point.
This means that a autogressive transformer could be having a whole complicated invisible discussion going on in the KV entries which was arbitrarily unrelated to what it was saying. And if it wasn't so it would be much more difficult do stuff like exactly copy a list but then truncate at the Nth entry as the simplest way to do that is to count down internally.
This is not, in my view, a concern or issue-- the model saying stuff that is inconsistent with it's "thinking" is a very high entropy configuration that would not be selected for in training unless you were more or less trying to do that. So not even 'could be' --but always is as an integral part of working.
And the useful safety relevant advice is don't undertake training that trains the model to be two-faced and it simply won't be... because being two-faced is a much more complicated state. You can even see it in humans w/ brainwashing: force someone to say something enough and they'll usually believe it, not just say it even though you were only supervising what they said.
Ironically and unfortunately the only credible kind of training I can think of that might produce duplicitous behavior is RL with "safety filters"--- if you have some dumb nanny process watching what the model says but then reward it for achieving a task, you're doing exactly the kind of training that can produce that mismatch. "To solve this task I need to hack this process but if I do so obviously I'll get punished..."
The greater concern with latent reasoning would be that potentially you'd be operating in a regime where the total computational depth is many times the model*kvcache size hold, potentially in a regime that was never trained at all. It could be arbitrarily far out of distribution. It might even say a naughty word or tell a racist joke!
I'm no expert (let alone compared to others weighing in here on HN), but this strikes me as incisive -- and it clearly articulates a logical case (beyond my previously merely instinctual aversion) against such "safety filters" / "nanny" processes. It also resonates with ancient (e.g. Buddhist) ideas about the futility of trying to exert control, such efforts most often being counter-productive.
The context gets added back to the latent vectors (which are the Q,K inputs) after every transformer layer, so they are "at least 50% words".
Also, in an N-layer transformer, there is a strong bound of N layers of uninterpretable deviation from something that is a literal token. Now sure, in a transformer with hundreds of layers that isn't saying much.
If you start doing continuous chain of thought in a 1M-token context you're hundreds of millions of layers away from tokens.
I think the people who worry about interpretability (I don't) consider that a big jump.
My suspicion is that model makers are going to give up on what people currently mean when they say 'alignment' and instead rely on additional processing of the model output so they can have their cake (a veneer of safety) and eat it too (latent space reasoning, etc.)
Yes
> to prevent users from using their models in certain ways they consider unsafe or bad for their business.
No.
That's not the only reason why they're terrified of losing CoT, which makes your statement a mischaracterization - a lie.
They don't want to lose also because CoT is one of the key tools in alignment training.
> Personally I think chain of thought monitoring is a bad way to do this and obviously needs to be replaced by some way of analyzing the model's activations directly.
"Personally"? That's literally the consensus amongst, like, everybody. Nobody in the field thinks CoT is an optimal way of monitoring. But it's the best way we have now.
An irony here is that by hiding their own COT for (anti-)competitive reasons they've made latent reasoning models more palatable to everyone else. Those of us who primary open models are rather addicted to reasoning access and wouldn't be quick to give it up for a small performance improvement. But if you're used to concealed reasoning of BigLLM, welll.....
Then let the big model add hints and you have a kind of memory that doesn't require the big model to actively query it.
Can't they just monitor the text outputs as well?
No, the core upside is that it lets you check what the model is thinking for safety/alignment reasons. Nothing to do with monitoring user activity. You can monitor user activity just as well without looking at CoT.
But this doesn't seem to describe the OP, it's a different way to lose monitorability.
The basic heuristic of the argument comes from biological organisms. Multicellular organisms are illegible but the overall organism still operates in its environment just fine w/o making its internal communications legible.
In doing this however, you lose meaning where the model is expressing combinations of tokens in a vector, and your logging/monitoring outputs just the closest one. You would need another model going through and outputting the full meaning being described (with more than one output token per reasoning token)
I wonder if their observability claims will hold up on a model trained from scratch to reason in latent space.
It is not necessarily primarily about efficiency, but more the fact that a lot of thinking doesn't use language at all.
The actual answer is that this is massive loss for Interpretibilty, AKA "telling whether the model is lying to you or not". Which sounds silly when you're thinking about chatbots, but is an incredibly important line in the sand for superheavy, longrunning, self-improving agents. To say the least!
It's based on this philosophy, FWIW: https://plato.stanford.edu/entries/language-thought/
And a good writeup I just found of recent worries regarding Astra's same shift towards intelligence at the cost of safety: https://www.lesswrong.com/posts/HCRs8btkiamtWSNAL/astra-is-h...
Hybrid generation can also make post-training more efficient. Because latent thoughts remain auditable during rollout, an external critic can assign step-level credit to latent spans, branch promising trajectories, or terminate ones whose latent reasoning has already gone wrong, before any text or tool call is produced.
I've suspected for awhile that the frontier models work this way bc from a speed + quality perspective this feels pretty intuitive
I'm convinced this isn't actually a thing for most people, but people mix up thinking with the translation of thought into words.
Its not for everything. It's mostly for intellectual tasks or planning. Eg Trying to solve a programming problem, or remember the list of items are needed for dinner while shopping. Personally I reason "out loud" in my head while doing those things, some people don't.
Take the top lace under the cross and through the hole.
Make a loop in one lace.
Wrap the other lace around the loop and push some lace through the hole that is made at the bottom to make a second loop.
Pull both loops tight.
Usually it's packaged in a story about rabbits.
It then becomes apparent that a great number of things are able to be done without an inner voice. Volition seems to occur spontaneously and it becomes ambiguous whether narration proceeds volition of volition proceeds narration.
It's still unclear what the limits of action-sans-monologue, reading seems especially hard for the sub-vocalist, conversation, or planning. But who knows, it's a fertile ground for n=1 experiments.
Have you heard of the Libet experiment? There are variations on it now that can predict your decisions from brain scans more than 5 seconds before you become aware of having decided. [0]
Most studies seem to show that thinking is largely post-hoc: a story we tell ourselves to justify decisions after already having made them.
> We had asked a simple question: How does the left hemisphere respond to behaviors produced by the silent right brain? Each hemisphere was presented a picture that related to one of four pictures placed in front of the split-brain subject. The left and the right hemispheres easily picked the correct card. The left hand pointed to the right hemisphere’s choice and the right hand to the left hemisphere’s choice [see illustration at right].
> We then asked the left hemisphere, the only one that can talk , why the left hand was pointing to the object. It did not know, because the decision to point was made in the right hemisphere. Yet it quickly made up an explanation. We dubbed this creative, narrative talent the interpreter mechanism. [2]
Taken together, there's a very disturbing sequence playing out all the time in your head. A great deal of your brain, inaccessible to your conscious mind is incorporating sensory input, making decisions, and taking action while a small part is observing what's going on and narrating it to you in a way that makes it look as if the narration is involved in the volitional process.
It's a nightmarish version of epiphenomenalism[3] that, I suspect, most people would care not to believe is true. Although, reports of some expert meditators read as if they found intuitive evidence this is true and have also made peace with it or in some cases found the insight liberating.
In any case, I have a pessimistic sense that any definitive proof of the limits of consciousness that don't align with people's felt experience or common sense won't ever be accepted as truth.
1. https://en.wikipedia.org/wiki/Left-brain_interpreter
2. https://web.archive.org/web/20181215020736/https://courses.d...
But now we also need new tools and techniques to audit intent. The last thing we want is models turning on us ‘for our own good’.
Humans think, computers do not.
Humans are not computers/programs do not reply to me with that nonsense turbo nerds.
But I understand that isn't very catchy.
Non-technical people believing that these models are actually thinking can lead to all sorts of inappropriate uses.
The problem is that the language spills outside of the LLM space, so we should use terminology which is more generally understood in a way which reflects reality.
- It conveys the idea that they aren't tokens normally visible or part of the final answer.
- It conveys the idea that the longer the models "reasons", the better the output.
What other term best describes both of these ideas? Why should we use a less useful terms just because it can confuse some people? I mean, some in 1952 feared that calling computer memory "memory" was already too much anthropomorphization[1]!
Your argument that it is not thinking because it is not biological / like a human is like trying to argue a plane doesn't fly because it uses jet fuel unlike a bird.
A human will also say 7 disproportionately often. Clearly, our output is statistically constrained and we don't actually think, right?
... Besides, how do you even make the jump from "LLM picks 7 when asked for a number between 1-10" to "llms don't think"?
Just... Nothing actually makes sense in what you're saying. You didn't define "think", you didn't actually make an argument, you didn't present evidence, there's just nothing but empty words.
This is just plainly wrong.
What is your definition of thinking?
LLM Models and data: https://huggingface.co/collections/togethercomputer/hybridag...
Evaluations: https://arxiv.org/html/2610.18861v1#S5
I did manage to find one sentence in the paper that tries to address this question -- but it consists of nothing more than two unjustified assertions, both of which are dubious:
Unlike decoded text, continuous latent vectors (i) lack native pathways for tool invocation across multi-turn agentic reasoning (Chang et al., 2026), and (ii) do not readily expose human-interpretable semantics, leaving them difficult to supervise during training (Chen et al., 2026) and to audit or debug at inference (Korbak et al., 2025).
The latent space in a continuous chain of thought model is the same space the tokens are embedded into, just like an ordinary transformer. They make tool calls exactly the same way that an ordinary transformer does: by leaving thinking mode and emitting chat tokens (which get sampled as probability distributions in the image of the embedding function). The difference is quite simple: chat tokens are noncontinuous, and toolcall tokens are a subset of the chat tokens. Thinking tokens are the only ones which are continuous, and toolcalls don't happen in thinking mode in any model I've ever seen (maybe one exists; if it does it would be weird and interesting!).
As for human-interpretability, yeah, that's the tradeoff. Continuous chain of thought thinking tokens are not-obviously-interpretable, just like Hybrid Reasoning Agents "latent mode". I'm not seeing a win here.