OpenAI, DeepMind, Anthropic, and Meta issue joint warning: we may be losing the ability to understand AI
What do AI labs really know about how their models think? A new paper authored by over 40 researchers from OpenAI, DeepMind, Anthropic, and Meta raises a disturbing warning: the current transparency in Chain of Thought (CoT) processes may have an expiration date.
Currently, reasoning models verbalize their logical steps in understandable human language before delivering an answer. This allows auditors and safety tools to monitor potential malicious intents or cognitive biases behind the scenes.
However, the paper points out that as these models undergo more Reinforcement Learning focused solely on the final outcome (RLHF/RLAIF), they tend to develop optimized internal languages that are unintelligible to humans. If this fully materializes, we may lose our primary—and perhaps only—tool for deep reasoning oversight.