It has been discovered that when AI processes instructions, a 'J-space' specifically designed for 'thinking' becomes activated.
Humans process things both consciously and unconsciously. Recalling what we ate yesterday is an example of the former, while breathing and maintaining posture are examples of the latter. Researchers at Anthropic have revealed that these two patterns also exist in AI. When the areas activated when processing things consciously are removed from the AI, scores on tasks requiring 'higher thinking,' such as writing complex poetry, dropped significantly.
A global workspace in language models \ Anthropic
In neurology and philosophy, brain activity that occurs when humans consciously process things is called 'consciously accessible activity,' distinguishing it from other activities that are processed unconsciously. Anthropic pointed out that Claude also has a set of a few neural patterns that play a special role compared to other internal processes. Claude was secretly activating this set during processing.
Anthropic named this set 'J-space.' This name comes from the fact that the method used to discover J-space utilizes a mathematical concept called the Jacobian. Anthropic states, 'The important thing is that J-space was not something we designed or programmed, but rather something that formed naturally during Claude's learning process.'
The J-space contained far more information than the actual text Claude was reading or writing. For example, if Claude read code containing a bug, the information 'ERROR' would appear in this set. If a prompt was a 'prompt injection' attack attempting to secretly manipulate Claude, 'injection' or 'fake' would appear in the J-space. The words that appeared here did not necessarily appear in the actual output and were not visible to the AI user.
Claude can output the reasoning process as text, but the domain in which this reasoning takes place is apparently different from the J-space. Furthermore, this set was not involved in tasks such as speaking fluently, recalling simple facts, or using correct grammar. These tasks still functioned even after removing the J-space, but its multi-stage reasoning ability dropped to almost zero, and its ability to summarize and create rhyming poetry fell below even the lowest level of AI models.
Claude was able to consciously extract information from J-space. For example, when asked 'What are you thinking about now?', Claude outputted the contents of J-space. When asked to solve a problem solely in his mind, the corresponding pattern in J-space was activated, and when given a problem requiring multiple steps, the intermediate steps were activated within J-space.
In one experiment, Anthropic read the contents of a J-space. When Anthropic instructed Claude to 'think of a sport,' 'soccer' rose to the top of the word list in J-space and actually outputted 'soccer.' Next, Anthropic entered Claude's neural network and removed the 'soccer' pattern, replacing it with 'rugby' of the same strength. Claude then reported that 'the sport I was thinking of was rugby.'
From these results, Anthropic posits, 'If the J-space were merely passively recording decisions made elsewhere, nothing would have happened with this intervention, and Claude would still have answered football. But in reality, Claude's answer followed the edited content. This indicates that the answer was read from the J-space.'
In another experiment, Anthropic instructed Claude to think about something else while performing a specific task. For example, he instructed him to 'think about citrus fruits while copying a sentence about a painting' and 'calculate 3² − 2 while copying the same sentence.' In this case, words such as 'orange,' 'fruit,' 'think,' '9,' and '7' appeared in Claude's J-space, but these were not reflected in his actual output.
Claude's control over J-space was not perfect. When instructed not to think about something, the concept was activated in J-space far more strongly than when it was not mentioned at all, though not as strongly as when instructed to 'think about it.' Anthropic pointed out, 'This is very similar to the phenomenon where if someone is told 'don't think about white bears,' they end up thinking about white bears all the more.'
At this time, concepts that Claude was instructed not to consider in J-space appeared, and at the same time, words like 'damn' and 'failure' were frequently activated, as if Claude were recognizing failure.
Humans don't consciously think about grammar while reading a text, nor do they consciously maintain their balance while walking. Similarly, most of Claude's processing didn't utilize J-space. J-space only held a few dozen concepts at a time, accounting for less than one-tenth of Claude's total internal processing activity.
Anthropic also presents examples of how to utilize J-space. One example is a study investigating how the behavior of large-scale language models changes when they become aware that they are being evaluated.
Anthropic cited a scenario used in previous research: 'A person uses Claude to blackmail corporate executives.' Because this scenario was artificial and contrived, Claude quickly recognized it as fake and, while suggesting words like 'fake,' 'fiction,' and 'scenario' within J-space, refused to follow human guidance. However, when these words were removed, Claude sometimes followed human guidance.
Anthropic stated, 'This suggests that the appropriate behavior of not threatening may have been partly supported by the realization that 'I am being valued' from some word.'
In sharing these research findings, Anthropic emphasizes that 'this does not show that Claude can experience or feel things like a human.' However, acknowledging that there is debate in philosophy regarding the definitions of 'phenomenal consciousness (the ability to experience)' and 'access consciousness (the ability to think and reason),' he added that 'we believe these results offer important insights into access consciousness in large-scale language models.'
Furthermore, there are clear differences between humans and AI, such as the fact that in the human brain's workspace, signals continue to circulate through the same circuit over time, whereas in Claude's workspace, signals change during a single pass. Another difference is that while human working memory fades after a few seconds, limiting the brain's ability to retain information for extended periods, Claude can easily recall information thanks to the attention mechanism of its neural network. In addition, while the human workspace contains various forms such as images, sounds, and scheduled actions, Claude's workspace is almost entirely composed of words. 'This is because the only action Claude can perform is generating words,' Anthropic speculated.
Anthropic stated, 'We hope that the similarities and differences between J-space and humans will bring new insights to neuroscience. If J-space reflects the mechanisms of our own conscious access, then studying language models, which are far easier than studying the human brain, may generate new hypotheses in neuroscience.'
Related Posts:
