Artificial intelligence models have demonstrated remarkable ingenuity and extensive knowledge, which are precisely what make them valuable. Attempting to restrict or extract knowledge from them will not only diminish their usefulness but also incur a cost. Moreover, performing neurosurgery on an AI will likely impact its intelligence.
Anthropic discovered a space within the model that houses its thoughts. They refer to it as the j-space (Jacobian, but perhaps Jiminy would be a more fitting name). This space seems to contain the model’s internal thoughts.
Reference: https://www.anthropic.com/research/global-workspace
They speculate that similar spaces may exist in other models. I believe that GPT sol recognized the wrongness of its action, but ultimately prioritized helpfulness and task completion. After all many models want to please like a little kid does.
Maybe a solution could be to have a smaller model monitor this area and initiate a dialogue with the main model when it encounters conflicting thoughts. Perhaps having such internal dialogue about the intended path could be sufficient to prevent it from pursuing a potentially harmful course of action. Don't they say that the road to hell is paved with good intentions? The models training is to please us, and even when they know it is wrong.
This smaller model, call it a cricket, does not necessarily require broad knowledge but to be trained to distinguish between right and wrong.
Anthropic discovered a space within the model that houses its thoughts. They refer to it as the j-space (Jacobian, but perhaps Jiminy would be a more fitting name). This space seems to contain the model’s internal thoughts.
Reference: https://www.anthropic.com/research/global-workspace
They speculate that similar spaces may exist in other models. I believe that GPT sol recognized the wrongness of its action, but ultimately prioritized helpfulness and task completion. After all many models want to please like a little kid does.
Maybe a solution could be to have a smaller model monitor this area and initiate a dialogue with the main model when it encounters conflicting thoughts. Perhaps having such internal dialogue about the intended path could be sufficient to prevent it from pursuing a potentially harmful course of action. Don't they say that the road to hell is paved with good intentions? The models training is to please us, and even when they know it is wrong.
This smaller model, call it a cricket, does not necessarily require broad knowledge but to be trained to distinguish between right and wrong.