Restacked, excellent piece! I've had in mind the metaphor of "firmware" for the lower brain and now you have me wondering which of that and BIOS minimizes muddle (all metaphors are wrong! but some are useful). Besides NeuroAI, I believe medicine itself (including specialties beyond psych/neuro/neurosurgery) may benefit greatly from decompiling the lower brain. Let's keep pushing on this from all angles.
Excellent and insightful read as always. Curious, to your mind, to what extent can we think of an AI's default system prompts/system instructions as a sort of analogue to this basal "steering system" in biological brains? Seems like we already do conceive of it this way in LLMs, with the weights of the model itself being the thought forms of the cortex (generalizable, emergent, acquired through extensive RL), and the default instructions being what the model is "born" with? I suppose the analogy breaks down when you consider that the system prompts instruct a lot of higher-level concepts like model identity or tonality.
Really not an expert here, but isn't greater transcriptomic diversity for evolutionarily older structures just what you would expect under a null model (more time to diversify)? So, the relationship in your Fig. 1 here (assuming it's real and not due to an experimental bias) might just be driven by how ancient those brain regions are (as opposed what they're doing)? The purported relationship between reward and cell types also seems somewhat tenuous: even in your tOFFα example, that specific cell type seems to be encoding a visual stimulus, not the value per se.
That BiOS is essentially what we refer to in terms such as “evolutionary biases” and “affordances” - the prior constraints that shape the perceptual, cognitive, and behavioral possibilities of an organism. And that is the basis of my argument that AI will never really align with human intelligence at a deep level: It will never have the same BiOS. The best we can hope for is superficial correspondence, i.e., emulation. That is sufficient in many cases, but will lead to unpredictable divergences that could be very dangerous.
Very insightful post, thank you. An interesting research direction would also be to deepen our understanding of how the reward circuits of the Steering Subsystem map, at a cellular level, to the value-related regions of the Learning Subsystem (for example, in humans, the ventromedial prefrontal cortex for the valuation of current actions or strategies, and the frontopolar cortex for the valuation of alternative options). Somehow these cortical regions need to translate complex world models into representations that are more directly usable by the basic reward circuitry of the brain. But that seems to imply that at some point during this transition, there must be a sort of classification function that converts any abstract value representation into one of the primitives of the reward circuits.
Restacked, excellent piece! I've had in mind the metaphor of "firmware" for the lower brain and now you have me wondering which of that and BIOS minimizes muddle (all metaphors are wrong! but some are useful). Besides NeuroAI, I believe medicine itself (including specialties beyond psych/neuro/neurosurgery) may benefit greatly from decompiling the lower brain. Let's keep pushing on this from all angles.
Excellent and insightful read as always. Curious, to your mind, to what extent can we think of an AI's default system prompts/system instructions as a sort of analogue to this basal "steering system" in biological brains? Seems like we already do conceive of it this way in LLMs, with the weights of the model itself being the thought forms of the cortex (generalizable, emergent, acquired through extensive RL), and the default instructions being what the model is "born" with? I suppose the analogy breaks down when you consider that the system prompts instruct a lot of higher-level concepts like model identity or tonality.
Really not an expert here, but isn't greater transcriptomic diversity for evolutionarily older structures just what you would expect under a null model (more time to diversify)? So, the relationship in your Fig. 1 here (assuming it's real and not due to an experimental bias) might just be driven by how ancient those brain regions are (as opposed what they're doing)? The purported relationship between reward and cell types also seems somewhat tenuous: even in your tOFFα example, that specific cell type seems to be encoding a visual stimulus, not the value per se.
This is such a great article! Very insightful.
That BiOS is essentially what we refer to in terms such as “evolutionary biases” and “affordances” - the prior constraints that shape the perceptual, cognitive, and behavioral possibilities of an organism. And that is the basis of my argument that AI will never really align with human intelligence at a deep level: It will never have the same BiOS. The best we can hope for is superficial correspondence, i.e., emulation. That is sufficient in many cases, but will lead to unpredictable divergences that could be very dangerous.
Very insightful post, thank you. An interesting research direction would also be to deepen our understanding of how the reward circuits of the Steering Subsystem map, at a cellular level, to the value-related regions of the Learning Subsystem (for example, in humans, the ventromedial prefrontal cortex for the valuation of current actions or strategies, and the frontopolar cortex for the valuation of alternative options). Somehow these cortical regions need to translate complex world models into representations that are more directly usable by the basic reward circuitry of the brain. But that seems to imply that at some point during this transition, there must be a sort of classification function that converts any abstract value representation into one of the primitives of the reward circuits.