Learning of Highest Order
Q) Why Algorithmic leaps can be better than Hardware leaps?
Ans) Hardware constraints create bottlenecks that are hard to tackle as uncertainty of physics at small scale (nano-meters and less) come into play (electrons start jumping around).
At this point, ideas (algorithms) can be used to unleash full potential of the feasible hardware.
Q) Why these improvements are so beneficial for Machine Learning?
Ans) Machine Learning (ML) programs can easily involve:
- Billions of computations
- Similar equations with hundreds to millions degree of exponents
- Most computations are independent of each other (can be parallelised or distributed)
Hence even a small optimisation can have very visible effects, saving you from a few hours to months!
Q) Why refer to the brain for optimisations?
Ans) Short answer, without the below mentioned evolution in human brain, we might have looked like:
The Memory-Prediction Framework
Q) What is Neocortex?
Ans) Neocortex is the largest part of the cerebral cortex which is the outer layer of the cerebrum.
Q) Why is neocortex important?
Ans) Neocortex, also called the isocortex, is the part of the mammalian brain involved in higher-order brain functions such as sensory perception, cognition, generation of motor commands, spatial reasoning and language (partially).
Extra : The neocortex is further subdivided into the true isocortex and the proisocortex. More about working of Neocortex can be found here, here and here (curiosity might just make you open the links).
Q) How is neocortex relevant to machine learning?
Ans) Hierarchical temporal memory (HTM) is based on interaction of pyramidal neurons in the neocortex of the human brain.
At the core of HTM:
- Learning algorithms that can store, learn, infer and recall high-order sequences
- Learn time-based patterns in unlabeled data on a continuous basis.
- Robust to noise and high capacity, meaning that it can learn multiple patterns simultaneously
When applied to computers, HTM is well suited for prediction, anomaly detection, classification and ultimately sensorimotor applications.
The HTM has a hierarchical topology on account of hierarchies observed in some naturally occurring neural networks, such as those observed in the brain.
Human Memory hierarchy can be really really complex!
Example:
Or in Simplified form:
As one moves up the hierarchy, representations have increased:
- Extent: for example, larger areas of the visual field, or more extensive tactile regions.
- Temporal stability: lower-level entities change quickly, whereas, higher-level percepts (mental-concepts) tend to be more stable.
- Abstraction: through the process of successive extraction of invariant features, increasingly abstract entities are recognised.
- Less resource intensive: require much less resources especially memory.
Human brain’s algorithmic efficiency is almost petrifyingly high!
Philosphy’s Treasure, The Introspection
Q) What is Introspection?
Asn) Introspection or metacognition, is self-awareness about one’s thinking or ‘ability to think about the thinking’. A high-level mental process (very high).
Accurate introspection ables discrimination between correct decisions and incorrect ones.
Basically, a learner developing a conscious. Being self-able to reject or encourage patterns during training. Hence a self-optimising learner, the very necessity of General Learning.
Example:
Bistable Stimuli (two pictures in binocular vision, one from each eye) are one of the most popular approaches to studying the neural mechanism of conscious visual perception.
Such stimuli contain conflicting information, which the visual system cannot integrate into a unified percept.
This causes the perceptual (ability to interpret) state of the observer to change every few seconds trying to figure out correct interpretation. All this while the physical stimulus remains the same.
In this case the frontal lobe of cerebrum makes a self aware (introspective) decision to reject either information or modify each information to eliminate conflict.
Details of this can be found in this paper by Natalia Zaretskaya and Marine Narinyan
Q) What is gray matter?
Ans) The interior of the central nervous system is organised into gray and white matter.
Gray matter consists of nerve cells embedded in neuroglia (nervous tissue made up of large number of nerve cells); it has a gray color.
Q) Why is gray matter important?
Ans) In humans the gray-matter volume might help clarify the extent to which a person’s confidence about his introspective abilities is supported. Also how accurate his introspective abilities are.
More gray matter usually signifies more intelligence.
As Einstein once said “Gray Matter bitches!” ….. no, he actually didn’t.
Q) Then why don’t we directly implement the working of gray matter?
Ans) Simply because the working of gray matter is very very complex. We do not know the exact correlation between ‘confidence in’ and ‘accuracy of’ one’s introspective abilities.
Q) So what do we know?
Ans) Introspection is still (after millions years of evolution) a rare abilities even for complex organisms with most organisms lacking this ability completely. This has been one of the most recent development in evolution.
Q) Which part of the brain is responsible for introspection?
Ans) Prefrontal cortex in cerebrum.
Prefrontal cortex (PFC) is the cerebral cortex which covers the front part of the frontal lobe.
Two parts of this area are of great interest:
- Anterior prefrontal cortex
- Medial prefrontal cortex
Q) What’s special about Anterior prefrontal cortex?
Ans) Anterior prefrontal cortex has been associated with top-level processing abilities that are thought to set humans apart from other animals.
Most of the gray-matter which plays the role in introspection exists in this region (right anterior prefrontal cortex, to be precise). The structure of neighbouring white matter plays some (unclear) role in introspective capabilities too.
This brain region has been implicated in planning complex cognitive behaviour, personality expression, decision making, and moderating social behaviour. The basic activity of this brain region is considered to be orchestration of thoughts and actions in accordance with internal goals.
Q) What does that mean in terms of machine learning?
Ans) I’ll answer this in short point
- Learner that is aware of it’s decision
- Leaner that adapts to it’s environment
- Learner that can set it’s goals on the go
- Learner that can Generalise massive information in just a few patterns
Q) And what’s special about Medial prefrontal cortex?
Ans) Medial prefrontal cortex (mPFC) is considered to be a part of the brain’s reward system.
The mPFC is part of the mesocorticolimbic dopaminergic system (basically it generates dopamine, the ‘happy enzyme’).
Evidence for the involvement of the mPFC in reward-related mechanisms comes mainly from three types of studies:
- Conditioned Place Preference (CPP) : measure the motivational effects of objects or experiences
- Intracranial Self-Stimulation (ICSS) : what conditions trigger brain reward system
- Self-Administration : which external drug (like insulin for diabetes) the brain wants
What’s interesting about this reward system is:
- Different subareas of the mPFC appear to be differently involved in the ‘rewarding actions’ of different drugs. This indicates existence of multiple reward systems.
- Some drugs can produce reward directly within the mPFC, and that some drugs, even though not having direct rewarding effects within the mPFC, depend on the function of the mPFC for the mediation of their rewarding effects. Which can be represented as multi-tier reward process.
Q) What does that mean in terms of machine learning?
Ans) Even shorter answer:
- WHOLE OF REINFORCEMENT LEARNING
Q) What’s reinforcement learning?
Ans) A type of machine learning that involves:
- No Supervision (trial and error on bases of rewards)
- Good or Bad is decided a few steps later, not instantaneously
- Dynamic system so time really matters, sequence of steps really matters
- Agent gets to take action and alter/influence it’s environment
- For every action the Agent gets a reward and it’s job is to maximise it
It works something like this:
Q) Confused?
Ans) _‘_Reward’ is what enables to prioritise one decision over other.
It is a scalar feedback, quantifying how well agent is adapting/interacting with environment.
Like you give treat to your dog if he sits first else no treat (why u no gud boi?!).
The reinforcement learner simply follows the steps which maximise the cumulative reward.
Multiple reward systems essentially mean that brain has inbuilt multi actor-critic reinforcement learning mechanism.
In simple words, actor-critic learning involves an actor who takes an action and critic finds pros and cons (criticises) in that action.
Q) Role of reward system sounds very similar to introspection??
Ans) Well that’s because reward system does play a major part in one’s introspective abilities. This relation, though, is REALLY COMPLEX.
“Reinforcemennt learning is simply Science of Decision Making. That’s what makes it so General.” — David Silver
So Prefrontal Lobe is frontier in field of General Intelligence!