https://ift.tt/3dwCHtI Recurrent Neural Networks, Geemo, Scooby-Doo, Design Thinking, and Diversity Illustrate Geemo the Bayesian ponders...
Recurrent Neural Networks, Geemo, Scooby-Doo, Design Thinking, and Diversity Illustrate
When considered together, two recent articles by Garner and Tong (2020) and Ananthaswamy (2021) lead one to think we are all Bayesians.
The heart of Bayes’ Theorem is that knowledge gained from the past (i.e., the prior) is the forerunner of future predictions. Our Bayesian minds are obviously imperfect though. Ask anyone trying to predict stock prices, or whether our next sniffle is really that or the harbinger of yet another case of COVID-19.
We are often wrong at predicting the future. We may incorrectly assess the causes of our maladies based upon the symptoms we experience. We do learn though, and we usually get better as we weigh more evidence. That may reflect Bayes’ Theorem in action.
Ananthaswamy describes how recurrent neural networks (RNNs) mimic brain function. The use of prior information in this prediction process suggests Bayesian roots to the RNN, and this is the subject of my post today.
The Brain as a “Prediction Machine”
Ananthaswamy (2021) says our brains are “prediction machines,” gobbling up input from our five senses to assess why things happen. We then update these predictions as we receive new information. “Through predictive processing,” he states, “the brain uses its prior knowledge of the world to make inferences or generate hypotheses about the causes of incoming sensory information. Those hypotheses — and not the sensory inputs themselves — give rise to perceptions in our minds eye.” The brain, it seems, is a machine learning device!
This means that the brain doesn’t just stock the mind’s shelf with what we see, hear, smell, taste, or touch and then immediately identify an underlying cause. There is an intermediate machine learning step between sensing something and interpreting it, and that step operates like a Bayesian RNN. Computational neuroscientists are testing this intermediate-step hypothesis, and growing evidence points to an RNN that mimics the biological process animals use to make sense of their environments.
An RNN is a deep learning process involving multiple layers of neurons. If the neuroscientists’ line of reasoning is correct, then the mathematical neurons created in their deep learning models resemble the real-life nerve cells animals use to make sense of things. Since humans are animals too, our thinking processes may be generating predictions the same way RNNs do. Errors in those predictions can be either perpetuated or corrected once more sensory input is collected and analyzed; this is where the Bayesian notion comes into play.
To see how this may operate, consider the “Perceptions Based on Predictions” image in Anathaswamy’s paper. This figure shows two interacting strands of neurons, one which reflects sensory input and one that reflects the predictions made from that input by RNN models. Once a prediction is made in one strand, it is sent back to the input layer in the other strand so it can be incorporated as a Bayesian prior, along with new information, to generate an updated prediction.
Mathematical Support and Logic Flow
Accepting the notion that perceptions are Bayesian also requires accepting that they are probabilistic. The math supporting probabilistic Bayesian RNNs is provided in an article by Garner and Tong (2020). Their derivation of Bayesian RNNs can be adopted to explain how animal and perhaps human RNNs continually update predictions that generate perceptions as new information arrives from our senses. This leaves us with the following summary, albeit still hypothetical, logic flow:
· Perceptions based on input from multiple senses are generated from biological versions of recursive neural networks.
· Evidence reviewed by Anathaswamy provides support for this hypothesis by showing how RNNs work in animal brains.
· In an unrelated paper, Garner and Tong provide the math showing that recursive RNNs have Bayesian roots.
· Jointly considered, their work can be construed as providing mathematical and biological mechanisms for how animals interpret their environments. Whether this mechanism is truly causal is still being debated.
· A final point in the logic flow is admittedly another hypothetical stretch — my view is that the Bayesian RNN works in humans too. More studies are needed to assess the validity of this hypothesis, but it seems compelling to me.
Some Analogies and Other Evidence of Bayesian RNNS in Action
Have you ever seen the classic Scooby-Doo cartoons that ran from 1969 until the mid-seventies? Newer versions are around today on YouTube. My friends and I watched the originals as part of our Saturday morning routines when we were kids. Each cartoon showed how Freddy, Daphne, Velma, Shaggy, and their canine companion Scooby would work together, assimilating and weighing more and more evidence, to solve one mystery after another. (Spoiler alert, the mysterious evil being they encountered was never a real ghost!) Check it out yourself to see how they continually updated their hypotheses to create more realistic and accurate perceptions of nefarious circumstances. Cartoon Bayesians indeed!
Two additional analogies come from illustrations of how humans can effectively work together in real life to understand their environments and generate better insights. One is known as Design Thinking, and the other is illustrated by a more diverse workforce and its downstream modeling approaches.
A nice description of Design Thinking is provided by Friis Dam and Siang (2020). Quoting their website:
Design Thinking is an iterative process in which we seek to understand the user, challenge assumptions, and redefine problems in an attempt to identify alternative strategies and solutions that might not be instantly apparent with our initial level of understanding. At the same time, Design Thinking provides a solution-based approach to solving problems. It is a way of thinking and working as well as a collection of hands-on methods.
They also provide a list of steps summarizing a Design Thinking approach. Quoting again, those steps include efforts to:
Empathize — with your users
Define — your users’ needs, their problem, and your insights
Ideate — by challenging assumptions and creating ideas for innovative solutions
Prototype — to start creating solutions
Test — solutions
Importantly, the authors note the iterative (one might say the Bayesian RNN) nature of this process. To arrive at the best solution to a client’s problem, participants are encouraged to challenge each other and move back and forth between the different phases. This will help cement their understanding of needs, challenges that must be surmounted to meet those needs, potential solutions, and the best ways to reach those solutions. The iterative Design Thinking process will also help them forecast the likely outcomes of the proposed solutions.
The notion of workforce diversity is similar. In the context of machine learning or other artificial intelligence modeling efforts, diversity means having a project design and analytic team whose members reflect the environments and cultural characteristics of their research subjects. It also means using data that adequately represent those subjects to predict outcomes of interest. Diversity also means testing models to see whether data generating processes or outcomes differ substantially according to age, gender expression, race, ethnicity, income, or other context-dependent / important factors. Sensitivity analyses like these are evidence of a Bayesian spirit and are key to making accurate inferences.
Ensuring diversity is hard. Our modeling teams may not adequately represent subjects of interest. Expert or interested stakeholder advice may not be available. Our data may be lacking, and our budgets and research environments are not always conducive to using optimal approaches. Working toward diversity will ensure that a variety of designs, methods, and interpretations will be considered, though. This will help us move iteratively through the analytic process in a way that can lead to better solutions and avoid bias in our models (Obereyer et al., 2020; Hall et al., 2021).
Limitations and Final Thoughts
If our brains are mathematical and probabilistic Bayesian machines, why aren’t we all perfect at interpreting our senses and generating accurate perceptions of reality? Isn’t math immutable? Why might my perceptions differ from yours, when only one can be correct? The answer is that our brain math is fine, but its sensory input is influenced by our unique historical experiences and by the environments we are in. These affect our priors, which in turn affect our perceptions. If we have interpreted history incorrectly, or if our environments are improperly influenced by other inaccuracies, our perceptions can be wrong.
The historical- and context-dependent nature of our Bayesian mind machines is also one reason why our predictions and perceptions have some inherent uncertainty around them. The sizes of our confidence intervals vary, so I might correctly or incorrectly interpret something with more conviction or assuredness than you do. Hopefully our certainty and accuracy get better as more information arrives and our mind machines continue to weigh evidence, but there is no guarantee this will lead to perfect understanding. If we always got better over time at describing reality, we would not be so burdened with ridiculous conspiracy theories. We would be better able to sort out truth from fiction and controversy would eventually subside.
A related point has to do with memory function; it can be viewed in a Bayesian sense too. Memory is a device to retain our priors, which then influence posterior probability generation. But memory accuracy varies from person to person and our memories degrade at different rates over time (Yassa, et al, 2011). Memory is also context dependent. Nevertheless, Anathaswamy (2021) notes that memory is key to efficient RNN functioning and is probably the reason why RNNs are useful at all. If we want to improve our mind machines, working toward improving our memories would be a great place to start.
Taking a science-based approach to sensory interpretation would improve the accuracy of our perceptions as well, but that is a story for another day.
As a final thought, the evidence in support of a Bayesian RNN perception production process is still circumstantial, but it does seem as if Bayes’ Theorem can be found in action almost any place we look. Thus it may be useful to incorporate at least one Bayesian approach in all our data science projects, if our goal is to generate increasingly realistic predictions and accurate perceptions of how our world works.
References:
A. Anathaswamy, Your Brain is an Energy-Efficient Prediction Machine (2021), on https://www.wired.com/story/your-brain-is-an-energy-efficient-prediction-machine/?bxid=5bea08b63f92a4046944073e&cndid=37560534&esrc=AUTO_OTHER&hashc=cc9cb28c4dbd883035113d8b812fa1ecdc6e97567c2ec2e0b3497dcc0e716535&mbid=mbid%3DCRMWIR012019%0A%0A&source=EDT_WIR_NEWSLETTER_0_DAILY_ZZ&utm_brand=wired&utm_campaign=aud-dev&utm_content=WIR_Daily_113021&utm_mailing=WIR_Daily_113021&utm_medium=email&utm_source=nl&utm_term=P5
R. Friss Dan and T. Y. Siang, What is Design Thinking and Why Is It So Popular? (2020), on https://www.interaction-design.org/literature/article/what-is-design-thinking-and-why-is-it-so-popular
P. N. Garner and S. Tong, A Bayesian Approach to Recurrence in Neural Networks (2020), on arXiv:1910.11247v3 [cs.LG] 20 Apr 2020.
P. Hall, N. Gill, and B. Cox, Responsible Machine Learning (2021), Boston, MA: O’Reilly Media, Inc.
Z. Obermeyer, R. Nissan, M. Stern, et al., Algorithmic Bias Playbook (2020), Chicago Booth: The Center for Applied Artificial Intelligence. Also on https://www.ftc.gov/system/files/documents/public_events/1582978/algorithmic-bias-playbook.pdf
M. A. Yassa, A. T. Mattfeld, S. M. Stark, and C. E. L. Stark, Age-related memory deficits linked to circuit-specific disruptions in the hippocampus (2011), PNAS 108(21):8873–8878, also onhttps://doi.org/10.1073/pnas.1101567108
Are we all Bayesian? Our Brains Think So was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
from Towards Data Science - Medium https://ift.tt/3yl7TWN
via RiYo Analytics
No comments