https://ift.tt/32qAJJl “Cogito, Ergo Sum” — Modeling Data and Decisions with offbeat thought frameworks How to make sense of data (or lack...
“Cogito, Ergo Sum” — Modeling Data and Decisions with offbeat thought frameworks
How to make sense of data (or lack of it) — when it behaves, and misbehaves
“Cogito, ergo sum”
“I think, therefore I am”
- René Descartes
Thinking or solving anything requires some kind of framework that we either consciously or subconsciously follow. They have entire fields of analysis devoted to them, either to understand and structure how one thinks/decides (read: Decision Analysis) or to understand how someone else thinks in a certain situation, say, retail shopping (read: Predictive Analytics).
Having been in an academic setting, now with a PhD program, I have collected some lessons along the way- lessons that help one think or decide efficiently when internalized. I have imbibed these so much that to call them a framework of any kind would be an overstretch, but these are simple, elementary concepts that I believe that every person, especially someone dealing with data, should read and introspect every once in a while. Over time I realized that the more I thought about these, the more natural they became, and the more correct they seemed.
Picasso’s Bull
At some point you might have come across a video or chart showing the minimalistic portrait of a bull that Pablo Picasso made. Here it is, for reference.
A lot of interpretations of it exist in the wild, but my personal favorite is the one which denotes simplicity. With each subsequent drawing, a little of the bull is taken away, but what remains is still something that is recognizable as a bull and an elegant looking one at that. Steve Jobs has mused about finding simple yet elegant solutions to problems, and to quote Picasso himself,
“…Whatever is most abstract may perhaps be the summit of reality”
Every time I work on a problem, almost all of which are data driven, after I write my code, infer my results, write my manuscripts, I spend a good amount of time to “Picasso” it. I remove any line, any inference, any mention of a concept, or any reference that an uninitiated reader doesn’t need to know of, and I keep just the barebones, that help a reader get from zero to one (to steal from Theil). This holds true for words, code, figures, charts, and almost anything else that one can make.
This idea of “Picasso”-ing something doesn’t just apply to presenting solutions or producing work. It represents a school of thinking where approaching a problem in a way where we retain its fulcrum, but negate anything that is fluff is paramount. Disregarding the distraction and focusing on the useful is an art.
Feynman’s Technique
It is perhaps true that great minds think alike, because Richard Feynman believed in the lucidity of simplicity too. An abstract explanation of this technique, often found on the internet, can be summarized as such:
- Learn something.
- Teach it to a kid.
- If the kid understands it, your understanding of it is robust. If not, repeat.
While I use Picasso’s technique to think about a problem, or present results, when I have to learn, I try to “Feynman” it. It wouldn’t be a stretch to say that this technique is a “learning” version of Picasso’s school of thought.
When it comes to learning anything new, be it something as simple as the Bayes Theorem, or as convoluted as understanding how neural networks or transformers work, this technique is something that one can swear by.
Countless pieces of literature on the internet swear by this technique to teach and learn and I will leave it up to them to convince you of this. This medium post is a great read to get started.
The Good Gut
More often than not, I have learnt that the so called “gut feeling” or intuition is a very good heuristic when it comes to thinking or solving. It often guides you to a great place to start when solving a problem or doing a task, is a fantastic indicator of whether or not a task is within your current knowledge and is splendid when it hints at something going wrong. It is so good that even Spiderman trusts it (Read: Spidey Sense).
I neither know, nor want to dive into the science behind the science of our “spidey sense”, but I do know that for me and almost all of the people around me, these experiences are common:
- “This math problem seems to have a very simple solution. Seems to suspicious.”
- “I have read through the problem statement and know that it will take me an all-nighter to solve it”
- “I feel like a simple linear regression will solve this problem just well enough”
Call it experience, call it wisdom. If it's the first thing that you feel, and not just rationally conclude, then you can know that your gut has given its verdict. So very often, I have not believed that initial feeling, fought against it and have spent countless hours pursuing other solutions until I repent and run back. That being said, much like any piece of knowledge in this universe, gut feelings alone will not help you wade through thick waters - one needs to couple it with a good dose of rationalization.
To quote Moffat and Gattiss’ Sherlock,
“Intuitions are not to be ignored. They represent data processed too fast for the conscious mind to comprehend”
Black Swan Paradox
“Eliminate all other factors, and the one which remains must be the truth.”
Sherlock also said this. But often, even with mountains of data or information, some things just don’t make sense. It could be that the data is just not suited to the problem. It could be that we don’t know how to use the data to solve the problem- maybe we are modeling it wrong.
In such cases, it is often helpful to look for the black swan. Let me explain.
In simple words, if you want to learn about swans (for some reason) and you observe them. If all the swans you have seen are white, it wouldn’t be a surprising descriptor of the bird when you write about it or report your results. It would, in fact, be a defining characteristic. If you see a black colored bird floating on the water, you would reject it as a swan without a second look. But what if, one day, you observed a black swan? Your understanding of swans has now taken a hit. You start to think if you even know them anymore. Now, what if this black swan behaves completely differently? What if they hiss instead of trumpeting? With hyperbole, what if it is a cannibal and wipes out other swans?
Such a situation is called an example of a “Black Swan” situation where one sudden, unexpected, and rare event throws your knowledge and understanding, and possibly the world you are dealing with out of whack. Black swans are common occurrences too. Without too much of a stretch, if you observe a bunch of people around the same time with the same symptoms of a disease which seems to be spreading rapidly, you could argue that they form the beginning of a pandemic. Demonetization of currency is another example- an event so rare that you are seldom expecting/preparing for it, and yet it hits- with often severe consequences.
But why do you need to know this? You don’t actually. It is not common to be able to predict a black swan, and it is not often necessary. But knowing that such a thing as a black swan event can exist, being able to identify them when they are happening, and being able to potentially enumerate them beforehand, can help form contingencies or mitigate them faster if and when they do strike. I am sure it will be helpful if you are managing a restaurant which now has to brace with delivery orders instead of in-house ones because of a pandemic.
When data doesn’t make sense, but you see a pattern, look for a Black Swan.
Life involves active thinking almost all the time. Be it managing finances, or building neural networks to solve probabilistic problems. There is no one bonafide way to think and solve these and there are formal fields that help guide you to an effective decision based on deep rational thought and mathematical logic. But how often do you really draw out a decision tree when house hunting or when figuring out why the price of your groceries are going up in one neighborhood but not in another? How often do you try to use a quantifiable way to understand if you solved an assignment the correct way or if you presented a piece of information well enough?
These are decisions that we make every day, quickly, with minimal interference from frameworks and tools and often with just a tiny bit of rational thought. It is that tiny bit that I am trying to shape to make better decisions with these ideas that I “incepted”.
Adithya Narayanan is a PhD Student at the University at Buffalo. He specializes in applying logic to probabilistic graphs and has a background in Computer Science. Reading news and commenting on it are his natural instincts.
He has an alter ego, who, when bored, loves creating digital content of all kinds but primarily prefers a pen and paper. He has created digital content for several larger than life sporting entities such as LaLiga Santander, Roland-Garros, and NBA Basketball School.
“Cogito, Ergo Sum” — Modeling Data and Decisions with offbeat thought frameworks was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
from Towards Data Science - Medium https://ift.tt/3DIsk1A
via RiYo Analytics
ليست هناك تعليقات