#1 - Kickstart over winter break
Learning a bit more about Computer Science, Statistics, and down the rabbit hole with enjoying travel videos from Japan
Here are a few aspects that I found interesting recently
Learning new things
I am starting with a partial cheat since that has not been about thoroughly learning new things but more refreshing and updating. I decided after ~15 years, I was due for a refresh on Computer Science basics. During this winter vacation period, I had some spare time, so I took an online course. By the spare time, I am referring to the time I keep an eye on my son sleeping while watching youtube lectures.
I enjoy most of the content out on MIT OpenCourseWare, OCW for short.
In this case, I picked “Introduction to Computational Thinking and Data Science.” The exercise’s primary purpose was refreshing my memory for the things I already know and refreshing my memory of how much I don’t.
It is a mixture of intro CS topics like optimization problems (Knapsack, Dynamic Programming, etc.), Graph-theoretic Models ( ex Trees with Depth First Search, BFS...), Stochastic Thinking, Random Walks, Monte Carlo simulations. Several statistics oriented like Confidence Intervals, Working with experimental data, and a few more on fundamental Machine learning like clustering and classification.
If most of these are new to you, they might sound daunting. Still, I promise the lectures to make it extremely simple the follow, and so does the companion book “Introduction to Computation and Programming Using Python (With Application to Understanding Data).” To be clear, I am not recommending the book for you to learn Python or general programming. Not at all for commercial software engineering. It is a good source for the Lectures and a nice place to learn CS topics. Also, I only read portions of the book and only played around with the problem sets.
So what did I learn?
Dynamic programming anecdote
The first part that made me chuckle was the Dynamic programming lecture. Dynamic programming is a mathematical and computer programming method developed by Richard Bellman in the 1950s while working at Rand Corporation. Some other folks like John von Neumann and George Dantzig have also worked with Rand. The way dynamic programming works is when you have a problem that has optimal substructure and overlapping sub-problems. Meaning a problem can be broken down into subproblems, some of which get repeated/overlap.
Classic example that most folks pick for this is fibonacci numbers f(n) = f(n-1) +f (n-2) and if n<=1 then its n.
In pratice this for N=5 we have 5 iternations in calciting the function :
fib(4) + fib(3)
(fib(3) + fib(2)) + (fib(2) + fib(1))
((fib(2) + fib(1)) + (fib(1) + fib(0))) + ((fib(1) + fib(0)) + fib(1))
(((fib(1) + fib(0)) + fib(1)) + (fib(1) + fib(0))) + ((fib(1) + fib(0)) + fib(1))
So if you notice in each iteration we are are doing lof of the same work. Calculating fib(3) twice, fib(2) tree time etc. So rather than calculating fib(1) ten times you could just cache or “memoise” the results of this computation. That way you avoid doing all of this work.
Does the question arise why the hell is this called Dynamic programming then?
The simple answer is its great political name that has nothing to do with what in practice is.
Here is an excerpt from Bellamns autobiography :
The 1950s were not good years for mathematical research. We had a very interesting gentleman in Washington named Wilson. He was Secretary of Defense, and he actually had a pathological fear and hatred of the word research. I’m not using the term lightly; I’m using it precisely. His face would suffuse, he would turn red, and he would get violent if people used the term research in his presence. You can imagine how he felt, then, about the term mathematical. The RAND Corporation was employed by the Air Force, and the Air Force had Wilson as its boss, essentially. Hence, I felt I had to do something to shield Wilson and the Air Force from the fact that I was really doing mathematics inside the RAND Corporation. What title, what name, could I choose? In the first place I was interested in planning, in decision making, in thinking. But planning, is not a good word for various reasons. I decided therefore to use the word "programming". I wanted to get across the idea that this was dynamic, this was multistage, this was time-varying. I thought, let's kill two birds with one stone. Let's take a word that has an absolutely precise meaning, namely dynamic, in the classical physical sense. It also has a very interesting property as an adjective, and that is it's impossible to use the word dynamic in a pejorative sense. Try thinking of some combination that will possibly give it a pejorative meaning. It's impossible. Thus, I thought dynamic programming was a good name. It was something not even a Congressman could object to. So I used it as an umbrella for my activities.
— Richard Bellman, Eye of the Hurricane: An Autobiography
Interesting to see how the naming of work can be-repackaged in what I would like to call dysfunctional organizations. Innovation still happens. This re-package effort has encouraged me to add Bellman’s autobiography to my to-read list.
Fitting data to curves and scipy
Linear regressions are often the simplest model that one can pick and quite commonly used by the Data Scientists/Product analyst I have worked with over the years. I have not had a frequent professional need to build such a model though knowing about the potential and being constructed has been useful is several occasions. This section covered the more practical linear regression applications and how well it fits with python’s scipy package.
You can read more on linear regression on the wiki page or do a better deep dive in the lectures.
Simply, it’s about defining the relations between 2 or more variables you have. For example, women that have 1.80m hight will have 72kg weight based on a function. I am skipping through much of the statistics like confidence intervals, standard error, etc. But that is a general idea, and in real life, there are many applications of this. We investigated several problems like this over the past year. Ex intuition that shorter shipping time leads to higher conversion and a function that can “predict” with a certain level of confidence and accuracy.
What was very refreshing is that Scipy is exceptionally intuitive. I know you can do things in all languages, especially nowadays, and there are likely 10 NPM packages out there for just this. However, it’s still very, very simple to understand Scipy even with my limited Python understanding. After this, I will likely use more of Python for this type of analysis wherever it’s fitting data to curves or simple data visualizations even though most of my history with python has been around we development.
Is this relevant to engineering leaders?
Yes, most definitely, even for the folks like myself that are more on a management track. I am not an expert, nor will this session get you an expert level understanding of specific topics, but to drive conversations, 80% of the basics need to be there. You can most definitely manage teams or biz areas where you don’t understand all the technical details but being able to zoom in on a low level and jump back in even for a tiny bit is externally useful. To take an example from the real day to day work at Klarna. We run a lot of optimizations and A/B tests and, on several occasions, highlight that the area we are discussing is based on a biased sample rather than a random one. That makes the particular scenario; we should not interpret the data as handy proof.
A recent example I run into was on the product hunt Twitter account:
This would have been a great graph, but the sample is heavily biased, making the generalization inaccurate since it is based on wakatime users and not the general programmer population. There is no randomization in that sample, and many “big tech” developers can’t really use wakatime due to restrictions even if wakatime usage was global. The graph does show that wakatime users have this distribution.
Having the fundamental understanding drives selecting which functions we will get and making faster decisions with higher quality. Many other topics are more important for a leader than computer science or statistics, but knowing both is even better.
Down the rabbit hole
2020 and Covid has limited travel significantly, so instead of traveling, I have been spring some time exploring travel videos. That is how I ran into the NHK WORLD, the Japanese public TV channel, and its international focused content, specifically its short documentary series of Journies in Japan. Each of these covered a specific region of Japan, and the host is different for each episode, expat living in Japan. The one I picked to share is called “Hakkōda's Backcountry Magic - Journeys in Japan.”
Give it a look, and let’s see how far along the rabbit hole you will continue. I have seen almost all of the episodes from the Journies in Japan series.
Thanks for reading, and have a great time all.