Author: mattchung

I just discovered dynamic programming and damn, I’m blown away by the concept.  The other day, hile working through a homework assignment, I compared the run times between two python functions that I wrote, one function written recursively and the other written in a dynamic programming fashion.  Starting with the recursive solution, I arrived at the following:

def fibonacci(n):
    if n == 0:
        return 0
    if n == 1:
        return 1
    return fibonacci(n-1) + fibonacci(n-2)

That’s a fairly standard implementation of Fibonacci. There are two base cases; n=0; n=1.  So when n is either of these two numbers, the function simply returns 0 or 1, respectively.  But for any other number, the function recursively calls itself until reaching the aforementioned base cases.

So far so good, right?  And for calculating small values of n, this implementation doesn’t really present a problem. But say we want to calculate fibonacci when n equals 40. How long does this take? Alarmingly, this computation hovers around 45 seconds:

./fibonacci.py 40
fibonacci(40) = 102334155 took 45.22154 seconds

Now, what if we run the same calculation. But this time, we run it using a dynamic programming technique? How much time does that shave off?

./fibonacci.py  --dynamic-programming 40
fibonacci(40) = 102334155 took 0.00002 seconds

What ?! From 45 seconds down to under a millisecond ?! How is that possible?

def fibonacci_dynamic_programming(n):
    fibonacci = [0,1]
    for _ in range(0, n):
        fibonacci.append(fibonacci[-1] + fibonacci[-2])
    return fibonacci[n-1] + fibonacci[n-2]

As you can see from the code above, instead of recurisvely calling fibonacci, we iteratively calculate all the values. In other words, this implementation runs linearly (i.e. direct proportion to n), unlike the first, recursive implementation, which runs exponentially.

Last Friday, I took the final exam for my (distant learning) discrete mathematics course and just now I had logged into the student portal (i.e. Blackboard), surprised to find that my exam had not only been graded but my final grade had been posted as well. I finished the course with an 88%, a B, a few points short of an A.  In the past, I would’ve incessantly beat myself up over not achieving the highest grade, denigrating myself with self destructive thoughts: if only I tried harder … if only I studied more … if only I was smarter …

 But not this time.

This time, I’m inhibiting those thoughts. Instead, I’m celebrating. Celebrating that I’m fortunate enough to be able to take this mathematics course, a course where I learned about: writing proofs, solving Diophantine equations, applying set theory, counting with modular arithmetic, proving assertions with mathematic induction, converting recursive functions into closed form functions using characteristic equations method. Prior to the course, I was never exposed to those concepts.  Looking back, I only vaguely heard of those terms.  And who knows if I get to apply any of those concepts as I pursue a master’s – maybe a PhD, one day. Who knows if I’m lucky enough to apply that knowledge to my job as a software engineer.

But who cares?  Because really, my goal was to stretch myself, learning more about my field and craft: computer science.

Although I sometimes find getting paged for operational issues enervating, I wouldn’t have it any other way.

It’s well known that software engineers at Amazon (Web Services) own their systems, end to end. This means that we not only develop and maintain our software, but we operate the underlying system, complete ownership.

From a software point of view, one can run into an infinite number of issues. Got a build error? Roll up them sleeves and start digging through the compilation errors.  You committed some code that broke the build? Fix or rollback the changes. You defined a dependency that’s deprecated? Update the dependency and ensure your unit and integration tests pass.

Similarly, we maintain the underlying system. If we deploy our systems to a servers, physical or virtual (e.g. EC2), we must keep it alive, like a breathing entity. From checking disk performance, to checking heap allocation, we monitor our systems closely, configuring monitors to alarm us when any component misbehaves.

In other words, there’s no divide between development and operations. There’s no separate team that handles operational issues, no DevOps. There’s no nonsense like writing our software and then chucking it over to another group to deploy it, a group that would otherwise find it annoying at best and frustrating at worst.  Because I’ve been in those positions, where I’m on the hook for deploying software that I cannot fix. Similarly, I’ve been in situations where the deployment fails and then I must ring in someone who’s more familiar with the code base.

But now, I’m in a position where I’m responsible for the code I write.

Why is this important?

Although some would argue that software developers should stick to software development, and that there should be a clear separation of duty, I believe that owning a system end to end promotes operational excellence and a sense of ownership.

Last week, I was sitting behind my desk at work, surfing hacker news and and at the top of the website floated an article by the co founder of “Daily Coding Program”, a small tech start up that e-mails daily newsletters with a programming question.  The article shared some of their insights over the past year and how they bootstrapped the company and eventually grew to $2,000 in monthly revenue.

Although I was interested in reading the blog post, I ended skipping over most of it and navigated to their front page, where I scrolled up and down and read more about the company’s product.  When I reached the end of the front page, I was confronted with a sample interview question:

There’s a staircase with N steps, and you can climb 1 or 2 steps at a time. Given N, write a function that returns the number of unique ways you can climb the staircase. The order of the steps matters.

This question, I had initially thought, could be solved using some techniques that I recently learned from taking my discrete mathematics course. But to confirm whether or not I could apply permutations and combinations, I hollered at my colleague, repeating the question out loud and asking him how this problem could be solved. And as I started reading the question out loud, “There’s a staircase with N steps, and you can climb …” he interrupted me, finishing my sentence with, “1 or 2 steps at a time.”

Apparently, while preparing for interviews with Google and Microsoft, he had stumbled across this same practice problem.

So after letting him interrupt me, I asked him if this particular problem could be solved with permutations or combinations. In other words, do these two mathematical concepts apply to the problem. He confidently answered, “No — dynamic programming.” He then proceeded to step me through his solution, scribbling down his work on a white piece of 8.5 x 11 that sat on my desk.

Fast forward to now.

I’m at home, starting the next lesson for my discrete mathematics course, the lesson title: “Counting using Recurrence Relations” and “Solutions to Recurrence Relations.” I’m skimming the chapter, building a mental model of what the entire chapters entail, an overview.  And when I reached the end of the first chapter, where exercise problems live, I couldn’t help but form a little grin.

To my surprise, the following chapters cover topics relating to the interview question that I had just read. In fact, the exercise in the chapter is phrased almost identically:

Sal climbs stairs by taking either one, two, or three steps at a time. Determine a recursive formula for the number of different ways Sal can climb a flight of n steps.

So, I find it always nice and serendipitous when what you are learning, either in school or on your own, can be applied to real life examples. Albeit, this application was only for an interview question.

But I’ll still consider that as a victory.