Author: mattchung

  • What I learned from writing synchronization barriers

    What I learned from writing synchronization barriers

    Before starting project 2 (for my advanced operating systems course), I took a snapshot of my understanding of synchronization barriers. In retrospect, I’m glad I took 10 minutes out of my day to jot down what I did (and did not) know because now, I get a clearer pictur eof what I learned. Overall, I feel the project was worthwhile and I gained not only some theoretical knowledge of computer science but I was also able to flex my C development skills, writing about 500 lines of code.

    Discovered a subtle race condition with lecture’s pseudo code

    Just by looking at the diagram below, it’s not obvious that there’s a subtle race condition hidden. I only was able to identify it after whipping up some code (below) and analyzing the concurrent flows. I elaborate a little more on the race condition — which results in a deadlock — in this blog post.

    Centralized Barrier

     

    [code lang=”C”]

    /*
    * Race condition possible here. Say 2 threads enter, thread A and
    * thread B. Thread A scheduled first and is about to enter the while
    * (count > 0) loop. But just before then, thread B enters (count == 0)
    * and sets count = 2. At which point, we have a deadlock, thread A
    * cannot break free out of the barrier
    *
    */

    if (count == 0) {
    count = NUM_THREADS;
    } else {
    while (count > 0) {
    printf("Spinning …. count = %d\n", count);
    }
    while (count != NUM_THREADS){
    printf("Spinning on count\n");
    }
    }

    [/code]

    Data Structures and Algorithms

    How to represent a tree based algorithm using multi-dimensional arrays in C

    For both the dissemination and tournament barrier, I had to build multi-dimensional arrays in C. I initially had a difficult time envisioning the data structure described in the research papers, asking myself questions such as “what do I use to index into the first index?”. Initially, my intuition thought that for the tournament barrier, I’d index into the first array using the round ID  but in fact you index into the array using the rank (or thread id) and that array stores the role for each round.

    [code lang=”C”]
    typedef struct {
    bool myflags[PARITY_BIT][MAX_ROUNDS];
    bool *partnerflags[PARITY_BIT][MAX_ROUNDS];
    } flags_t;

    void flags_init(flags_t flags[MAX_ROUNDS])
    {
    int i,j,k;

    for (i = 0; i < MAX_NUM_THREADS; i++) {
    for (j = 0; j < PARITY_BIT; j++) {
    for (k = 0; k < MAX_NUM_THREADS; k++) {
    flags[i].myflags[j][k] = false;
    }
    }
    }
    }
    [/code]

    OpenMP and OpenMPI

    Prior to starting I never heard of neither OpenMP nor OpenMPI. Overall, they are two impressive pieces of software that makes multi-threading (and message passing) way easier, much better than dealing with the Linux pthreads library.

    Summary

    Overall, the project was rewarding and my understanding of synchronization barriers (and the various flavors) were strengthen by hands on development. And if I ever need to write concurrent software for a professional project, I’ll definitely consider using OpenMP and OpenMPI instead of low level libraries like PThread.

  • Daily Review – Day ending in 2020/10/11

    Daily Review – Day ending in 2020/10/11

    I’m thrilled to be “off call” in about 4.5 hours, no longer tied to my pager and no longer anxious from possibility of waking up to the sound of nasty alarm. Really, the anxiety revolves around the randomness and the unknown of being paged.  What’s also variable is the length of these engagements: sometimes the troubleshooting takes 5 minutes and sometimes 5 hours. You just never know.

    The point it this: I’m happy to return to a normal work week.

    Best parts of my day

    • Laying in bed next to my wife at night. For the past four or five months I’ve been sleeping on the floor on a foam tri fold out mattress laid out on the uncomfortable carpet floor. And finally, now that we are finally moved into our new home, I’m sleeping on a real bed and last night my wife and I laid next to one another. Sure, it was only about 5-10 minutes but hey: it’s the little things right?

    Graduate School

    • Wrote up a paper summary on “Building Reliable High Performance Communication Systems from components
    • Drew figures of barrier synchronization on my iPad
    • Wrote up Project 2 (barrier synchronization) work log into Google Docs to share with my project partner
    My doodle of dissemination barrier
    My doodle of dissemination barrier

    Thoughts

    • Amazon Web Services draws inspiration from academia. For example, the techniques and principles used to build overlay networks within EC2 Network resemble the principles from the paper Active Networks (although there’s probably even earlier papers to draw inspiration from)
  • Weekly Review – Week ending in 2020/10/11

    Weekly Review – Week ending in 2020/10/11

    I’m shattered. This past week really broke me, the numerous 3:30 AM wake ups and the long operational issues running until 09:30 PM (past the time I’d like to be asleep). To recover from this taxing work week, I’m taking next Thursday and Friday off.

    Despite the rough week, I’m relieved that my wife and I are totally moved into our new home — not fully unboxed — but at least I’m waking up to a warm home.

    Work

    • Worked consumed my entire life this week
    • Being tied to the laptop and pager impacts not just me but my family as well
    • Sporadic wake ups and late nights off the entire schedule, breaking many of my rituals
    • Because of all this I’m taking 2 days off next week to recover and catch up on lost time eaten by the heavy work week
    • Over 300+ signed up for my event at Amazon
      • Hosting a panel discussion with (4) senior engineers at Amazon on career growth and promotions
    • This particular week was more difficult than others
      • Waking up at 03:30 AM multiple nights in a row
      • Operational issues lasting until 10:00 pm

    Family

    • First week living in the new house in Renton
    • Having a child underscores the fact how fleet time is
    • Family has changed my entire world, flipping it upside down
    • I don’t notice the changes every day but sometimes I’ll pause and take a look at her and she’s not only radically physically changing but developmentally as well
      • She’s taking her first steps
      • She’s couch surfing
      • She’s uttering her first words (surprisingly its “ball”)
    • Moving to a bigger home in Renton in retrospect has been the best thing that has happened
      • Neighbor was mowing their front lawn and offered to mow ours at the same time (took them up on that offer)

    Marketing

    • Chipping away at “This is marketing” book while in bed at night

     

  • Don’t break the (writing) chain … has been broken

    Don’t break the (writing) chain … has been broken

    This week, my cumulative “write every day” streak has been broken (almost 2 months of consistent writing every day), thanks to one of the roughest weeks at work. I normally start every day off with some light blogging — even if its for 5 or 10 minutes — but almost every day this week I was prematurely woken up due to my pager alarming me out of bed. So honestly, I couldn’t be more happier that it’s Friday (TGIF, for real) even though I have 2 more days (over the weekend) of being on call; I haven’t felt this physically and mentally and emotionally exhausted in a long time. Every time I get paged out of bed I’m forced to get my mental gears in motion and it’s very difficult switch off, making it nearly impossible to go back to sleep for a nap.

    So the days have been … very long.

    Oh well.

    So now, it’s time to reset the “cumulative days” of writing counter

  • Super long day & Having good co-workers

    Super long day & Having good co-workers

    In yesterday’s post, I had mentioned that I was paged out of bed at around 03:30 AM because of an operational issue. And for the rest of the day, my mind was fried and I practically looked like a zombie the rest of the day, my word constantly slurred. On top of all that, a different issue cropped up in the evening, the event lasting several hours and robbing me from having dinner with my family. Ugh.

    Silver lining: my colleague Paul, a god damn saint, overrode the night time shift and he took my on call over night, allowing me to get some much needed rest. The next morning, I sent an e-mail over to my manager, praising Paul and making sure that his good deed(s) does not go unnoticed.

  • Being paged out of bed at 3 AM …

    Being paged out of bed at 3 AM …

    Sadly I didn’t get to start the day off with writing, my morning routine, since my phone paged me out of bed at 3 AM due to an operational issue from work that lasted about about three hours. Because of this, I know that I’ll feel “off” the rest of the day given that I’ve been awake for over 5 hours and the time is barely 9 AM. Oh well. Time to heat up another Chai Latte to keep me awake for the day …

    Sometimes I wonder if this is all worth it …

  • What does “invariant partial ordering” mean in Leslie Lamport’s “Time, Clocks, and the Ordering of Events in a Distributed System”

    What does “invariant partial ordering” mean in Leslie Lamport’s “Time, Clocks, and the Ordering of Events in a Distributed System”

    In the conclusion of Time, Clocks, and the Ordering of Events in a Distributed System, Leslie Lamport states that: the concept of “happening before” defines an invariant partial ordering of the events in a distributed multiprocess system.

    According to a stackoverflow post, Jacob Baskin states that an invariant is a property of the program state that is always true. Tieing that together with the original question that I had asked in the previous paragraph, I think what Leslie is trying to say is that because of the happening order event, we know that [in a distributed system] events will be always be partially ordered — not totally ordered.

  • Farmhouse style doors – sliding door ideas for my home office

    Below are a couple photos of farm house doors that I think have good taste. I’m thinking of adding one of these type of doors to my new home office and wanted to share a couple of styles that I thought were both warm and aesthetically pleasing, two qualities I’m aiming to bring out of this new set up of mine.

    Grey farm house door. Credit: jettsetfarmhouse.com

     

    Beautiful brown grey farmhouse door. Credit – https://www.instagram.com/sheltercustombuiltliving/
  • Weekly Review & Week ending in 2020/10/03

    Weekly Review & Week ending in 2020/10/03

    This weekly review is the first one that I’m typing from my home office in our new Renton house. And being in this new house feels good, feels great. I feel extremely blessed and luckily.  And although I do know that Jess and I and Elliott and the dogs could live contently even if the roof over our head was constructed from a cardboard box, I know that we’re much happier knowing that we have some more floor space for Elliott to crawl around, more space in the yard for the dogs to sprawl, more space for Jess to have her own own work space (maybe it’ll turn into a writing or arts and crafts room) and more space for me to work out of an office that’s finally separate from where I sleep at night.

    All the hard work of moving into the new house has definitely made up for the horrible work week. On Wednesday, just before I took two days off for “vacation”, my manager and I had our 1:1 and he was supposed to provide me with some written feedback from a principle engineer that I had closely worked with on a big project last year, feedback that would be used for my (hopefully upcoming over the next quarter or two) promotion and feedback from someone who I had (up until that point) considered a mentor, someone who I thought had my back. Long story short, I won’t be receiving any feedback from this one person (despite four other people providing glowing remarks) because our relationship has basically shifted (for the worst) after he had asked me to take a project management role for some some product and I turned that down. In short, he didn’t like me saying no to him. Anyways, I’m straight fed up (and disappointed) with this person at work and I realized within the Amazon organization (and just about every organization) there will be other people just like him, people who give you the impression that they advocate for you but behind closed doors they do the opposite. This adds fuel to my fire and I hope that one day, when I am in a senior engineering role, I’ll be the person helping people move up in the organization, not someone who keeps them down.

    Writing

    Family & Friends

    Jess holding Elliott (on her 1st birthday) at Maple Leaf Park
    • Sadly didn’t get to really celebrate neither Jess’s 29th birthday nor Elliott’s (first) birthday. Their birthdays landed in the midst of us packing and moving homes so I feel a little bad for not properly celebrating it. We’ll make it up this week now that we are done moving and will celebrate the events properly in the comfort of our new home
    • Talked with my friend Brian (a marketer) over the phone while I was driving the U-Haul to the new house in Renton. I was able to pick his brain a little bit about personal brand and marketing and my visions for wanting to become an established non-fiction technical writer over the next few years. After talking about that topic, we just played some catch up: always nice to sync up with friends.
    • Got pretty emotionally while snuggling with Metric one of the nights we were moving. I noticed that three of her whiskers are now completely white, the colors reminding me that she’s aging and that she’s no longer and puppy and that one day she’s no longer going to be in my life. The day that comes, I don’t have any idea on how I will handle it. I love her so damn much and she’s been through so much with me over the past six years.

    Graduate School

    • Took the midterm exam (one day early – hooray) one week from today and glad that I just got it out of the way because the following Monday was hectic and tiring
    • Watched the OpenMP tutorial series published by Intel and Tim Mattson
    • Watched the Introduction to Distributed Systems from Advanced Operating Systems course
    • Wrote two barrier synchronizations using OpenMP: centralized counting barrier (with sense reversal) and dissemination barrier
  • Deadlocking in centralized counting barrier implementation

    Deadlocking in centralized counting barrier implementation

    For project 2 of my advanced operating systems course, I quickly typed up what I thought was a functional simple centralized counting barrier implementation. However, after launching the compiled executable multiple times in a row, I noticed the program would hang and not immediately exit … damn deadlock. Instead of loading the debugger and inspecting each of the thread stack frames, I revisited the code and reasoned about why the code would deadlock.

    In the code (below), lines 29-32 are the culprit for the race condition. Just as one thread (say thread B) is about to enter the while (count > 0) loop, another thread (the last thread) could reset the count = NUM_THREADS. In this situation, thread B would continue spinning: forever.

    Centralized Barrier Example from Lecture Slides

    Centralized Barrier

    Code Snippet

    [code lang=”C” highlight=”29-32″]
    #include <stdbool.h>
    #include <omp.h>
    #include <stdio.h>

    #define NUM_THREADS 3

    int main(int argc, char **argv)
    {
    int count = NUM_THREADS;
    bool globalsense = true;

    #pragma omp parallel num_threads(NUM_THREADS) shared(count)
    {
    #pragma omp critical
    {
    count = count – 1;
    }

    /*
    * Race condition possible here. Say 2 threads enter, thread A and
    * thread B. Thread A scheduled first and is about to enter the while
    * (count > 0) loop. But just before then, thread B enters (count == 0)
    * and sets count = 2. At which point, we have a deadlock, thread A
    * cannot break free out of the barrier
    *
    */

    if (count == 0) {
    count = NUM_THREADS;
    } else {
    while (count > 0) {
    printf("Spinning …. count = %d\n", count);
    }
    while (count != NUM_THREADS){
    printf("Spinning on count\n");
    }
    }

    }

    printf("All done\n");
    }

    [/code]