According to a stackoverflow post, Jacob Baskin states that an invariant is a property of the program state that is always true. Tieing that together with the original question that I had asked in the previous paragraph, I think what Leslie is trying to say is that because of the happening order event, we know that [in a distributed system] events will be always be partially ordered — not totally ordered.
Below are a couple photos of farm house doors that I think have good taste. I’m thinking of adding one of these type of doors to my new home office and wanted to share a couple of styles that I thought were both warm and aesthetically pleasing, two qualities I’m aiming to bring out of this new set up of mine.
Grey farm house door. Credit: jettsetfarmhouse.com
Beautiful brown grey farmhouse door. Credit – https://www.instagram.com/sheltercustombuiltliving/
This weekly review is the first one that I’m typing from my home office in our new Renton house. And being in this new house feels good, feels great. I feel extremely blessed and luckily. And although I do know that Jess and I and Elliott and the dogs could live contently even if the roof over our head was constructed from a cardboard box, I know that we’re much happier knowing that we have some more floor space for Elliott to crawl around, more space in the yard for the dogs to sprawl, more space for Jess to have her own own work space (maybe it’ll turn into a writing or arts and crafts room) and more space for me to work out of an office that’s finally separate from where I sleep at night.
All the hard work of moving into the new house has definitely made up for the horrible work week. On Wednesday, just before I took two days off for “vacation”, my manager and I had our 1:1 and he was supposed to provide me with some written feedback from a principle engineer that I had closely worked with on a big project last year, feedback that would be used for my (hopefully upcoming over the next quarter or two) promotion and feedback from someone who I had (up until that point) considered a mentor, someone who I thought had my back. Long story short, I won’t be receiving any feedback from this one person (despite four other people providing glowing remarks) because our relationship has basically shifted (for the worst) after he had asked me to take a project management role for some some product and I turned that down. In short, he didn’t like me saying no to him. Anyways, I’m straight fed up (and disappointed) with this person at work and I realized within the Amazon organization (and just about every organization) there will be other people just like him, people who give you the impression that they advocate for you but behind closed doors they do the opposite. This adds fuel to my fire and I hope that one day, when I am in a senior engineering role, I’ll be the person helping people move up in the organization, not someone who keeps them down.
Published notes on OpenMP (from Intel’s YouTube tutorial series)
Family & Friends
Jess holding Elliott (on her 1st birthday) at Maple Leaf Park
Sadly didn’t get to really celebrate neither Jess’s 29th birthday nor Elliott’s (first) birthday. Their birthdays landed in the midst of us packing and moving homes so I feel a little bad for not properly celebrating it. We’ll make it up this week now that we are done moving and will celebrate the events properly in the comfort of our new home
Talked with my friend Brian (a marketer) over the phone while I was driving the U-Haul to the new house in Renton. I was able to pick his brain a little bit about personal brand and marketing and my visions for wanting to become an established non-fiction technical writer over the next few years. After talking about that topic, we just played some catch up: always nice to sync up with friends.
Got pretty emotionally while snuggling with Metric one of the nights we were moving. I noticed that three of her whiskers are now completely white, the colors reminding me that she’s aging and that she’s no longer and puppy and that one day she’s no longer going to be in my life. The day that comes, I don’t have any idea on how I will handle it. I love her so damn much and she’s been through so much with me over the past six years.
Graduate School
Took the midterm exam (one day early – hooray) one week from today and glad that I just got it out of the way because the following Monday was hectic and tiring
Watched the OpenMP tutorial series published by Intel and Tim Mattson
Watched the Introduction to Distributed Systems from Advanced Operating Systems course
Wrote two barrier synchronizations using OpenMP: centralized counting barrier (with sense reversal) and dissemination barrier
For project 2 of my advanced operating systems course, I quickly typed up what I thought was a functional simple centralized counting barrier implementation. However, after launching the compiled executable multiple times in a row, I noticed the program would hang and not immediately exit … damn deadlock. Instead of loading the debugger and inspecting each of the thread stack frames, I revisited the code and reasoned about why the code would deadlock.
In the code (below), lines 29-32 are the culprit for the race condition. Just as one thread (say thread B) is about to enter the while (count > 0) loop, another thread (the last thread) could reset the count = NUM_THREADS. In this situation, thread B would continue spinning: forever.
/*
* Race condition possible here. Say 2 threads enter, thread A and
* thread B. Thread A scheduled first and is about to enter the while
* (count > 0) loop. But just before then, thread B enters (count == 0)
* and sets count = 2. At which point, we have a deadlock, thread A
* cannot break free out of the barrier
*
*/
if (count == 0) {
count = NUM_THREADS;
} else {
while (count > 0) {
printf("Spinning …. count = %d\n", count);
}
while (count != NUM_THREADS){
printf("Spinning on count\n");
}
}
The main take away with the introduction to distributed systems lectures is that as system designers, we need to carefully inspect our program and identify what events in our system can run concurrently (as well as what cannot run concurrency or must be serialized). To this end, we need to identify what events must happen before other events. And most importantly, we should only consider running distributed systems to increase performance when the time it takes to process an event exceeds the time it takes to send a message between nodes. Otherwise, just stick to local computation.
Lots of similarity between parallel systems and distributed systems. Symbiotic relationship between hardware and software (protocol stack)
Quiz: What is a distributed system?
Summary
Distributed systems are connected via a LAN/WAN, communicate only via messages, and message time is greater than the event time (not sure what this means really)
Distributed Systems Definition
Summary
Key Words: event computation time, message transmission time; Third property is that message communication time is significantly larger than the event computation time (on a single node). Leslie’s definition is: A system is distributed if the message transmission time (TM) is not negligible to the time between events in a single process. The main take away is that the algorithms or applications run on distributed noes must take longer (in computation time) than the communication otherwise no benefit of parallelism.
A fun example
Summary
Key Words: happened before relationship; Set of beliefs ingrained in a distributed system example. Within a single process, events are totally ordered. Second belief is that you cannot receive the receipt of a message until after the initial message is sent.
Happened Before Relationship
Happened Before Event description
Summary
The happened before relationship means one of two things. For events A and B, it means that 1) A and B are in the same process or 2) A is the sender of the message and B is the receiver of the message
Quiz Relation
Summary
You cannot assume or say anything about the order of A and B
Happy before relationship (continued)
Summary
When you cannot assume the ordering of events, they might be concurrent. One event might run before the other during one invocation. Then perhaps the order gets flipped. So, these types of events are not happened before, no apparent relationship between the two events. But my question is: what are some of the synchronization and communication and timing issues that he had mentioned?
Identifying Events
Identifying concurrent and dependent and transitive events
Summary
Remember the events that “happened before” are not only between processes but within processes themselves. Also, two concurrent events are basically when we cannot guarantee whether or not an event in process A will run before an event in process B, again between processes
Example of Event Ordering
Summary
Basically the events between processes can be concurrent since there’s no guarantee in terms of wall clock time when they will execute, no ordering
I’m implementating the dissemination barrier (above) in C for my advanced OS course and I’m not quite sure I understand the pseudo code itself. In particular, I don’t get the point of the parity flag …. what’s the point of it? What problem does it solve? Isn’t the localsense variable sufficient to detect whether or not the threads (or processes) synchronized? I’m guessing that the parity flag helps with multiple invocations of the barrier but that’s just a hunch.
My body aches from the first day of moving houses, my body sore from all the loading and unloading of tightly packed boxes from the house and into the back of the 15″ foot U-Haul truck.
Rant
U-Haul at Burien employes some of the most unprofessional staff with the worst customer service. The staff were not only rude to customers but extremely denigrating to one another, the manager even shaming her employee in front of customers, the manager saying (and I quote): “You are just not doing a very good job today are you?”
No electricity
Stepped into our new home and I immediately noticed that the lights didn’t flicker on when I hit the switch. I had also noticed that the digital clock on the stove was not displaying the time. No electricity.
Called Puget sound energy (PSE) over the phone and within 5 minutes of talking to the representative, had them activate the electricity. They mentioned it would take up to 24 hours maximum
I want to learn how to better market myself and what it means to create my own personal brand and how I might be able to apply these marketing skills in my career (as a software developer and computer scientist) and as a writer. Because I do wonder what sort of impact and influence I would have if I applied even an ounce of marketing or branding.
I’m planning on sinking my teeth into two different marketing books. My guitar instructor had recommended Seth Godin’s This is Marketing and a hacker news user recommended an e-book titled Authority by (someone I’ve never heard of) named Nathan Barry. I’ll take a crack at these two books and report back on the main takeaways and whether or not I recommend you reading them.
At this moment in time, I have no clue what it means to build a personal brand (in all honestly I don’t even know what a personal brand means). Nonetheless, I do think learning about a little marketing and branding (really what’s the difference between the two) will be a worthwhile, non-technical skill to develop since I’d like down the line to become a full time writer and teacher and think marketing and branding will play a key role in making that happen.
Yesterday … was exhausting. A few times throughout the day I actually felt my body shut down and I nearly fell asleep while working. Although I cumulatively got like 7.5 hours of sleep, my sleep was constantly interrupted since Elliott has been (presumably) teething and letting out these screams at 1:00 AM and 3:00AM and 04:00 AM, the screams piercing through out thin walls and echoing throughout the rest of the house.
On top of all this, I felt so overwhelmed with the house move, knowing that we only have just a few days left and there’s so much left to do still. But fortunately Jess and I partnered up and split some tasks up between the two of us. That really helped.
What did I learn
The last true symmetric multiprocessor machine was around the late 1980s. I thought that they were much more prevalent but it appears that most of the hardware today run on non uniformed memory access (NUMA) machines
How to identify and trace concurrent events between processes (a little more difficult than I had originally anticipated) using a good old pen and paper
Politics
Watched about 30 minutes of the first debate between Joe Biden and Donald Trump. Honestly, it’s like watching a circus. Trump constantly interrupts Joe Biden and Joe Biden often goes completely off topic. For example, while talking about his position on supporting the military, he brings up the fact that his son served in the military and currently recovering from drug addiction. While nice to know …. I wonder, it’s totally irrelevant to the conversation.
Family and Friends
Jess and I operated as a team yesterday. Together in the morning, we signed an hours worth of paperwork with a notary sent by the escrow company. After, we divided and conquered. She and Elliott performed the final walk through in Renton while I sorted out how to get us internet (so painful dealing with internet sales representatives) and then I drove to Wells Fargo for my scheduled appointment to transfer the big amount for the down payment of the house.
Interesting Quotes or Idioms
Prime the pump (from video lecture series in Distributed System, Quiz on “relation”)
I’m watching the YouTube learning series called “Introduction to OpenMP” in order to get a better understanding of how I can use the framework for my second project in advanced operating systems. You might find the below notes useful if you don’t want to sit through the entire video series.
Introduction to OpenMP: 02 part 1 Module 1
Moore’s Law
Summary
Neat that he’s going over the history, talking about Moore’s Law. The key take away now is that hardware designers are now going to optimize for power and software developers have to write parallel programs: there’s no free lunch. No magic compiler that will take sequential code and turn it into parallel code. Performance now c
Introduction to OpenMP – 02 Part 2 Module 1
Concurrency vs Parallelism
Summary
The last step should be picking up the OpenMP Library (or any other parallel programming library). What should be done firs and foremost is breaking down your problem (the act of this has not been able to automated) into concurrent parts (this requires understanding of the problem space) and then figure out which can run in parallel. Once you figure that out, then you can use compiler syntactical magic (i.e. pragmas) to direct your compiler and then sprinkle some additional pragmas that will help the compiler tell where the code should enter and where it should exit.
Introduction to OpenMP – 03 Module 02
OpenMP Solution Stack
Summary
It’s highly likely that the compiler that you are using already supports OpenMP. For gcc, pass in -fopenmp. And then include the prototype, and add some syntactic sugar (i.e. #pragma amp parallel) which basically gives the program a bunch of threads.
Introduction to OpenMP: 04 Discussion 1
Shared address space
Summary
OpenMP assumes a shared memory address space architecture. The last true SMP (symmetric multi-processor) machine was in the late 1980s so most machines now run on NUMA (non uniformed multiple access) architectures. To exploit this architecture, we need to schedule our threads intelligently (which map to the same heap but contain different stacks) and place data in our cache’s as close as possible to their private caches. And, just like the professor said in advanced OS, we need to limit global data sharing and limit (as much as possible) the use of synchronization variables since they both slow down performance
