Thursday, August 21, 2014

TopKShortestPaths and Big-O Issues

This is the story of one of those times when Big-O complexity of an algorithm really really matters. The problem I was dealing with was to find equal-cost multiple paths in a data-center network. My initial approach was to use JGrapht library's KShortestPaths to find top K shortest paths and then iterate over them starting from least-cost path, to select all the paths with equal cost. This also allows to alter the definition of "equal-cost" by allowing some tolerance in costs to be considered equal .So far, so good. Until, I ran the algorithm on a medium-sized graph with ~400 nodes and ~1100 links. Who could have guessed that a single such calculation of shortest paths would take almost 3000 ms?! Well, you could have guessed it, had you noticed the Big-O complexity of JGrapht's KShortestPaths. It's  O(k*n*(m^2)), where m is the number of edges (here, 1100) , n is the number of vertices (here, 405) and k is the number of paths. A simple rule of thumb to estimate if  an algorithm is going to run in reasonable time (less than 1 second) is to see if the quantity mentioned in Big-O (here, k * n * m ^2) is less than 10 ^ 6. With JGrapht's KShortestPaths, it's more like 10^9 (16 * 400 * 1100 ^ 2, here k = 16), and therefore it was destined to take too long even on a medium-sized graph. 

In comes, Yen's Algorithm to find top-k shortest loop-less paths (who wants a loop anyway!) and the execution time for one such calculation goes down from ~3000 ms to ~1 ms. I used the implementation given here, which although has a worst-case complexity of O(k*n(m + nlogn)), usually performs better than that as observed in computational experiments. For n = 400, m = 1100, the quantity inside Big-O is well below 10 ^ 6, and therefore performs much better as compared to JGrapht's KShortestPaths. In short, Yen's algorithm deserves to be in JGrapht for its sheer performance advantage. 

Friday, August 15, 2014

A Note on ECMP Layer-2 and Layer-3 Hashing

This took me quite a while to figure out. Damn you hazy terminologies!

So, coming to the point, Equal-Cost Multi-pathing, or ECMP in short, comes usually in 2 flavors: Layer-2 ECMP and Layer-3 ECMP. As the name suggests, ECMP is a technique used to balance "flows" (will come to this soon) over multiple paths in Layer-2 or Layer-3. Layer-2 multi-pathing is useful in Data-centers, where the alternative Spanning Tree Protocol implies wastage of redundant links. Layer-2 ECMP allows better utilization of the available bandwidth by balancing "flows" over the multiple paths available. Note that, each "flow" sticks to its "path" during its lifetime. Ditto happens for Layer-3 paths.

Now, the tricky part that took me so long to grasp was how do you determine a "flow"? My assumption was that Layer-2 ECMP would make use of a subset of Layer-2 header fields (Source MAC, Destination MAC etc.) while Layer-3 ECMP would make use of  a subset of Layer-3 (and optionally Layer-2) header fields (Source IP, Destination IP etc.) and the hash would be calculated over this subset to find the best next-hop. However, it turns out that the matter of defining "flows" is entirely disjunct from the "Layer"(2 or 3) at which ECMP is being done. So, "flows" can be defined using a 2-tuple (Source IP, Destination IP) or a 5-tuple (Source/Destination IP, Source/Destination Port, Protocol) or even a 7-tuple. Each of these 2/5/7-tuples can be used as input to the hash function irrespective of whether it's Layer-2 or Layer-3 ECMP. Effectively, "Layer-2" ECMP can look into header-fields of layers above it (in this case, IP addresses, TCP/UDP Ports). Similarly, "Layer-3" ECMP can look into header-fields of Transport Layer (TCP/UDP ports) thereby breaking the principal that a network layer does NOT look into  header-fields of layers above it. They should have called it "Layer-2 (Layer-3) ECMP with Layer-3 (or Layer-4) Hash" depending on the definition of "flow" being used. Would have saved me a lot of time.  

Reference: http://tools.ietf.org/html/rfc6438