Fields of Fiber

    Low Latency Datacenter Networks – Why Nanoseconds Matter

    Posted by Jonathan Reeves on Aug 11, 2015 11:22:00 AM
    Author Image
    Find me on:
    Speed of Light - Eevlva

    Fiber Mountain OPX measured at 5 nanosecond port-to-port latency, to one or many output destinations

    A significant and important performance metric in any network is latency, or the time taken for information to be relayed from an input port to an output port. There have been many industry papers written on the topic, and recent advances in computing performance, storage performance and network bandwidth bring greater focus to the impact of latency on user experience and application performance.

    • Recent advances in storage technology such as NVMe drives, next generation flash / ssd technologies and RAM-based distributed memory architectures rely on low network latency to perform at optimal levels. Higher latency means slower applications.
    • Many network monitoring and security applications rely on the accurate time stamping of packets to correlate events between systems. A discrepancy in the time stamping of packets due to network latency may impact network troubleshooting and analysis.
    • The most familiar example for many is high frequency trading, where distribution systems require the simultaneous delivery of data to multiple destinations. A miniscule difference in delivery time in these networks can have a huge impact on the bottom line.

    With Ethernet bandwidths increasing beyond 10-gigabits per second (gbps) to 25-, 40- and 100-gbps, the impact of latency becomes even more noticeable and significant.

    We knew that our Optical Path Exchange (OPX) was faster than any comparable technology available, but it was only recently that we tracked down test equipment which could measure such low latency with precision, the Anritsu BERTWave test system.


    Click here for high resolution image and notes.

    Where does network latency come from? Several sources:

    • Fiber Latency: Light traveling through one meter of multimode optical fiber cables takes approx. 5 nanoseconds.
    • Switch Latency: Whether an Ethernet switch, Infiniband, or other options commonly used in today’s networks, the standard for low latency has been on the order of 0.5 to 1.0 microseconds. (500 to 1,000 nanoseconds)
    • Queuing & Oversubscription: When lightly loaded, the queuing delay through a switch is very low. However, once a level of over subscription occurs the latency will climb, often to hundreds of microseconds.

    And that test of the latency of Fiber Mountain’s OPX?

    Less than 5 nanoseconds.

    That’s right, based on FIFO (first in, first out) methodology, and controlling for the fiber lengths connecting the test system to the OPX, we validated a 5 nanosecond optical switching time for the OPX, or 1/100th of the latency through a typical “low-latency” Ethernet switch. The 5 nanosecond time recorded includes internal fiber cabling and based on eliminating these effects we estimate the switching time of the optical, electrical, optical path within the OPX to be under 3 nanoseconds.

    Also important is that the same low latency was observed when sending a single optical input to many outputs, i.e. broadcast mode or optical tapping of data for monitoring or security purposes.

    At Fiber Mountain, we are seeking to break the barriers of data center network latency through the use of optical switching technology. This test result shows that our OPX, and our entire Glass Core approach to network architecture, is well on the way to achieving that goal.

    Press Release: Fiber Mountain™ Shatters SDN Latency Barriers; Reduces Network Latency By Up to 100 Times

    Learn more about Layer 1 SDN solutions!


    Topics: Glass Core, OPX