Sampling synchronization with Gigabit Ethernet

Abstract

In the recent past we have seen open standards based data acquisition and telemetry systems supplant proprietary systems. Asynchronous Transfer Mode (ATM) was a great choice for sonar and related synchronous sampling systems as the network was built on the concept of a synchronous backbone from which system timing could be derived. Unfortunately for all but the largest of backbones, ATM has been replaced with other open protocols, chiefly Ethernet, which lacks the underlying precept of the synchronous backbone. We have seen open standards develop in the last several years with the intent of addressing the need for synchronicity, such as IEEE 1588 and Synchronous Ethernet. QNA-TSG has developed a method of using standard Gigabit Ethernet to achieve synchronous sampling using COTS networking equipment and minor modifications that allow GPS locked, synchronous sampling. With this new extension of locking sampling to GPS timing, the technology exists now to create long baseline arrays never before realizable. Synchronization is conceptually trivial, have two things happen at the same time, which in practice is quite difficult to implement. Synchronization, at its root, requires two things: that all nodes in the system count time in the same manner, and that one moment in time can be uniquely identified as time zero. Said another way, all nodes in the system must use the same clock, and all nodes need a synchronization event. Fiber optic based Gigabit Ethernet uses a synchronous link layer that allows the physical layer to continuously recover the transport clock. Many Gigabit Ethernet COTS switches use a single clock source to drive all output data link clocks. The result is an architecture that allows all devices plugged into the same switch to have access to a common clock. For more complicated architectures that use cascaded switches it is necessary to have a switch that can carry the clock forward. This requires that the switch recover the data clock, clean the clock up, and then use that clock as the transmit clock for another port. Although no Tier 1 supplier currently offers such an option, QNA-TSG has built, demonstrated, and deployed such a device. The second step in synchronous sampling is the time zero event. An obvious standards based approach to developing a time zero event is to use NTP or one of the extensions to NTP. Unfortunately, the standards based approaches are limited in their accuracy depending on the implementation. Most implementations are limited by the interrupt latency and the distribution of that latency. Typically NTP can achieve synchronization on a typical computer of approximately 1–0.1ms, using nothing but network communications. Systems can be constructed that use GPS timing devices that feed into the computer a one pulse per second (1PPS) to improve timing accuracy to on the order of 1–10us. QNA-TSG developed a technique where the time stamping for NTP is performed at the hardware layer. Combined with an understanding of packet latency through a switch, it is possible to time stamp and achieve time zero synchronization to within +/−75ns across multiple platforms spaced far apart. In this paper we outline the techniques implemented to achieve synchronous sampling of data across multiple networked nodes using COTS networking equipment. Additionally we present a case study demonstrating synchronization of multiple devices connected to the same switch, a review of available components to implement the data acquisition and telemetry systems, and the modifications to existing Tier 1 components that allows Gigabit Ethernet to distribute GPS locked timing signals.