Achieving and maintaining an ultra-low latency FX trading infrastructure

NOTE: This article was originally published on e-Forex.net by Paul Golden.

Ultra-low latency trading can be defined as a system capable of processing data in nanoseconds, compared to standard low latency which is measured in milliseconds or microseconds. Bridging this gap is expensive, though, and requires specialised hardware and software.

“It is not just about going faster; it’s about designing a sleek, custom built engine capable of conquering those tiny fractions of time that make all the difference in high stakes trading,” explains Ariel Silahian, Director of Electronic Trading at SiS Software Factory.

You can pay to be as close to the exchange as possible, but with equitable access you can only get as close as other participants states Gordon McArthur, CEO Beeks Group. “If you have fixed latency budgets then your competitors generally do as well, so ultra-low latency is about ensuring all other elements of your trading system are as fast as possible,” he says.

The number of participants in the market at different stages of implementing ultra-low latency limits the benefits since interactions between two parties will only be as fast as the slowest participant observes Alexander Culiniac, CTO/ Managing Director of the commercial banking & payment product business group at SmartTrade Technologies.

Attention to detail is vital

“An FX trading platform must contend with various latency types, such as network, propagation, processing, and software-related delays,” says Culiniac . “Achieving ultra-low latency requires a holistic approach that meticulously optimises each step from data transmission to execution.”

The primary users of these systems include high-frequency trading firms, hedge funds, and market makers, although as the value of improved performance rises relative to infrastructure costs, firms across the sell-side and buy-side are looking to achieve ultra-low latency.

Silahian notes that the decentralised, fragmented nature of the FX market presents some specific challenges. “Unlike equities, FX is more about exploiting delayed prices across different platforms,” he says. “Execution speed is generally a little slower – even the fastest margin FX broker isn’t as quick on the draw as Nasdaq. In addition, looser regulation in FX leads to protective measures by market players against high speed trading strategies, such as speed bumps or last look.”

The primary protocol for FX market data and trading is slow, lacks traceable timing, and is peer-topeer rather than multicast observes McArthur. “Some FX markets have started to offer lower latency multicast market and binary trading protocols, but these are still in the minority,” he says. “With no single source, it takes a lot more time and effort to apply the methodical approach to ensuring latency-tuned access between participants.”

Specific factors that impact latency include:

• Slow infrastructure (servers/network cards/switches)
• Additional infrastructure adding hops (firewalls, layers of switches)
• Physical distance from trading counterparties
• Code that has not been optimised for speed
• Subscribing to too much market data

To determine whether these issues can be resolved via software or hardware solutions, the first step is to identify the type of latency (network, disk, application) suggests Eugene Markman, ION FX Chief Operating Officer.

“Each requires a different approach,” he continues. “Network latency issues, for instance, may be addressed with software optimisations or by upgrading network hardware. A root cause analysis is important to determine whether the source is a software bug, inefficient code, network congestion, or hardware limitations.”

From this, bottlenecks will need to be identified and resolved. If the bottleneck is due to hardware limitations such as a slow disk drive or insufficient RAM, a hardware upgrade may be necessary, whereas optimisation can resolve inefficient resource usage. Hardware upgrades or replacements can be expensive, while software optimisations are often more cost-effective.

Don’t rest on your laurels

Enabling ultra-low latency trading is an ongoing process. In addition to inadequate or ageing hardware components, slow network connections, poorly designed software architecture, and inefficient code, there are a host of other factors that can degrade operational performance. “Relying on third party data providers or trading platforms can introduce latency, especially if these services experience delays or downtime,” says Markman. “Inefficient data processing or a lack of parallel processing can also lead to latency, as can inefficient order routing algorithms or connections to exchanges.”

Performing thorough risk checks and compliance validation can add latency to trading systems, which underlines the challenge of balancing low latency execution and effective risk management.

“Rapidly changing market conditions can stress trading systems and lead to latency degradation, as can overloading the trading system with too many orders or data feeds,” adds Markman. “It is also essential to carefully validate software updates or patches before deploying them in a trading environment.”

Inevitably, cost is a major consideration. Software fixes tend to be cheaper, but new hardware may provide a long term solution. The final decision is often influenced by whether the firm has in-house software or hardware expertise, as well as the closeness of its vendor relationships.
Distance from the participant is yet another crucial factor. McArthur notes that just 200 metres of cabling can introduce approximately one microsecond of latency in each direction.

“The choice between store and forward switching and low latency cut-through switches is also significant,” he explains. “Enterprise switches (designed for capacity and throughput) often involve large buffers and store and forward operation, taking tens of microseconds to pass on packets. Conversely, purpose built low latency cut-through switches can deliver packets in hundreds of nanoseconds.”

Where there is redundancy built into the architecture (for example, high availability switches, bonded NICs, or primary and secondary network links) it can be easy to forget to test the failover or monitor the performance of secondary routes.

“A good system will have ongoing failover/resiliency testing and monitor the impact on performance,” adds McArthur.

Keep your eyes on the prize

To establish an ultra-low latency framework it is crucial to have clear targets, controls, and an understanding of the trading platform’s scope.

Any deviation from these parameters can result in performance declines, suggests Culiniac. “A comprehensive monitoring and analysis system is therefore essential, one that integrates both technical and business indicators to detect and address any early signs of potential degradation,” he adds. Markman refers to a number of analytical toolsets that can help firms monitor the state of their trading infrastructures:

• Specialised market data feed handlers like MarketFactory enable efficient and low latency handling of market data feeds from various ECNs and data providers
• In-memory databases such as Apache Kafka can store and process data with extremely low latency
• Tools like Grafana and Kibana enable the creation of customisable, real time dashboards for monitoring trading system performance and latency metrics
• The network monitoring capabilities of Crovill and Geneos can help to identify and address network latency issues
• Cloud-based platforms such as AWS Lambda, Google Cloud Functions, and Microsoft Azure Functions provide scalable resources for real time data analytics and can be integrated into trading systems

Ultra-low latency architecture is underpinned by a comprehensive suite of monitoring and analytical instruments which go beyond conventional systems. These tools include advanced statistical models capable of predicting potential bottlenecks through forecasting. “Additionally, AI enhances the system by grouping related incidents, allowing for more efficient troubleshooting and resolution,” says Culiniac. “Alert engines are fine-tuned to promptly notify technicians of any emerging issues. These sophisticated tools operate on top of the basic monitoring infrastructure, providing a multilayered defence against latency-related performance degradation.”

The slightest disruption – network congestion, for example, or a snag in scalability – can throw a spanner in the works, while dependencies on external services such as data providers are another wildcard. “Even a minor tweak can ripple through the latency landscape,” says Silahian. “This is where specialised monitoring systems earn their keep.”

The more information the better

Increased market volumes demand analytics that can scale in real time. McArthur observes that the global expansion of cloud infrastructure and lower latency links has shifted focus from a localised race to zero latency to a more globally connected trading infrastructure.

“A key trend is combining analytics from multiple sources,” he says. “Openness of data is crucial for seamless integration with other toolsets.”

The current toolkit for monitoring ultra-low latency trading setups is dispersed across individual tools. Network analytics tools use AI to catch network hiccups in real time and suggest fixes, and there are applications offering precise timestamping to track data flow down to the picosecond. Development frameworks are also on the table, easing the creation and upkeep of ultra-low latency setups, although Silahian says most firms will build their own tools and monitoring systems based on their specific needs. The decision to either undertake to build an ultra-low latency infrastructure in-house or outsource it to a specialist provider is usually influenced by financial resources and in-house expertise according to Markman.

“Most true low latency is built inhouse as firms like to retain the IP, but finding employees with the right skill set to be able to do this can be difficult as there are not many low latency engineers in the market,” he says. “Alternatively, hiring consultants can be very expensive. In general, the project would be costly, so budget will be a large deciding factor.”

In-house development demands a team with specialised knowledge in network and system architecture and if a firm lacks this expertise, it may lean towards outsourcing suggests Culiniac. “Building and maintaining an ultralow latency infrastructure can be expensive, so firms need to consider whether the potential return on investment justifies the upfront and ongoing expenses,” he says. “Outsourcing can be faster compared to building in-house – a critical factor in markets where timing is crucial – and companies may choose to outsource if ultra-low latency trading platforms is not their core business, allowing them to focus on their primary market activities.”

McArthur reckons the average timeto-money for an in-house build ranges from 18 months to three years if the firm can guarantee the right resources and location. “On top of that there is the capex for the infrastructure and also the cost of hosting and connectivity,” he adds. “People might question the security of outsourcing infrastructure rather than keeping it in-house, but the right infrastructure and provider will provide a solution that addresses these concerns.”

He suggests firms question each provider’s scalability and security offering, the measures they have in place to reduce downtime (as well as how compliant they are with industry regulations), their proximity to financial hubs in order to ensure real time data access, and the scalability of their solution.

“When your business grows your infrastructure will need to be able to accommodate increasing trading volumes,” adds McArthur.

Ask the difficult questions

When considering specialists for trading network and infrastructure, experience with ultra-low latency is vital according to Silahian, who cautions that although there are a plethora of providers, not all understand the intricacies of ultra-low latency operations.

“Customisation is another factor and specifically whether they can tailor solutions to fit your trading operations,” he says. “The support they offer post-setup – and how they price their service without skimping on quality – are other considerations, as are scalability, the ability to stay on the right side of regulators, and security.” Markman agrees that ultra-low latency expertise is vital and that working with a partner that has successfully built and integrated similar systems will reduce risk.

When deciding to partner with a specialist trading network and infrastructure provider, Culiniac also believes it is crucial to prioritise firms with a proven track record in achieving ultra-low latency since achieving such performance levels demands substantial R&D and time investment. “Additionally, the chosen provider should offer a comprehensive solution that optimises infrastructure, network, and software,” he concludes. “They should support a deep functional scope and have the capability to maintain and extend solution capabilities while maintaining ultralow latency.”