Real-Time Database Technology you can bank on

Advances in Real-time Database technology are driving faster order throughput rates. This blog looks at how the application of lock-free programming technology has dramatically improved Fidessa’s performance.

Fidessa has been using fast in-memory database tables and indexes with lock-free reads in our database technology for the last 4 years.

Transaction performance has become more critical for our clients as their trading volumes have considerably increased over the last 18 months. Simultaneously, the complexity of the compliance and risk checks required to conduct that business has become a lot more challenging. Entering an order in our OMS now involves checking hundreds of database lookups across many multi-million record database tables.

In late 2019, we looked at whether we should add optimistic lock-free write technology to our database transactions. We couldn’t be sure of the impact of this enhancement when we began this development, or whether we could really achieve this change. Everyone familiar with multi-process, multithreaded lock-free programming knows this technology is complex and can introduce very subtle bugs that are hard to find. However, we still decided to take the plunge and implement this technology. It took us months of effort to get it right, and we rigorously tested it through 2020. In May 2021, we started rolling out our latest infrastructure with lock-free writes. All our testing paid off as we have had no production issues across billions-of-dollars of trading. We now have many trading platforms integrated with this technology.

Before this change, the improvement we saw from adding threads to transaction processing tasks, started to diminish after 5 threads because of database lock contention. Each thread

would start to interfere with the others. In production, we now see near linear improvement through 20 simultaneous threads which has greatly increases the throughput capacity of our trading hubs. This capability increase has proved timely as many of our clients are processing record order volumes.

We are now looking at offering Fidessa services in public cloud. Public cloud software still runs on an X86 or an ARM chip with caches and on-board memory. The speed of light is of course the same in public cloud as private cloud. The latency of processing complex transactions is still limited by the large number of database look ups needed to check all the validation, compliance, and risk. The throughput of order flow is governed by the number of transactions the system can simultaneously process. The Fidessa CASE real-time database technology with its scaling capabilities across one server and the ability to shard across multiple servers is working well in our public cloud testing. Public cloud will offer interesting new opportunities to make Fidessa scaling capability elastic for all our customers.

Front-Office technology is increasingly complex and the underlying technology underpinning it really does make a difference now and in the future. The Fidessa CASE real-time database technology is a key asset that we must utilize, and continue to improve for Fidessa services and across ION Markets whenever we want to do something complex, fast, with high throughput, and in a resilient way.