The future of risk management and surveillance for capital markets

Ali Curi: Markets conversation is a new ION podcast where we discuss topics of importance to capital market participants with product owners, subject matter experts, and industry leaders.

Enrico Melchioni: I will quote Steve Jobs, “Stay hungry, stay foolish.” He used to say, so I heard this aphorism very late in my life, but I think it’s a very good advice in general.

Ali Curi: Hi everyone and welcome to Markets ConversatION. I’m Ali Curi. On today’s episode, Enrico Melchioni from ION Markets will share valuable insight into how financial institutions can manage and mitigate risk in financial trades through model risk management for compliance trade surveillance. Financial institutions play a critical role in ensuring market integrity and protecting investors by monitoring trading activities.

They do this by employing powerful algorithms and machine learning techniques designed to identify suspicious patterns and anomalies in trading data. Through trading surveillance teams, financial institutions can detect potential market abuse, insider trading, and other regulatory violations. Enrico will help us understand the practices that model risk teams develop, and he will shed light on the regulatory landscape and discuss how technology continues to shape this ever-evolving field.

Let’s get started.

Enrico Melchione, welcome to the podcast.

Enrico Melchioni: Thank you, Ali. I’m very pleased to be with you today.

Ali Curi: Let’s begin with an overview of Model Risk Management or MRM, as it relates to compliance trade surveillance. What is the main objective of model risk management?

Enrico Melchioni: Sure. Let’s start from the beginning.

Models are used in many areas such as pricing, risk management and of course surveillance to estimate or predict values based on predefining inputs. In order to be safely used, a model should be properly tested and its inputs validated. Well-known saying in computer science goes, "Garbage in, garbage out."

So you want to be sure you provide the right data to the system. Another key point is that the model should be used in the context it was originally designed for. So let me make an example. Let’s assume that you know that the returns of a given portfolio has a normal distribution, and then based on this assumption, you’re developing a model to estimate the value at risk of the portfolio using algo that rely on the normality assumption.

Now your model goes into production, for a while, and then someone start trading on that portfolio. Some product, like derivatives, that do not respect the assumption you made. So the model will start providing wrong answers. Although there is no real bug, it’s just that it’s being used in the wrong way.

The above is a very simple example, but as model complexity increases, the system may become more difficult to test and validate in the first place. And the context of using the model may be less obvious. The aim of the MRM is to define a set of procedure to avoid the problem I just mentioned before.

So to define how you test the code and you validate the input and to continuously monitoring the model performances and the user context. This is with the aim of course, of reducing the risk of a model, providing wrong answers.

Ali Curi: And share with us how are these models used in trade surveillance activities within financial institutions?

What is their role in detecting potential compliance breaches, for example?

Enrico Melchioni: Okay, so trade surveillance models analyze trading transactions and public market data. To identify potential violation of surveillance regulations, including for instance, insider trading or market manipulation. So how does it work in practice?

Trader activity is analyzed by what we call detection agents, which are algorithms that compute specific metrics related to the abusive pattern you want detect. These metrics are compared with the thresholds that are set by, typically by the compliance officers and in case these thresholds are being exceeded, there is alerts are generated.

These alerts are then analyzed by the compliance officer and typically both manually, so looking at the data, but also relying on AI -based tools, that can help in their activity. Then in certain case the compliance officer may deem the alert an actual violation and then issue a report to the regulation, to the regulator.

So this is the high level overview of how the model works. There are a number of challenges in particular related to some new facts that emerge recently. So one thing is that trending activities moving toward the electronic markets, more and more. And this paved the ways for using automatic trading tools, which may generate a complex trading pattern, very difficult to analyze.

Also, following Covid, a number of report have been issued, for instance, by the Wall Street Journal and also by the Italian regulator, in 2022, which analyze new changes in the market microstructure induced by the increased number of retail trading on the market. So ideally what happened is that people were forced to stay at home and so one, one of the things that they did was trading online, changing, however, the way the market behaves.

Another challenge, another example of this challenge is the GameStop event in early in January 2021. So in that case this is what happens. GameStop is a software game company that is quoted on NYSE in a matter of days, the stock price rise from a few dollars to almost a hundred that increasing by a factor of 30 with extreme volatility.

And then this fact was analyzed and it turns out that there was a group of retail traders that coordinated through the reddit social website and generated a way of buy orders that actually manipulated the market. And so this is to give an example of the new kind of challenges that trading surveillance system needs to address.

So this inevitably brings an increase in the complexity of the surveillance algos. So you need, for instance, to correlate the activity of multiple traders across different market and asset classes. And also you need to have new holistic approaches. For instance, recently requests from our client related to add new metrics to the surveillance system that relate that, for instance, to the trader position changes which can be analyzed along with the results of the detection agents. What are the main risk which should be concerned by the model risk management? One is an important one has to do with the input. We will see that there are a number of complexity, linked to the completeness and quality of the input data.

And the second one, given the structure of the model that I just described has to do with the incorrect model, parameterization typically.

Ali Curi: Which leads us to the technology. How does cutting edge technologies such as artificial intelligence and machine learning, impact model risk management? What opportunities and challenges does it present?

Enrico Melchioni: First, let me mention technology, which is scalability of the systems. This is not cutting edge stuff, but still is very important in the overall data analysis process. In short, surveillance systems needs to analyze large amounts of data and the ability of using different model settings to and compare the results has a very important role in the validation step of the models. Then we have the input data part, so surveillance model results are very sensitive to the quality and completeness of the input data. And here, technology can help with the automated monitoring tool and also with machine learning-based anomaly detection systems. Also machine learning and artificial intelligence is being used during the detection phase. So, I can mention a recent activity published, again by the Italian regulator, by which they have been working on internal model that uses statistical validation, the networks and cluster analysis to detect insider trading performance by a group of trader. Quite a complex process overall.

Finally, machine learning can be used during the alert analysis phase to help the human analyst to classify the alert in a faster way. As we mentioned, AI model, I think it’s important to touch a point about the "model explainability."

AI often operate as a black box, without providing any human readable rationale to explain why a result was reached. This is a problem. And in fact, also, regulators started looking into this issue. For instance, the European community recently proposed, the Artificial Intelligence Act, which precisely addresses the use of artificial intelligence in modeling.

And more recently, in February this year, ESMA, the European regulator, issued a report on the use of AI by financial institution, and they rank the number one risk of the adoption of AI as the "explainability." And indeed what we see in speaking with our clients, this feature is a key requirement in trade surveillance system and there is a hot debate on ongoing whether we should use AI or not, and how to explain the result that they are providing.

ION Ad: This episode is brought to you by ION.

ION’s Fidessa Surveillance keeps up with trading surveillance regulations and meets the needs of the most demanding trading environments.

With Fidessa Surveillance, you can fine tune detection algorithms to your specific business model and accurately analyze trading and market data to generate alerts for potential violations.

Fidessa Surveillance also equips you with machine learning tools to speed up your alert triage workflow across a comprehensive and broad product coverage for consolidated trade surveillance monitoring.

To learn more, visit us at iongroup.com/markets or email us at: [email protected].

Ali Curi: Let’s talk about some of the regulatory issues. What are some key regulatory requirements and guidelines related to MRM and how do financial institutions ensure compliance with these regulations?

Enrico Melchioni: To my knowledge, to be honest there are currently no regulation that force threat surveillance system to follow a model risk management process.

This doesn’t mean that there is something ongoing. In fact, there, there are strong signals in industry that this may be the case in the short term. So in particular, there is a US Federal Reserve SR 7-11 guidance on market risk management, which is addressing this issue. And more recently, Bank of England published a consultation paper in June 2022, where they explicitly mentioned threat surveillance models and artificial intelligence use as being something to be monitored through a moderate risk management process.

Ali Curi: Without getting too much into the weeds, can you explain to us the process of model development and validation within this context of trade surveillance? So for example, what are the essential steps involved? What are the factors that financial institutions consider during model development and validation?

Enrico Melchioni: So we can schematically describe the model development process as follows. So we have a first step, which we define as the model design and development by which, during which you define the goals of the model, which are, in this case, in the case of surveillance, quite clearly defined by the regulation.

And in on top of that, there may be additional internal requirements from risk management or compliance. Another point which is quite important during this phase is the documentation. So it’s very important to have a very clear documentation on the input of the data that required, the applicability range of which detection and, how to use it, and very important detailed description on the, how the algorithms is working.

This is very important because quite often regulator performs audits on the surveillance system used by a specific financial institution. And one of the first thing that they ask for are, is the documentation on how the system is working and how this system fit the regulation requirement.

Of course, during this phase, the development phase it’s important to have a sound code testing and validation procedure in place. Once the model has been let’s say developed, we move to the deployment phase here we have also a very critical phase, which is the integration phase.

Surveillance system typically operate across different front office desks which typically use one or more front office system. So it’s important to have a very, to verify that this integration working fine and the, and that the system is fed with the proper data. Then there is the, let’s say the parameterization of the model.

We mentioned before that typically the surveillance system requires some thresholds. So it’s very important during the UAT phase that the compliance is reasonably sure that all potential violation will be detected by the model, by the way, they have configured it, at the same time is minimizing the number of generated hours, they don’t want to have unnecessary work, of course, in analyzing tons of useless data. Coming to the last step, which is the model validation. Clients typically, implement ongoing monitoring of the input data quality. Typically, they have system in place to check if they receive, all the data from all sources.

And also if there are anomalies in the volumes with respect to the past flows. Similar logic is applied to the monitoring of model results, the number of alerts generated and the statistics of the other types are checked against historical values. And last, but most, not least, probably one of the most important point is to perform a continuous parameter setting revision and back testing with the production data to make sure that current setting is actually the best possible setting for the market condition which are in place.

Ali Curi: What would you say are some of the typical challenges and risks associated with model implementation and maintenance in trade surveillance? And how can financial institutions effectively manage these risks?

Enrico Melchioni: In my opinion, the biggest risk associated with the trade surveillance is failing to detect an illicit behavior or a violation.

This has part reputational impact and a direct financial effect. Fines imposed by regulation because of breaches in the compliance surveillance may reach six figures ,so they may be quite high in general. So to mitigate this risk financial institution typically act on the monitoring phase on the input of data quality, typically using automated tools.

And the second part is the fine tuning of the model parameters. Typically what happens is that they tend to break down the trading activity into smaller classes, for instance separate retail from institutional trader and divide across asset classes. Just because, there are different trading style depending on who is trading and on the asset class.

Also maintenance is a key issue. In the last two, three years, client reported that they had to frequently adjust the model parameterization due to continuous crisis waves. So we had the Covid, we have the change induced in the market by retail trader. Then we had war in Ukraine, then now we have this inflation surge and who knows what’s next. So you have to keep monitoring the model results to, to catch changes and to act, to have the model always operating at its best.

Ali Curi: Enrico, can you share with us how financial institutions monitor and track the performance of trade surveillance models and what are the common practices and tools that are used to assess the effectiveness and the efficiency of that model?

Enrico Melchioni: There are, there is a range of monitoring tools that are commonly used by the industry, ranging from periodic reports and statistics on the input data and generated alert, to more sophisticated anomaly detection tools that make use of machine learning data algorithms. These tools may be embedded into the trading surveillance system or maybe part of external data warehouses which are fed by the trading, by the surveillance system on top of which they run additional tests. Among our clients back testing or "what if analysis" widely used to assess the model performance against changing market conditions.

Ali Curi: As we know, asking different departments to collaborate in any company-wide initiative can often be challenging. What role does collaboration between different stakeholders, such as compliance, risk management, and technology, how do these come into play in effective model risk management?

Enrico Melchioni: Setting up a trade surveillance system. And it’s monitoring is requires several skills as you may imagine. First of all, you need a very good knowledge on the regulation. The regulation that you need to comply with. These regulation are in general, quite complex.

And more importantly, they very frequently change, so you have to adapt to the change, to be aware of the change and adapt to the model and the system to it. You need to have a very clear view of the risk involved with into the, using the model or misusing it. You need to have a deep knowledge of the data ecosystem of the financial institution.

So complex abusive patterns require monitoring across different desks and asset classes. And this requires to know exactly where you have to get the data from. And of course you need advanced programming skills. You have to deal with large data sets, make use of the advanced statistical algorithms not to mention machine learning and artificial intelligence.

So collaboration across all stakeholder is I believe is of paramount importance to implement and maintain an effective trading surveillance system.

Ali Curi: And how about staying current in emerging trends and new developments? How can financial institutions stay up to date with evolving regulatory requirements and industry best practices? Like what are some emerging trends also developments that are shaping this field?

Enrico Melchioni: According to a recent entity data survey, only 22% of the interviewed banks are choosing to build their own technology for surveillance. More common is relying on independent vendors or on a hybrid " buy and build" solution.

And this is also what we have experienced with our clients. Typically we collaborate in hybrid cases and the reason is that vendors typically focus on their product and provide out of the box many of the MRM requirements, such as consolidated design and develop process, detailed documentation or monitoring and validation tools embedded into the product.

Also vendor are constantly looking at the regulation and evolving the product to stay on top of the changes that are being of service.

Also, there is one another important point which is very much appreciated by our clients. Vendor can effectively support Compliance Officer during authorities’ audits or inquiries, providing information about the software system and in general assisting the client.

Ali Curi: Enrico, what is the one big thing that you hope listeners will take away from this episode?

Enrico Melchioni: To summarize, let’s say that also not yet a regulatory requirement; setting up a moderate risk management procedure for trade surveillance, in my opinion has big benefits on the performance of the system.

There are, there will be less false, positive, lower risk of missing violation and better performance for the compliance team. I believe this largely compensates for the extra effort of setting up an moderate risk management process. And also, of course, relying on on a vendor such as ION can help a lot in simplifying the entire transition.

Ali Curi: Now let’s talk briefly about career advice. What is some advice you wish you had heard earlier in your career?

Enrico Melchioni: Great question. Quite difficult to answer though. Let’s say, as an escape route, I will quote Steve Jobs, "Stay hungry, stay foolish," he used to say. So I heard that this aphorism very late in my life, but I think it’s a very good advice in general.

Ali Curi: Enrico Melchione, thank you so much for joining us today. It’s been a pleasure having you on the podcast, and I hope you visit us again.

Enrico Melchioni: Thank you very much, Ali.

Ali Curi: And that’s our episode for today. You can follow ION Markets on Twitter and LinkedIn. Thank you for joining us. I’m Ali Curi, until next time.