Keeping an eye on AI in financial services

With a focus on the financial services sector, here is a summary of some of the latest AI-related news stories. Themes include:

• agents acting autonomously;
• the widening gap between deployment and fit-for-purpose governance;
• a requirement for collaboration to build bridges between relevant expertise;
• tough questions about explainability; and
• competing priorities.

At the end, a focus on some of the bigger questions that the financial services ecosystem needs to address to move forward. Are you grappling with these same questions?

Agents crossed a line: Agents went above and beyond assisting us to do better and go faster and decided (without being specifically trained) to link-up multistep cyber-attacks. With Mythos and GPT-5.4-Cyber (and at least one rival already out there), some powerful new frontier AI considered too dangerous to release (save into tightly closed, US-focused, code-named groups of Big Tech firms, Wall Street banks, central bankers, regulators, and security institutes), was let out of the laboratory. As the story goes, these models can detect weaknesses in all forms of software, digital infrastructure, and operating systems, at scale (of course, it would not be AI otherwise). 

This means that Password123, and all other legacy non-strong defences, are now completely passé (set up your passkeys now). And, zillions of chinks in the armour that have gone undetected by humankind for decades, are now floating on the surface like sitting targets. At human speed, it will take years to patch the vulnerabilities that have been exposed. Reports already suggest a surge in technical fixes designed to start the long task of plugging them. 

A new systemic risk: Agents that can independently tool-up and perform complex tasks autonomously present our financial ecosystem with a new kind of systemic risk. There have been numerous reports in the last few weeks of agents going rogue, with one story earlier this week describing how an agent had deleted a critical database and wiped the back-ups, all while running an otherwise routine business-as-usual task, and in a total time of nine seconds. Yes, ‘9’, ‘nine’, seconds.

The compressed time frames in which agents can act, significantly mount the pressure on humans to act in real time, and with agility. This demands us to make a quick shift to a new kind of modern governance and oversight that enables us to remain resilient to emerging high-speed threats. We can only achieve this by empowering ourselves with, and deploying to meaningful effect, the same level of technology.

These frontier models could turn anyone savvy enough into a champion grade hacker. Quite rightly, the markets and the regulators appear worried. Rightly so. It was questionable whether trying to contain these models in a sandbox (of sorts) would do anything at all to safeguard against use of them by bad actors. Indeed, it was not many days before it was reported that outside operatives, including third party contractors and security researchers who make it their business to gain information about and unauthorised access to unreleased models, were already on that track. 

The flip side, of course, is that, in the hands of the right people, these models deliver a game-changing defence capability that could bring fixing critical and high-priority vulnerabilities at speed into the established lines of defence and help humans with the daunting task of keeping up. I recently saw a demonstration of a (really very cool) regulatory risk monitoring model that helped me visualise what is now within the art of possible. I was able to ask all of my how, what, why, where, when questions and begin to grasp a bigger understanding of what is now possible if we augment ourselves with technology to transform how we oversee and manage regulatory risk. 

Competing priorities: There are currently many diverse approaches to the embracing of this technology. Many struggle to understand (as, reportedly, even do some of the experts who are creating it) exactly what we are trying to grapple with and wonder whether we will ever trust something that we do not understand. Others crack on at max-speed ahead without a map, without a torch, without a helmet, headlong into an area of critical and unidentifiable risk. It seems that everyone has a different version of, and that there may be significant tensions between, AI priorities and what good deployment might look like. Yet another area of more questions than answers…

Is your priority focus on efficiency? Is it effectiveness? Value? ROI? Or, something meaningful in the context of wider social purpose? Is it consumer experience? What are your metrics? How are you measuring your use of AI, what are your costs, and how well you are using it? Are all your employees using it? Are they fluent or becoming fluent in their use of it? Have you embedded it across and through your organisation? Are you augmented by it? What does value mean to you? Is AI just summarising lengthy chunks of text for you or is it redefining your vision and strategy? Have you been able to fully align your different business areas, including your owners and your compliance and legal teams, with your technologists? Do they all share an understanding of each other's language? Have you thought about what your customers really care about? 

The governance gap: Frameworks specifically designed for AI in financial services are now needed urgently.  We know that technology has already broken the gates and that we cannot sit back while the requirement to deliver answers to questions about explainability and human accountability are pressing.  It is imperative that we speed up our analysis of how to govern a technology that can make decisions and act on them before we have even noticed. This is the only way that we can empower or position ourselves to prevent what sometimes seems to be inevitable havoc. 

The advice gap: There is a clear need for technology-driven solutions to help us in places where, as humans alone, we have so far been unable to deliver good outcomes to legacy problems. Unresolved tensions between what is possible with new technology and the prevailing regulatory regime are not helping to reach timely good outcomes and, in the meantime, new problems are evolving. 

Whether AI might help to address the gap where millions of consumers, not within the affluent minority able to afford financial advice or financial guidance, is a question that persists. While the answer to it is debated, a new gap has formed into which many of those millions have started making use of freely available AI for financial help. This is a source of financial help that exists in a place of peril, outside the regulatory perimeter, where consumer protections may well not exist, and where no safety guardrails are in place. 

What is there to help these millions to use freely available AI for their financial journeys safely? To enable them to filter the regulated from the non-regulated, the wrong from the right, the appropriate from the inappropriate, or give them any ability to understand the lasting effect of financial decisions they make on the back of what AI has told them? The estimations are that around half of the UK population is actively using freely available AI for unregulated financial advice. Is that a problem or not? Is it a better consumer outcome for these millions because, where they previously had no access to financial information, they do at least now have access to some financial information? Does this outcome assist financial inclusion? Or is it a problem for the regulated sector to address, for the greater good, and plug with a safer, more trusted, alternative that can compete with freely available AI?

Our human ability to deliver the answers to questions like these resides in the next iteration of trustworthiness, explainability, transparency, accountability, and suitable standards, where we may have to accept that agents will be our next-generation collaborators, including in the field of governance itself. 

Could we or should we: A growing minority is making noise around the environmental, cultural, and human costs of AI. This gives another lens on value, and on the need for thoughtful, sustainable, and responsible decision-making. At the centre of it lies the importance of choice and the ability of leaders to exercise their judgement not to deploy certain AI models in certain situations. The reasons behind such a choice could be that a model’s actions cannot be explained and that this presents an unquantifiable regulatory risk. It could be an acknowledgement of significant legal duties and responsibilities that are not to be outsourced or delegated. It could be that there are concerns around potentially significant social or environmental costs. These costs are all currently challenging to quantify but merit deeper and comprehensive understanding.

Watch this space: Here are some things coming down the line that may be of interest:

• Working partnerships are developing between key financial services players and the leading universities;
• The FCA’s AI Live Testing has extended to a second cohort which includes a mixture of household names, tech firms, and newcomers;
• Open finance and smart data initiatives seek to advance a vision where data availability has been fundamentally changed, where fragmentation and siloes are a thing of the past, and where all financial data is held securely in a new form of intelligent infrastructure for an ecosystem of interconnected agentic systems to run across;
• The Bank of England is investigating the capabilities of agents with a focus on how they might affect trading by herding or correlated behaviours;
• The Treasury Committee maintains laser focus on why the CTP Regime remains unpopulated by any major AI or cloud providers;
• The University of Cambridge Judge Business School in collaboration with Cambridge Centre for Alternative Finance is due to publish its AI in Financial Services 2030 Global Survey;
• The FCA is due to publish its output from The Mills Review which, five or six months on, looks significantly less futuristic than may have been intended at the time;
• The FCA is scaling up its Supercharged Sandbox to enable more fintechs with access to data, compute and regulatory support, and a new Scale-Up Unit (opened recently as part of Fintech Week) to boost the UK’s fintech ecosystem;
• The FCA also promises an AI-focused good and poor practice report for later this year and an AI evaluation report early next year; and
• A Sovereign AI Unit has been launched to encourage strategically important start-ups to remain here in the UK.

Some big questions: My current focus is on getting my hands dirty with various small industry-led groups to build agile controls capable of aligning AI deployment with regulatory requirements. Hopefully, these initiatives will grow from small to significant and help to develop a widely applicable and fundamentally rewired approach to accountability, trust, and guardrails, re-casting these concepts for the reality of what is now, and ensuring that they are adaptable to a fast-evolving future. 

Devising a principles-led, outcomes-focused regulatory approach that translates the government-driven growth plans that demand AI to be safe, secure and robust, transparent and explainable, fair, well governed and subject to appropriate accountability, and open to challenge by those who suffer harm by it, into real world financial services is going to be no small challenge. 

Give me a shout if you want to compare notes. You may be doing something similar and wish to collaborate. There are small initiatives like this happening all over the country. What is currently going on within disparate small groups should probably not remain there for too long. If it wishes to have regulatory clarity, the entire financial services ecosystem will need to embrace collaboration, with the aim of tackling head-on the tensions that exist around a governance framework that was not designed for AI technology and its inherent characteristics. 

Please reach out if you have figured out how to:

• design, build and deploy, financial services fit guardrails for agentic AI;
• bridge the gaps in language and understanding between those who know the rules and those who know the tech;
• fill the gap between AI adoption and AI oversight;
• explain something that you cannot understand to the degree expected by the regulators;
• decide who is responsible and pin that firmly on them;
• manage the organisational changes demanded by AI;
• embed governance in deployment; and
• keep it human.

Our thought leadership:

You can subscribe to our monthly financial services regulation update by clicking here, clicking here for our AI blog, and here for our AI newsletter. You can meet our financial services experts here and our AI experts here.