Keeping an eye on AI: the future of retail financial services

“AI and the future of retail financial services” or “The Mills Review” has just landed. Hot off the press today. Here is a first blush overview, based on an initial read, with more detailed analysis to follow over the next few days.

The most important person in the room is….

“Citizens” or “consumers” are front and centre of the Review.  Flagged in the Foreword is the opportunity for AI to end legacy “asymmetries and frictions” and support the goal of ensuring that consumers are able to make good financial decisions. 

The Review states that AI has the potential to address the advice gap, the protection gap, financial exclusion, and “suboptimal saving” by enabling consumers to “make better decisions, access more suitable products and manage their finances more effectively”.

Pace

AI is moving at an unprecedented rate. The vision of autonomous agents ("effectively absent….two years ago") acting on behalf of consumers is likely to be just around the corner. 

No new rules

The Review does not point to any desire on the part of the FCA to write new rules. The focus is on doubling-down on the existing outcomes-based approach with Consumer Duty, SMCR, and operational resilience frameworks remaining at the forefront. These are parts of the regulatory framework that are specifically noted as having been “designed to flex across changing business models”. 

Autonomy

It is not yet clear how the move towards autonomous agents will fit with the current regulatory architecture, save that the Review acknowledges that “full autonomy seems unlikely to sit well with the current regulatory framework or UK societal appetites”. That said, financial services that are “AI-enabled, continuous and delegated” are clearly in sight, with the shift for now only “tempered by the constraints of model performance and the continuing need for human oversight”. 

The Review sets out an “autonomy spectrum” which addresses the evolving role of humans in relation to agentic capability and envisages humans moving from one end of the spectrum, performing tasks themselves, to the other end, where they have oversight of outcomes produced by agents on their behalf. 

System-wide risks

Agentic AI changes the risk landscape, moving the focus from the risks that sit within individual firms, to a series of interconnected risks that percolate across the entire ecosystem. This change demands a move to system-wide regulatory monitoring with a focus on the shared reliances and connected dependencies that could impact entire markets. This requires the regulators to be empowered with “AI-enabled supervisory capability to identify cross-firm patterns, emerging harms and system-wide risks that no individual firm can see”.

Systemic shifts 

The FCA envisages four shifts that will redefine financial services over the next few years:

• AI will transform firms: Firms will increasingly utilise autonomous agents and their people will transform into “observers who monitor outcomes and step in when systems move outside agreed parameters”. This will require organisational changes, new skills, and a revised model of human oversight. 

  In terms of workforce changes, the Review states that firms will “need people with different skills: people who can understand AI-enabled journeys, risk and compliance specialists who can challenge AI-enabled workflows, supervisors who can interpret system behaviour, and senior managers who can evidence reasonable steps in automated environments”. 

  Oversight and accountability will be critically important and firms will need to be clear about what their people are “expected to do, what information they receive, when they can intervene, how challenge is recorded and how escalation works”. Governance, specifically tailored to AI, will become a ”core capability" and will have to adapt to “operate continuously alongside the systems it is designed to control”. 

  With distinct echoes of the car brakes analogy that the FCA have often been heard using, the Review states that those firms that “can demonstrate auditability, explainability where needed, robust testing, clear permissions, effective monitoring and escalation will be able to deploy AI more confidently”. The Review mandates these safeguards as critical for the deployment of AI in a regulated environment.

• Consumer journeys will become agent-led: Consumers are predicted to increasingly delegate to trusted AI agents that act within agreed boundaries and provide non-stop financial management, helping them to “achieve more while doing less”. Consumer uptake is predicted to be determined by “useful and reliable AI services, trust and control” with a need for consumers to “be able to oversee, understand and challenge AI-driven decisions, especially when things go wrong”.

  The Review is explicit about the commercial advantage that lies in consumer trust, stating that “acquiring a reputation for trusted AI processes could win business”. 

  The Review is underpinned by a degree of recent consumer research. It acknowledges that some consumers are already reaching out to openly available AI for various kinds of products and services. And that some consumers are already connecting agents into their personal finances, permitting them to execute trading strategies and follow instructions to spend money on their behalf. The Review also acknowledges that some consumers have concerns about data misuse, lack of protection if things go wrong, and the power of large firms over their finances. Overall, the ratio of consumers that see the advantages of AI is roughly fifty-fifty, with around half of consumers seeing “no benefit of AI in financial services”. 

  While the consumers engaged with in the FCA's research suggest fairly deeply divided “attitudinal cohorts”, the basic need for consumers to trust in AI before it can develop at scale, is clearly underscored by the Review. It states that “[t]rust will depend on consumers being able to understand decisions, challenge outcomes and access redress. Firms will need auditable records, clear complaints routes and enough understanding of AI outputs to explain decisions and mitigate harm” and concludes that “the regulatory response should focus on trusted adoption - enabling AI to improve access, capability and outcomes while preserving consumer control, inclusion, contestability and redress”. 

• Power and competition will be reshaped: The Review highlights a number of forces (costs, innovation, data, trust, bargaining power etc.) that could impact competition in the markets and redefine the regulatory perimeter. 

  The Review points to the significance of the “AI-mediated customer interface” as a potentially “major source of market power” and one that could shift the customer relationship away from financial services providers. AI-mediated competition is likely to “coexist with more traditional forms of competition”, such as consumer preference for human interaction, at least while consumer demand for autonomous AI is nascent. 

  The Review highlights two other significant drivers of competition: competition in the upstream supply chain and AI capability between rival financial services firms.   

  The Review notes the likelihood of an increasingly interconnected “agentic finance infrastructure that enables AI systems to act across firms and markets” and the need for the management of these dependencies to become a foundational part both of firm management and, given the wider implications of concentration risks and supply chain issues, the coordination and oversight of the entire financial services ecosystem. 

• Threats and defences must evolve at the same speed: The Review mandates that the ecosystem's “[d]efensive, supervisory and enforcement capability” must evolve just as fast as the fraud and cyber risks, which in the hands of attackers, are becoming “faster, cheaper, more scalable and more persuasive – and at the same time harder to spot and stop”. 

  The Review notes that “AI-powered identity fraud and scams" are “among the most significant global threats to financial crime prevention” and that AI ”lowers the cost of reconnaissance, personalisation and iteration for criminals, while increasing the burden on firms to distinguish genuine customers from convincing deception". The Review predicts that the evolving fraud risk, including the speed at which criminals can operate, demands a need not only for strengthened controls at the firm level but also increased “system-level coordination” because “no single organisation has the visibility, data or operational reach to act alone”. Of necessity, this will require enhanced foundational components such as common data standards and clear information-sharing gateways that will enable the defence response mechanisms to evolve to act at machine speed.  

  The Review refers to the most recent frontier model cyber risks as “a signal” and reiterates the need for firms that have “under-invested in fundamental cybersecurity” to act on this as an immediate priority.

Regulatory implications

The foundations of the regulatory framework, SMCR, the Consumer Duty, operational resilience requirements and the conduct framework, are recognised as being both challenged and complex in their practical application as firms move along the agentic autonomy spectrum and as the human role changes along that spectrum.  The Review states that these “developments do not undermine the framework itself but place increasing pressure on how it operates in practice”. The Review highlights five key challenges to the regulatory regime:

• SMCR: At lower levels of agent autonomy, it may be relatively simple to ensure human oversight of AI-assisted tasks, but as the level of autonomy augments, the demands placed on human oversight to challenge the AI output and exercise meaningful control become more challenging and this eventually reaches a pressure point where accountability becomes difficult to exercise and evidence.
• Consumer Duty: At the lower levels of the autonomy spectrum, good outcomes can be understood and evidenced through existing approaches, but at higher levels of autonomy (where journeys may become hyper-personalised, dynamic and delegated) it will become challenging to evidence consumer understanding, there may be concerns about fair value and suitability, and good outcomes may be difficult to evidence.
• Operational resilience: Shared dependencies and concentration risks increase as more firms rely upon the same infrastructure and become less identifiable at the level of the individual firm, common points of failure emerge and disruptions may impact several firms simultaneously, coordinated system-wide responses may be needed but could be challenging to deploy at speed.
• Perimeter: AI moves from being a source of “information” and begins to have an impact and influence on consumer journeys. Inevitably, this will blur the boundaries between what appears to be regulated and what is not, and the perimeter may no longer function in practice. This could have potentially serious implications for consumer protection (who have a new route to the “market” but may not understand the implications) and create an unlevel playing field for regulated firms and technology providers.
• Advice, guidance and targeted support: The boundaries between these activities will blur and AI outputs or “recommendations” may sit outside the regulatory perimeter, the boundaries of which no longer align with how decisions are made.

Seven priority recommendations

The Review makes “seven priority recommendations…..to adapt the regulatory framework, enable effective supervision, and support better consumer outcomes” highlighting the need for the regulator to move down the autonomy spectrum to an AI agentic supervisory model that delivers “dynamic, system-wide, near continuous real-time monitoring and more proactive intervention, supervisory engagement and enforcement”. The transition to this agentic supervisory model will involve building on and adaptation of the regulatory foundations, enhancing the regulator's firm-based supervision, and building capability for new risks and system-wide oversight. This is essential to enabling “safe, trusted and innovative AI adoption by firms and consumers, while mitigating emerging risks”. The seven recommendations cover four broad areas aimed at balancing the risks and opportunities presented by AI for the financial services industry:

1. Regulatory frameworks and the perimeter;
2. Supervision and coordination;
3. Foundations and capability; and
4. Consumer access and outcomes.

The seven recommendations are designed to be “mutually reinforcing” and are these (re-ordered to reflect the four areas noted above):

Regulatory frameworks and the perimeter

• Secure and adapt the regulatory perimeter: Launch a review into “the scale, nature and impact of general purpose LLMs outside the perimeter” and ensure that the perimeter remains robust and reflective of how financial outcomes are influenced and how financial services are delivered, closing the emerging "gap between consumer expectations and regulatory reality".  This will require ongoing monitoring, continuous market engagement, targeted intervention, enhanced cross-jurisdictional and cross-regulator collaboration, and potentially new regulatory powers.
• Monitor the transition to autonomous models and adapt the regulatory frameworks: Monitor the progress of the transition to greater autonomy, determine what requires closer supervisory attention, and clarify how accountability, governance, and consumer protection continue to apply and can be evidenced, as AI becomes more autonomous, adapting the regulatory framework to ensure that it remains effective. 

Supervision and coordination

• Strengthen system-wide coordination and oversight: Strengthen oversight of system-wide risks, dependencies and emerging issues by collaborating with “domestic authorities and international partners” in relation to financial services-centred risks; fraud, financial crime and cyber risks; and consumer protection, data and competition risks.
• Build and adopt an AI-enabled agentic supervisory model: Enhance firm-level supervision and system-wide oversight by developing AI-enhanced supervisory capabilities. In the agentic world, effective supervision will need to be “more continuous, system-wide oversight so that harm can be detected and acted upon more quickly”. This will involve deploying AI-enabled regulatory tools capable of supporting end-to-end regulation, including “triaging firm submissions, testing evidence against supervisory expectations, generating information requests, maintaining issue logs and supporting earlier intervention” to augment human-led, expert, supervisory judgement.  

Foundations and capability 

• Scale up the FCA’s AI LAB to support AI model and system innovation in financial services: Build capabilities “anchored” in the AI LAB to assess, understand, and evaluate both AI in current use in financial services and emerging AI. The intentions behind the LAB include cross-expertise collaboration, learning, testing and assessment, and are aimed at positioning the FCA to anticipate and respond to developments in capabilities that could impact financial services, and retain the UK's position as a leader in the field of AI “compliance, assurance and innovation”.
• Enable the foundations for agentic finance: Enable the trusted foundations required for agentic finance, including “consent mandates, identity, control and liability, creating the conditions for safe adoption of more autonomous AI-enabled services”. An effective unlocking of the advantages of agentic finance depends upon trust being placed in agents to “operate safely in financial services” and the removal of current constraints including those relating to fragmented data, identity verification, and infrastructure. The Review recommends that a trusted agent framework is evolved, based on existing regulatory foundations, to build on the FCA's work on the Open Finance initiative. 

Consumer access and outcomes

• Develop a trusted public-interest AI-enabled financial capability service: To support better outcomes, the FCA should develop a free and inclusively designed, AI-enabled agentic supervisory model to enhance what is currently human-led and enable better outcomes, greater trust and increased inclusion by supporting inclusive access to trusted AI-enabled guidance, capability and support. The Review notes that in order to return the advantages that AI is capable of delivering, the risks that these advantages are unevenly delivered will need to be addressed to ensure that those consumers who could benefit the most do not face the greatest barriers due to costs, information, or other hurdles.  

More coming soon…..

We are expecting the Bank of England's Financial Policy Committee to publish its updated assessment tomorrow, 7 July.  We are also expecting, from the FCA, output from the second cohort of the Live Testing initiative and a good and bad practice report specific to the use of AI in financial services.

Connecting with our thought leadership

As noted above, more focused breakdowns of the Review will follow in the coming days. 

You can read more updates like this by subscribing to our monthly financial services regulation update by clicking here, clicking here for our AI blog, and here for our AI newsletter. 

If you would like to discuss how current or future regulations impact what you do with AI, please contact me, Tom Whittaker, or Martin Cook. You can meet our financial services experts here and our AI experts here.

The FCA Board asked me to conduct this Review into how advances in AI could transform retail financial services by 2030 and beyond. It has been an honour. We drew on 140 written submissions and a wide range of expertise through meetings, panels and roundtables across financial services, technology, academia, consumer groups and regulation both in the UK and abroad. I am immensely grateful to everyone who contributed and gave up their time, and especially to the team and FCA colleagues who supported me..... ....What emerges is set out in the executive summary, which includes our recommendations. Taken as a whole, I believe the Review offers a practical agenda for the responsible development of AI in retail financial services which enables our world-leading industry to continue to grow and innovate, and meet the needs of all consumers. (Sheldon Mills)

https://www.fca.org.uk/publication/corporate/the-mills-review.pdf