AI Governance – why good governance is good business and why trust is key

Last week I was with Tom Whittaker, Burges Salmon's AI team lead, in Amsterdam at InventU’s inaugural AI Governance conference, which brought together some of the leading minds from the AI governance community. It was an immersive experience. Tom has already posted some insightful reflections on:

• Sovereign AI;
• Competing priorities in AI governance; and
• Scaling AI governance. 

Here is a super-summary of my own reflections:

• TRUST (I put this important word in CAPITAL LETTERS for a good reason):
  • Trust needs rules
  • Consumers need to trust your business
  • Your business relies on your reputation
• Customers:
  • What your customers care about really does matter
  • Do not allow a trust gap between your c-suite and your customers
  • Identify your customers and engage with them
  • Understand the lived experiences of your customers
• Values:
  • Your core values will define how you present yourself in the digital world
  • Decide what you wish to be known for
  • If you wish to be trusted for something, this is key to your strategy
  • Trusted core values are your competitive advantage
• Have a clear mission:
  • Have an AI strategy, the way forward needs to be clear
  • Make it customer (human) centric, your AI needs to exist with your humans
  • Speed without strategy is guaranteed chaos
• Govern with real accountability:
  • Responsibility is a hot potato, but it is real
  • You need real-time accountability, continuous oversight, and continuous improvement
  • Identify who is going to take ownership
  • Good governance is good business not a blocker
  • Culture is crucial but under-estimated
• AI cannot do governance:
  • There are AI tools that can assist you with your governance, but accountable humans must check, understand, be capable of explaining, and take full responsibility for the outcomes
• Mind the ‘middle management’ gap:
  • Your approach needs to be top down, and bottom up, and must not omit the middle on the way
  • Collaboration and teamwork throughout your business are key
• Collaboration:
  • Rally your talent
  • Find your critical thinkers
  • Gather all relevant skill sets and get them in the room, let nothing fall through the gap
  • The people in the room do not want each other’s jobs, they want to co-exist and collaborate
  • Collaborate in a way in which people can bridge knowledge gaps, no single skill set, no single person, can do this alone
  • You may already have existing talent in your data, legal, procurement, compliance, and privacy teams, you may already have good processes and systems, don’t reinvent the entire wheel for AI
• Find your influencers:
  • Change requires influence, your influencers will make things happen
  • Create ambassadors
  • Have a “Responsible AI team”
• Employment:
  • Humanoid robots are not science fiction, but they will not take all the jobs, re-skill and re-purpose your workforce
  • Everyone in your business needs to be AI literate in a role relevant way
  • Literacy will mitigate the risk of human error
  • Train, train, and keep training
• Data:
  • It is your data and not your AI that is the problem
  • Specifically, where AI touches your data, is your real issue
  • Issues around data are slowing deployment
  • AI will expose any issue in your data at speed and at scale
  • Data: sort it, classify it, sanitise it, put controls around it, make sure you know where it has come from
• Bias:
  • Historical human bias is embedded in your data
  • AI will amplify bias
• Be clear about your risk appetite:
  • The risk surface is massive
  • Identify your use cases, you do not need AI for everything
  • Sort your use cases into risk lanes – not every application will need the same treatment, some you can fast track, for others you will need to exercise greater caution
  • Knowing your data will help you isolate your risks
  • Some of your risks will be acceptable, others will not
  • The highest risks require greatest focus
• Agents:
  • Autonomous agents are good news
  • Autonomy needs trusted data
  • Agents must have clear rules of engagement, intervention and monitoring built in, and a clear owner with full visibility
  • The agent owner must be human, skilled, and informed
  • Every agent action should be transparent, reversible, and explainable
• We are only at the start of the journey:
  • We are all learning, every day, and we have a long way to go
  • Inflated expectations need to meet realism
  • Many AI projects have been abandoned
  • Many senior people have resigned under public scrutiny
  • The fail rate will increase as more AI deploys, and so will the ‘fall on your sword’ rate
• Kill switch:
  • Make sure you have one
• Test, monitor, train, evaluate, pilot……
  • The journey is continuous
  • AI will scale your gaps
• Tech is the easy part:
  • People, processes, and culture are hard
  • Change management is hard

Although my focus sector is financial services, there was important messaging from a healthcare collaborator (and I view the patient as being not so different, in many ways, from a consumer of financial services), that resonated deeply: 

• The patient is at the heart

This company’s desire to be known for its core values, and for its culture, made it standout. If I was in the position of being the patient, I would care about these things. Would you?

And an important message around ethics that connected me back to memories of a data protection experience that I had many years ago in my career:

• It may be legal
• It may be feasible
• But, should we?

AI is not all about speed. We can and should STOP and think about the outcome and ask ourselves, “Is this OK?”, “Is this right?”

You can read more updates like this by subscribing to our monthly financial services regulation update by clicking here, clicking here for our AI blog, and here for our AI newsletter. 

If you would like to discuss how current or future regulations impact what you do with AI, please contact me, Tom Whittaker, or Martin Cook. You can meet our financial services experts here and our AI experts here.