A cautionary tale of using AI in law; UK case finds that AI generated fake case law citations
18 Dec 2023
3 min read
The First-Tier Tax Tribunal in England ('FTT') has found as a fact that a respondent cited cased law that did not exist and instead ‘have been generated by an AI system such as ChatGPT’.
In summary:
- Mrs Harber disposed of a property and failed notify her liability to capital gains tax. HMRC issued her with a ‘failure to notify' penalty. Mrs Harber appealed in the First tier Tax Tribunal and cited the names, dates and summaries of nine FTT decisions in support.
- The FTT and, separately, HMRC had been unable to find those cases on the FTT website. The cases were plausible but incorrect. A couple of cases cited had similarities to real cases but were ones in which the apellants had lost their case (so did not support Mrs Harber). American spellings were used for cases apparently in the English courts, and certain phrases were repeated across cases.
- Mrs Harber said that the cases had been provided to her by ‘a friend in a solicitor’s office' and acknowledged that they could have been generated by AI. Having asked how the FTT could be confident that the cases HMRC relied upon were genuine, the FTT pointed out that the full text of each had been provided and the judgments were on both the FTT site and BAILLI (an online database for some British and Irish case law).
- The case had similarity with the US case of Mata v Avianca 22-cv-1461(PKC), in which two barristers sought to rely on fake cases generated by ChatGPT. The FTT agreed with the view of Judge Castel in that case about the seriousness of citing fake judgments:
"Many harms flow from the submission of fake [judgments]. The opposing party wastes time and money in exposing the deception. The Court's time is taken from other important endeavors. The client may be deprived of arguments based on authentic judicial precedents. There is potential harm to the reputation of judges and courts whose names are falsely invoked as authors of the bogus opinions and to the reputation of a party attributed with fictional conduct. It promotes cynicism about the legal profession and the…judicial system. And a future litigant may be tempted to defy a judicial ruling by disingenuously claiming doubt about its authenticity."
The risks of using AI in law are known and under the spotlight. The Solicitors' Regulation Authority ("SRA") recently said this about results obtained from AI systems:
"All computers can make mistakes. AI language models such as ChatGPT, however, can be more prone to this. That is because they work by anticipating the text that should follow the input they are given, but do not have a concept of 'reality'. The result is known as 'hallucination', where a system produces highly plausible but incorrect results."
The case emphasises the need for caution and diligence on output from AI systems, esppecially when used in higher-risk contexts such as legal proceedings.
If you would like to discuss how current or future regulations impact what you do with AI, please contact Tom Whittaker, Brian Wong, David Varney, Lucy Pegler, Martin Cook or any other member in our Technology team.
The case is Harber v Commissioners for His Majesty’s Revenue and Customs [2023] UKFTT 1007 (TC) (available here on BAILII). A detailed write up of the case is here:
In a written document ("the Response") Mrs Harber provided the Tribunal with the names, dates and summaries of nine First-tier Tribunal ("FTT") decisions in which the appellant had been successful in showing that a reasonable excuse existed. However, none of those authorities were genuine; they had instead been generated by artificial intelligence ("AI"). ... We accepted that Mrs Harber had been unaware that the AI cases were not genuine and that she did not know how to check their validity by using the FTT website or other legal websites.