UK Home Office AI Tools Face Scrutiny Over Reliability Gaps

The UK Home Office is facing potential investigation by the Independent Chief Inspector of Borders and Immigration (ICIBI) over its deployment of generative AI to handle asylum applications, as Wired first reported. The watchdog is considering the probe following concerns regarding the accuracy of the tools and a lack of government transparency.

According to documents obtained by the nonprofit Open Rights Group (ORG), the Home Office utilizes two tools built on Microsoft Foundry and OpenAI’s 2023 GPT-4 model: Asylum Case Summarisation (ACS) and Asylum Policy Search (APS). Piloted in late 2024 to address a backlog of approximately 100,000 cases, the tools are designed to save caseworkers roughly one hour per case by summarizing interview transcripts and country policy information.

However, the operational reliability of these tools has been questioned. ORG found that the rate of ACS summaries deemed unfit for use due to misrepresentations or factual errors is significantly higher than the 9 percent error rate reported in pilot evaluations. Additionally, ORG found no quality control for the APS tool. In July, a UK judge accused the Home Office of using an "AI hallucinated" document to deny an asylum claim for a Moroccan woman and her child, Wired reports.

Critics, including Labour MP Bell Ribeiro-Addy and the Public Law Project, have raised concerns that cultural biases in GPT models could lead to discriminatory decisions. Furthermore, while the UK’s "AI Playbook" requires transparency for algorithmic tools, the ACS and APS are not listed in the public repository, and asylum seekers are not notified when the tools are used in their cases.

A Home Office spokesperson told Wired that AI tools only assist human decision-makers and do not automate decisions. An OpenAI spokesperson noted the company is working to improve accuracy and will retire GPT-4 in January.