The University of Otago has raised concerns about what it says is the increasing use of AI based predictive analytics by government departments and has called for the creation of a body to regulate such practices.
The call has been made by members of the University’s Artificial Intelligence and Law in New Zealand Project (AILNZP) — which is funded by the NZ Law Foundation — in an article, Proposed Oversight of Predictive Analytics published on the current affairs and culture blog hub Pundit.
The AILNZP suggests such an agency would publish a complete list of the predictive tools used by government departments and other public institutions such as Accident Compensation Commission.
For each system it would also supply some basic information about its design; which variables constitute its input and output, and which techniques are used to learn mapping from inputs to outputs.
In addition, it would answer some of the key questions about the performance of the system, and about its use.
In the article they say: “Last year, there was a minor furore when it emerged that ACC uses a predictive tool to profile its clients.
"Three years ago, there was a larger controversy around a study proposed by the Ministry of Social Development to help build a tool for predicting children at risk of abuse.”
Professor Colin Gavaghan of Otago’s Faculty of Law, and director of the Centre for Law and Policy in Emerging Technologies, said public information about predictive analytics in government was coming mainly from media exposés’.
“In the latest case with Immigration New Zealand, even the immigration minister himself was in the dark about the system,” Gavaghan said. “These systems can be of great use, but there must be more transparency about how predictive systems are being used in government.”
Even where there was information about government predictive AI applications, AILNZP says it is incomplete.
AILNZP member associate professor Alistair Knott, of Otago’s Department of Computer Science, said: “We’re told, for instance, that the Immigration NZ tool is ‘just an Excel spreadsheet’.
"But many algorithms can be run in Excel: what algorithm is being run in this case? On what data? With what results? And what margin of error?"