NZ government warns of algorithm bias
- 29 October, 2018 14:38
First report to scrutinise use of algorithms by government agencies
The New Zealand government has produced its first report into the use of algorithms by government agencies, saying there are few safeguards against biased algorithms, and there is ample scope for government agencies to lift their game.
The report follows one in May issued by the New Zealand Human Rights Commission warning that public sector use of algorithms for predictive purposes could lead to unfair treatment of individuals or groups. It called for steps to be taken to ensure such practices conformed to human rights and ethical standards.
The government said the report, by the government chief data steward and the government chief digital officer, provided valuable insights into the use of algorithms by government agencies, and suggested how their use could be improved for both fairness and transparency. It examined the use of algorithms in 14 government agencies.
The report said data bias posed a significant challenge for effective algorithm use, but there was little monitoring to detect any bias.
"Even the best algorithms can perpetuate historic inequality if biases in data are not understood and accounted for," it said.
"Only a minority of participating agencies described a formal process for considering the limitations of algorithms as a part of the development process for these tools.
"Few agencies reported any regular review process for existing algorithms to ensure they are achieving their intended aims without unintended or adverse effects.
"This suggests a risk that the limitations of algorithms, and the data they draw upon, may depend on the skills and experience of individuals in particular roles and, therefore, may not be systemically and consistently identified to decision-makers."
Recommendations in the report include maintaining human oversight, involving those who will be affected, promoting transparency and awareness, regularly reviewing algorithms that inform significant decisions, and monitoring for adverse effects.
Government chief data steward Liz MacPherson said: “New Zealand has robust systems and principles in place around the safe use of data, but as techniques become more sophisticated we must remember to keep the focus on people and make sure the things we are doing are for their benefit."
Government chief digital officer Paul James said algorithms were an important part of government and were "evolving to provide services that work better for all of us, and also make it easier for citizens to engage with government."
He said the report included case studies that highlight how algorithms are already enabling innovative solutions to complex problems.
"One example is an algorithm being used by Work and Income to identify young people at risk of long-term unemployment, so they can be offered assistance. This provides a great example of the way these techniques can help those who may be in need."