Human Rights Commission warns on discrimination by algorithm
- 11 May, 2018 13:21
The New Zealand Human Rights Commission has warned that public sector use of algorithms for predictive purposes could lead to unfair treatment of individuals or groups and says steps should be taken to ensure such practices conform with human rights and ethical standards.
This is one of a number of warnings contained in a new report from the Commission: Privacy, Data and Technology: Human Rights Challenges in the Digital Age that provides an overview of the domestic and international human rights principles applying in New Zealand.
According to the report, the application of algorithmic risk assessments to big data is seen as underpinning the implementation and operation of the ‘social investment model’ in which ‘citizen-based analytics’ are used to provide empirical support for social policy (including funding and service delivery).
It cites the proposed predictive risk modelling programme for the child protection and welfare sector as an example of a specific policy initiative that uses algorithmic techniques to identify risk and target service interventions accordingly.
The report says the UN Special Rapporteur on the right to privacy has highlighted a number of well documented risks associated with algorithmic techniques, including that they: are value laden, may be based on imperfect or selective data, may be used for profiling, and are opaque and unaccountable.
“Operational and procedural safeguards in the social sector are an essential bulwark against the risk of human rights breaches occurring and becoming normalised in relation to personal data,” the report says.
“The New Zealand Government’s proposed Privacy, Human Rights and Ethics Framework therefore will be a critical front end procedural safeguard for ensuring that the deployment of algorithmic big data techniques in the social sector conform with human rights obligations,” the report says.
The Ministry of Social Development is developing a privacy, human rights and ethics framework that will apply to predictive risk modelling initiatives in the social sector (specifically the child protection and social security sectors), the report says.
It suggests this framework could be expanded to apply more generally to all information sharing initiatives within the social sector.
Computerworld reported in February that The New Zealand Government is to lead a working group on digital rights, set up by D7, a group of five nations – New Zealand, United Kingdom, South Korea, Estonia and Israel – considered amongst the most advanced in the provision of online government services.