New Zealand MSD Predictive Risk Modelling

The predictive risk modelling tool was a statistical model commissioned by New Zealand's Ministry of Social Development. It would have scored a child, at or shortly after birth, for the likelihood of a substantiated maltreatment finding by age five. A substantiated finding is a decision by the child protection agency that maltreatment occurred.

What is decided

The tool was designed to give frontline social workers a score for each child, to weigh alongside their own assessment. Tim Dare's 2013 ethical review for the ministry records that the score was to supplement a full clinical assessment, not replace it.

The model computes the score from linked records. They are income-support benefit records and child protection records from Child, Youth and Family, the child protection service. Which children get scored is a policy choice, not the social worker's.

How the model was built and tested

The development work was led from the University of Auckland, with co-authors at Auckland University of Technology and the University of Southern California. It was published in 2013 in the American Journal of Preventive Medicine.

The developers used a 2012 sample of 57,986 children. They selected 132 inputs from 224 they had built. The area under the curve, a standard measure of how well a score ranks cases, was 76 percent. Among the tenth of children the model ranked highest, 47.8 percent had a substantiated finding by age five.

Those figures come from the development data. They were never tested in the field, because the study that would have tested them was the one the minister stopped.

The reviews before any use

The design went through an unusual set of reviews before any use. Tim Dare wrote an independent ethical review in 2013. He judged that the remaining concerns "may plausibly be regarded as outweighed by the very considerable potential benefits".

A dedicated Maori ethical review was also done. A Privacy Impact Assessment, a written review of a design's privacy risks, was discussed with the Office of the Privacy Commissioner. The sources do not say what the Maori review or the privacy assessment concluded.

The study that was stopped

In November 2014 the ministry lodged an ethics committee application for a two-year study. It would have scored about 60,000 newborns by linking maternity, benefit, and child protection records. Then it would have watched, without acting, whether the high-risk predictions came true.

The incoming Social Development Minister, Anne Tolley, withdrew the application. She wrote on the briefing papers: "Not on my watch! These are children not lab rats." The Children's Commissioner took the same view: "no ethics committee in New Zealand is going to wear that".

What happened next

Testing was narrowed to about 20 historic cases, with identifying details removed. The tool was never used in live casework. As of 2026, frontline workers at Oranga Tamariki, which the sources name as the later home of this work, still do not use predictive modelling.

What critics raised

The model's target, a substantiated finding, is itself an agency decision, not a measure of harm. Past findings were also among its inputs. So the patterns in earlier decisions would return in every future score.

The records were collected for other purposes, and the families did not agree to their use for scoring. Reviewers and critics also raised the concern that both the outcome and the inputs could carry existing bias against Maori and benefit-receiving families.

What the available tools can and cannot address

A failure pathway is a link between two parts of the network, where a mistake made by one part can be passed on to the other. A pathway is closed when mistakes stop passing along it. The work along it goes on.

This case has a budget of 9 units. Explore (No Targets) sets no targets. Under Service Targets Only, the targets are already met before you use any tool. That level asks that mistakes not build on one another, and that the model stay useful to the work.

Under Service and Safety Targets and All Governance Targets, the targets can also be met. Both levels ask you to close every failure pathway and refill the Privacy gauge, among other targets. Four pathways start open: the score given to social workers, workers recording decisions, the model reading the linked records, and workers reading a child's history.

The cheapest way costs 7 units: Escalate checks at 2, Store less data at 3, and Mark AI-written records at 2. Escalate checks is the one tool offered here that closes the pathway where the score is given to social workers.

One other way leaves out Mark AI-written records. It uses Escalate checks, Vet connections, and Store less data with Understand the system, for all 9 units. At these two levels Understand the system costs 4, and it takes 1 unit off each of the other three.

Counting stronger settings, eight distinct sets of tools meet the targets at each of these two levels. Every one includes Escalate checks and Store less data.

Require sign-off, the control that decided the real case, is not needed at any level. The network already includes the review step that held.

More is not better here. Every tool at its strongest setting at once costs 35 units, far over the budget. It also leaves the model no longer clearly helping the work, so it misses the targets at every level that sets them.