Oregon Safety at Screening

The Safety at Screening Tool was software that Oregon's Department of Human Services built in house through its analytics office. For each child named in a report of abuse or neglect, it estimated two risks from the state's own child-welfare records. Hotline screeners saw the scores after entering a report and before deciding whether to investigate.

What the scores estimate

One score estimated the chance a child would be removed from home within two years if the report was assigned for investigation. The other applied if the report was screened out, meaning closed without an investigation. It estimated the chance the child would later be named in a new report that was assigned for investigation.

Two separate models produced them. One was trained on past reports that were investigated, and one on past reports that were not. The agency split them because the two groups of children can go on to very different experiences.

Each probability was shown as a tier from 1 to 4. Tier 1 covered the lowest tenth of probabilities, and tier 4 the top 5 percent. The screen also showed a report score, the highest score among the children on the report. Children already in substitute care, such as foster care, were not scored. Reports about these children made up 4 percent of incoming reports.

What the scores are built from

Both models used only the agency's own records in OR-KIDS, Oregon's child-welfare case system, online since August 2011. No call text or voice was used. The models learned from hundreds of thousands of past reports.

The agency's report lists more than 180 variables. They cover past reports, investigations, and foster care, the current report's details, the alleged perpetrator, and who made the report. The list includes the child's race or ethnicity category.

The agency's view was that other data can stand in for race, so hiding it would not remove bias. It chose to address bias directly instead.

The fairness correction

After the models ran, Oregon applied a fairness correction. It set different tier cut-offs for different race and ethnicity groups. The goal was error rate balance: similar rates of two errors across groups.

One error rates a child high risk when the predicted event, a removal or a new investigated report, does not happen. The other rates a child low risk when it does happen. The agency's 2019 report says the correction raised its error rate balance measure from 0.43 to 0.76 on past data. A measure of 1 would mean identical error rates across groups. Accuracy of the removal model fell from 81 to 79 percent.

The sources read do not report the correction's effect once the tool was in use.

Who decides

Hotline screeners are trained state employees who take reports of abuse or neglect, usually by phone. They decide whether to screen a report out or assign it for investigation by Child Protective Services.

They saw the scores only after entering the report, so the scores would not shape what they gathered. They affirmed they had reviewed the scores, then decided. The decision stayed theirs.

The agency framed the scores as historical indicators, not answers. It used four tiers so screeners would not set their own cut-offs. It also planned to track data entry, to check that it stayed consistent with past patterns. One concern it named was screeners changing entries to get a desired score.

Why the agency built it

In 2018, screeners assigned 51 percent of 85,974 reports for investigation, up from 43 percent of 67,466 reports in 2012. The agency's 2019 report says that volume strained Child Protective Services.

It also found that, over two years, 13 percent of assigned reports led to a child's removal from home. So did 8 percent of reports that were screened out.

The tool was derived from the Allegheny Family Screening Tool, a screening tool used in Allegheny County, Pennsylvania. Oregon narrowed its version to the agency's own child-welfare records.

How it ended

The tool was first used in 2018. On 19 May 2022, deputy director Lacey Andresen emailed staff. After what the agency called “extensive analysis”, hotline workers would stop using the tool at the end of June. “We are committed to continuous quality improvement and equity,” she wrote. The Associated Press reported the aim was to reduce disparities in which families are investigated.

The agency replaced it with Structured Decision Making, a screening process with no algorithm. Spokesman Jake Sunderland said the tool's score, built from aggregate data, could not be used in that family-specific process.

What each side says

The decision came weeks after Associated Press reporting on the Allegheny tool. That reporting cited a Carnegie Mellon study. It found that if the Allegheny tool had decided on its own, with call volumes held comparable, it would have recommended investigating about two-thirds of Black children. For all other children, it was about half.

U.S. Senator Ron Wyden of Oregon then asked the agency again about racial bias. He said such decisions are “far too important a task to give untested algorithms.”

Sunderland told Willamette Week the decision was unrelated to the coverage. He said the agency chose in 2021 to replace its screening process. Oregon's own tool was never publicly shown to reproduce the Allegheny disparities.

What this case shows

The score is computed from records that earlier screening decisions helped write. The fairness correction was built to offset the bias in those records, not to cut that loop.

The agency also held, and used, the authority to stop the tool. The case file calls this a rare case: the agency stopped its own tool before a documented crisis forced it. The most heavily documented cases, it notes, ended in courts or commissions. It also notes that the two stated reasons, equity and a poor fit with the new process, sit in tension.

What the available tools can and cannot address

A failure pathway is a link between two parts of the network where a mistake made by one part can be passed on to the other. A tool closes a pathway when mistakes stop passing along it. The work along it goes on.

This case has a budget of 9 units. Explore (No Targets) sets no targets. There, the cheapest single tools that keep mistakes from building on one another across the network are Escalate checks and Mark AI-written records, at 2 units each. Several others do it alone at a higher cost.

Under Service Targets Only, each of them also meets the service target. That target asks that the score stay helpful to the screeners' work. No combination that includes Pause AI on alarms meets it, because halting the score leaves it no longer helping.

Under Service and Safety Targets and All Governance Targets, the targets can be met. Both levels also ask you to close every failure pathway, among other targets. The cheapest way costs 7 of the 9 units: Escalate checks, Mark AI-written records, and Store less data.

Every combination that meets these targets includes those three. Counting sets that need a stronger setting, five sets of tools within the budget meet them at each of the two levels.

Escalate checks closes Scores shown to screeners. Mark AI-written records closes Past records used in scoring and Case history read by screeners. Store less data closes Screening decisions entered in OR-KIDS.

Understand the system costs 3 units, or 4 under the two higher levels, and 6 at its stronger setting. While it is on, four tools cost less, never below 1 unit: Review the riskiest first, Keep skills sharp, Pause AI on alarms, and Require sign-off. It is in no combination that meets the two higher levels.

Lingering effects is a Dynamics setting in which damage outlasts its cause. Under All Governance Targets, lingering effects are always on. Then Store less data, Vet connections, and Review the riskiest first work at reduced strength unless Understand the system is also on. Store less data still closes its pathway there.

More is not better here. Every tool at its strongest setting at once closes every failure pathway, but costs 39 units, far over the budget. It also leaves the score no longer helping, so it misses the targets at every level that sets them.