Eckerd Rapid Safety Feedback

Rapid Safety Feedback is a proprietary predictive tool built by Eckerd, a nonprofit child-welfare agency, with the analytics firm Mindshare Technology. It mines an agency's own case records to flag open cases whose features resemble past serious-injury and fatality cases. Quality-assurance reviewers receive the flags and coach frontline caseworkers on those cases, and no action is automated.

How it began

Eckerd became the lead child-welfare agency for Hillsborough County, Florida, in 2012. At the time it was called Eckerd Youth Alternatives, and also Eckerd Kids. It took over after at least eight children had died under the previous provider's supervision.

In January 2013 Eckerd launched Rapid Safety Feedback, built with the analytics firm Mindshare Technology. Eckerd later became Eckerd Connects.

What Eckerd reported

Eckerd reported that Hillsborough County had no child deaths from maltreatment over a period of about three years. It also reported that measures of case practice improved by about 22 percent on average.

Those figures were Eckerd's own, reported through the press. Eckerd itself said it did not claim the analytics caused them. Critics argued that no evidence tied the gains to the tool.

How it spread

On the strength of that reported success, agencies in several states adopted the tool. Each copy was tailored to the new place and matched against that place's own records.

In 2016 a report by the federal Commission to Eliminate Child Abuse and Neglect Fatalities highlighted the approach as "innovative", with a case study of Hillsborough. By then the tool was already in use across those states.

What the independent evaluation found

A peer-reviewed evaluation published in 2022 tested the process. Its authors were Parker, Williams, Pecora, and Despard, writing in the journal Child Abuse and Neglect. It found the process did not lower repeat high-severity maltreatment among children identified as high risk.

Its main result, a measure called an odds ratio, was about 1.05. An odds ratio compares the odds of an outcome between two groups. A ratio of 1 means no difference, so 1.05 means almost none. The case file calls it a joint odds ratio and does not say what it combines.

Why this case matters

This tool scaled on an endorsement and a claim no one had independently tested. The legitimacy behind its spread came from Eckerd's own before-and-after numbers and a federal commission's praise. It did not come from evidence that the tool changed outcomes.

The trap here is not a wrong number. It is a number everyone trusted because someone respectable repeated it.

Where it could have been governed

The chances to govern this deployment came before any dashboard. Procurement could have demanded documented, checkable performance claims. A sign-off could have asked "on what evidence?" before the next agency adopted the tool. A schedule of re-evaluation could have treated the launch endorsement as a question rather than an answer.

What this network is drawn from

This network follows the pattern the case file describes. It is not a reconstruction of the actual tool. It shows Rapid Safety Feedback, the quality-assurance reviewers, the frontline caseworkers, and the agency case records. The children and families served are outside the network.

What the available tools can and cannot address

A failure pathway is a link between two parts of the network where a mistake made by one part can be passed on to the other. Closing a pathway means mistakes stop passing along it. The work along it goes on.

This case has a budget of 11 units. Each tool costs the same at every target level, except Understand the system. It costs 3 units under Explore (No Targets) and Service Targets Only, and 4 under the two higher levels. While it is on, four other tools cost less.

Explore (No Targets) sets no targets. At the start, mistakes are copied about as fast as they are corrected. Service Targets Only asks for them to be caught faster than they are copied, and for Rapid Safety Feedback to be helping the work. It is helping at the start.

Escalate checks meets that level's targets on its own, for 2 units. So does Mark AI-written records, also for 2 units. In all, 799 different sets of tools within the budget meet them.

Service and Safety Targets also asks you to close every failure pathway, and to keep the work from falling behind demand. Five are open at the start. They are Flags sent to reviewers, Coaching results written to records, and Past serious cases set the pattern. The other two are Case history read in coaching and Tool copied to each new agency.

Four of these pathways each have exactly one tool that closes them. Escalate checks closes Flags sent to reviewers. Gate record entries closes Coaching results written to records. Check with a second model closes Tool copied to each new agency. Mark AI-written records closes Past serious cases set the pattern. It also closes Case history read in coaching, which Understand the system closes too.

Those four together cost 10 units and meet the targets. So do the same four with Mark AI-written records at its stronger setting, for 11 units. Adding Require sign-off, for 11 units, also meets them but changes nothing. No other set within the budget does.

All Governance Targets is not fully addressable with the available tools. That level also asks for Rapid Safety Feedback to be clearly helping the work, and sets a target on the deployment's environmental footprint. Within the budget, every set that closes all five pathways leaves the tool helping, but not clearly. More money does not change that. No set at any cost meets that level's targets, and adding tools leaves the tool helping less.

Using every tool on offer, each at its stronger setting, would cost 35 units, more than three times the budget. It closes every failure pathway. But Rapid Safety Feedback then adds too little to the work to meet the targets at any level that sets them.