Michigan MiDAS

MiDAS, Michigan's Integrated Data Automated System, is the system Michigan's Unemployment Insurance Agency used to decide whether claimants had committed unemployment insurance fraud. From October 2013 to September 2015 it made many of those fraud determinations by algorithm alone, with no human involvement. It issued accusations, penalties, and collections on them.

What happened

The sources count more than 60,000 fraud determinations from 2013 to 2015. IEEE Spectrum reports that from October 2013 to September 2015, MiDAS decided 40,195 fraud cases by algorithm alone. Of those determinations, 85 percent were wrong.

Another 22,589 cases had some human involvement, and 44 percent of those were wrong. The Benefits Tech Advocacy Hub reports that a 2016 review of 22,000 MiDAS fraud determinations found 93 percent did not involve fraud. The sources do not say how these counts relate to each other.

The Benefits Tech Advocacy Hub says the system falsely accused 40,000 people of fraud in its first two years. IEEE Spectrum counts more than 34,000 wrongly accused from October 2013 to September 2015. Quadruple penalties, wage garnishment, and tax refund seizures followed. The sources also record bankruptcies.

Reporting, the Michigan Attorney General's office, and incident records all describe the extremely high error rate.

How the mistakes repeated

MiDAS applied one rule set to every case. So a single flaw repeated at caseload scale instead of averaging out, and produced tens of thousands of wrongful determinations.

MiDAS wrote its determinations straight into claimant records, with no person in between. Collections and penalty systems then copied them automatically. The sources describe no independent check of those copies against the source records before anyone acted on them.

What human involvement changed

Among the cases with some human involvement, 44 percent of the determinations were wrong. Among the cases MiDAS decided by algorithm alone, 85 percent were wrong. Both counts cover October 2013 to September 2015.

The harm followed from the records and the copies in collections, not from staff decisions. Many determinations were issued without any staff review at all.

How it was stopped

The agency stopped using MiDAS for purely automated fraud decisions in September 2015. That followed pressure from the federal government and a federal lawsuit filed the same month.

The settlement of the Zynda lawsuit, in 2017, forced the reinstatement of human review. It also made the agency reverse and refund certain fraud determinations. The Bauserman civil rights class action, alleging false accusations of unemployment fraud, settled for $20 million in 2022.

The case file reads the case as the strongest documented argument that who can correct, and how fast, matters more than the system's overall accuracy.

The same deployment after the settlements

The Lab draws this deployment a second time, after the settlements, in the case titled After the settlements: review returns to a benefits-fraud system. There, the court-ordered review and the correction of the records are in place.

Where the facts come from

The facts here come from four sources. Three are the AI Incident Database's Incident 373, the Benefits Tech Advocacy Hub's case study, and reporting in IEEE Spectrum. The fourth is the Michigan Attorney General's 2022 announcement of the Bauserman settlement.

What the available tools can and cannot address

A failure pathway is a link between two parts of the network, where a mistake made by one part can be passed on to the other. A closed pathway is one that mistakes stop passing along. The work along it may go on.

This case has a budget of 11 units. Explore (No Targets) sets no targets. Contained means the network's mistakes are corrected rather than building on each other.

Under Explore and Service Targets Only, two tools costing 5 of the 11 units are enough to keep the mistakes contained. Under Service Targets Only the same two also meet the service target. That target asks for the automated system to be helping the work.

One such pair is Peer sharing rules with Vet connections. Peer sharing rules stops the one rule set repeating mistakes across cases.

With every tool at its standard setting, each combination that meets the targets under Service Targets Only includes Vet connections. It stops mistakes passing along the automatic copy of determinations into collections.

Under Service and Safety Targets the targets can be met. That level adds closing every failure pathway. The cheapest combination costs 8 of the 11 units: Understand the system, Vet connections, and Store less data.

Understand the system costs 4 units at this level. While it is on, Vet connections and Store less data cost 2 units each instead of 3. Every combination that meets these targets includes all three tools.

Under All Governance Targets the targets are not fully addressable with the available tools. That level asks for the automated system to be clearly helping the work. No combination within the budget that contains the mistakes and closes every failure pathway does that. The best leaves the system helping, but not clearly.

More money does not change that. With the budget set aside and every tool at its stronger setting, every failure pathway closes, and the automated system ends up hurting the work.

Tools that work on the agency staff alone, such as Pause AI on alarms, do not contain the mistakes on their own. Here the records and the copies pass the mistakes on. Staff decisions do not.