Skip to content

PAN Lab example

Robodebt (Australia)

After the Commission: refunding the debts an engine raised

Robodebt raised hundreds of thousands of wrongful welfare debts in Australia. Outside controls ended it. This case asks what the agency needs to check itself.

See more

Robodebt, officially the Online Compliance Intervention, was an automated scheme that raised welfare debts in Australia. The Department of Human Services averaged a person's annual income from the Australian Taxation Office across fortnights. It compared the result with the income the person had declared, raised debts, and left the person to disprove them.

Two points in time

This network shows the same deployment as the Robodebt Scheme network, at its second documented point in time. The Robodebt Scheme network shows the scheme running. This network shows the refund work after the scheme ended, and the reform that follows it. It runs from the class-action settlement at the end of 2020 through the reform after the Royal Commission's July 2023 report.

What went wrong

The Royal Commission into the Robodebt Scheme documented hundreds of thousands of wrongful debts raised by an unlawful income-averaging method. It documented the human toll of automated debt collection against vulnerable people. It also documented the institutional failures that kept the scheme running for years despite internal and external warnings.

The Royal Commission's report, released in July 2023, found the scheme unlawful. It found that income averaging, as Robodebt used it, "was in fact inconsistent with social security legislation." It called Robodebt "a crude and cruel mechanism, neither fair nor legal."

The refunds and the settlement

After the scheme ended, about 470,000 wrongful debts were to be refunded to recipients.

At the end of 2020, the government settled a class action covering the refund of about 430,000 debts, worth more than $720 million before damages and legal fees. The Federal Court approved the settlement in 2021, in Prygodicz v Commonwealth.

Who acted

The case file records that warnings existed, but no one inside the agency was positioned, or willing, to act on them. The work of correcting debts fell on recipients, each appealing alone.

The case file says the scheme's end required the heaviest outside actors: courts and a Royal Commission. By the sources' dates, the class-action settlement came first, at the end of 2020, and the Federal Court approved it in 2021. The Royal Commission reported in July 2023.

What this case asks

This case asks what the agency needs to halt, check, and correct such a system itself. The halt, the method check, and the correction that ended Robodebt came from outside.

In this case, the reform means whoever carries the Royal Commission's recommendations into the agency. The sources read for this case do not name them. The reform can improve the engine, and add a check as it assesses each person. It can raise officers' capacity to correct debts. It can also limit how much engine output is sent to officers, and how much the engine writes to the debt register.

The Royal Commission made 57 recommendations. One was a body to monitor and audit automated decision making.

What the available tools can and cannot address

A failure pathway is a link between two parts of the network where a mistake made by one part can be passed on to the other. A tool closes a pathway when mistakes stop passing along it.

This case has a budget of 10 units and offers five tools. Upgrade model costs 3 units, or 5 at its stronger setting. Verify output costs 4. Train the staff costs 2, or 4 at its stronger setting. Pause AI on alarms costs 4. Gate record entries costs 3. These prices are the same at every target level.

Explore (No Targets) sets no targets. Under Service Targets Only, the targets are met before you apply any tool. Service here means the agency's work getting done. Two kinds of combination miss those targets. Every combination that includes Pause AI on alarms misses them, because halting the engine also halts the assessments sent to the internal review. Every combination with both Verify output and Gate record entries misses them too.

Under Service and Safety Targets and All Governance Targets, this case is not fully addressable with the available tools. Both levels ask you to close every failure pathway, among other targets. Every combination within the budget was checked, and none meets them.

Money is not what stands in the way. With every tool at its strongest setting, costing 20 units, two failure pathways stay open. They are officers reading debts for refund work, and the debts read into the proceedings.

No tool offered here acts on either of them. The first is the refund work this period exists for. The second is the courts' and the Royal Commission's reading of the debt register. That is a finding about the deployment, not a gap in your approach.

Stylized model of a documented deploymentPublic benefits & eligibility

Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.

What this models

This example runs on the Robodebt-class remediation network after the Royal Commission network: 5 components and 10 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.

Evidence base: 5 assumed · 3 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.

Show all 8 assumptions
  • assumed

    This network models the same deployment as the Robodebt Scheme network, at a later point. That point runs from the class-action settlement at the end of 2020 through the reform after the Royal Commission's July 2023 report. It is a simplified model of the refund work and the reform, not a copy of the real system.

  • baseline

    This period comes from the settlement and the Royal Commission. About 470,000 wrongful debts were to be refunded to recipients. A class action settled about 430,000 debts for more than $720 million. The Royal Commission found the scheme unlawful. The network shows refunds and cancelled debts at a scale of hundreds of thousands of debts, because the sources report that many. It shows the settlement's direction to officers as working, because the sources report the settlement approved and almost $112 million of the compensation claimed paid.

  • assumed

    Pathways the sources show ended with the scheme are left off this network. One is copying the register into automated debt recovery. Another is feeding earlier debts into later assessments. The others are applying one rule to every recipient, and staff presuming that a debt is owed. All four belong to the running scheme. The Robodebt Scheme network and the case file describe them.

  • baseline

    The pathways for averaged debts to the register and averaged figures to officers stay on the network, though the scheme ended. The reform can limit how much each may carry. That is the question this case poses. Gate record entries and Pause AI on alarms are the tools for it.

  • baseline

    The internal pause on debt-raising and the standing check on the averaging method are on the network as things the agency could build. The halt and the method check that ended the scheme came from outside the agency. The case file names courts and a Royal Commission. This case treats three things as the reform's to build, not as finished. They are correction capacity, catching mistakes as the engine assesses a person, and assessments that can be checked against actual income.

  • assumed

    Two things the reform can do have no tool in this case. One is cleaning the debt register by checking what each debt says. The other is raising the share of assessments that can be checked against actual income. So the cancelling of wrongful debts appears in the refunds and cancelled debts officers write, not in a tool. The Lab offers no tool that audits stored records by their content. A Lab tool that deletes records without reading them would not do this cleaning.

  • assumed

    Workload and staffing follow the same reasoning as the Robodebt Scheme network, applied to this later period. The standing workload is the nationwide refund work itself. Without the engine, the alternative is the lawful manual process: the agency obtains actual fortnightly income and bears the burden of proving a debt. This case treats that process as the working one the scheme replaced, and this period as restoring it.

  • assumed

    Robodebt's harm fell on welfare recipients: people on low incomes, students, and people with disability. The case file and its sources record the human toll, with the sources' own caveats. This network traces how mistakes pass between the agency's parts, not that harm. Recipients are not a part of the network. The network counts refund sums as part of the institution's work, not as an outcome for any recipient. It does not estimate what anyone the agency serves gained or lost.

What this example does not show

Show all 2 limitations
  • Robodebt's harm fell on welfare recipients. Refunding a wrongful debt does not repair that harm. The case file and its sources record the human toll, with the sources' own caveats. This example traces only how mistakes pass inside the institution. It counts refund sums as part of the institution's work, and it leaves recipients out of the diagram. It does not estimate what anyone the agency serves gained or lost.
  • Two things the reform can do have no tool in this case. They are cleaning the debt register by checking what each debt says, and raising the share of assessments that can be checked against actual income. The network's assumptions describe both. The cancelling of wrongful debts appears in the refunds and cancelled debts officers write, not in a tool.

Sources and evidence

What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.

  • The Robodebt Scheme's end came from outside the deploying institution: the Federal Court approved a class-action settlement covering roughly 430,000 debts for more than $720M (Prygodicz v Commonwealth (No 2) [2021] FCA 634), around 470,000 wrongful debts were to be repaid, and the Royal Commission (2023) found the scheme unlawful.

    empirical
    • Government Prygodicz v Commonwealth of Australia (No 2) [2021] FCA 634 (Federal Court of Australia) https://robodebt.royalcommission.gov.au/publications/exhibit-2-2598-rbd999900010225-prygodicz-v-commonwealth-australia-no-2-2021-fca-634
    • Government Royal Commission into the Robodebt Scheme, Report (2023) https://robodebt.royalcommission.gov.au/publications/report
    • Government Royal Commission into the Robodebt Scheme (2023) https://robodebt.royalcommission.gov.au/
    • Investigative Law Society Journal, Crude, cruel and unlawful: Robodebt findings https://lsj.com.au/articles/crude-cruel-and-unlawful-robodebt-royal-commission-findings/
    • Reference Royal Commission into the Robodebt Scheme (Wikipedia overview) https://en.wikipedia.org/wiki/Royal_Commission_into_the_Robodebt_Scheme
  • The Royal Commission into the Robodebt Scheme documented hundreds of thousands of wrongful debts raised by an unlawful income-averaging method, with the onus placed on recipients to disprove automated assessments.

    empirical
    • Government Royal Commission into the Robodebt Scheme (2023) https://robodebt.royalcommission.gov.au/
    • Government Royal Commission into the Robodebt Scheme, Report (2023) https://robodebt.royalcommission.gov.au/publications/report
    • Government Prygodicz v Commonwealth of Australia (No 2) [2021] FCA 634 (Federal Court of Australia) https://robodebt.royalcommission.gov.au/publications/exhibit-2-2598-rbd999900010225-prygodicz-v-commonwealth-australia-no-2-2021-fca-634
    • Investigative Law Society Journal, Crude, cruel and unlawful: Robodebt findings https://lsj.com.au/articles/crude-cruel-and-unlawful-robodebt-royal-commission-findings/
    • Reference Royal Commission into the Robodebt Scheme (Wikipedia overview) https://en.wikipedia.org/wiki/Royal_Commission_into_the_Robodebt_Scheme

Where this connects

Institutional pressures in this domain

  • Austerity & recovery incentives — Cost-cutting and overpayment-recovery targets tilt the system toward denial and enforcement errors.
  • Vendor opacity — The deploying institution cannot inspect the model, data, or update pipeline it is accountable for.
  • Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.
  • Reviewer bottleneck — One fixed-capacity checking stage sits between AI output and consequence; everything queues behind it.

All of them in context on the Public benefits & eligibility domain page.

Levers available here and the patterns behind them

Documented case histories