PAN Lab example
Benefits Data Trust wind-down
The node that could not be kept: winding down a benefits-navigation nonprofit
In June 2024 Benefits Data Trust's board voted to close the benefits-navigation nonprofit within 60 days. Its government partners had no designated successor.
See more
Benefits Data Trust was a Philadelphia nonprofit whose navigators, its call-center staff, helped low-income people apply for public benefits. They screened each caller, working out which of many programs the caller likely qualified for, with custom tools such as Community Prism. It was not an algorithmic decision or risk-scoring system, and the failures here came from its closure.
How it worked
Benefits Data Trust was founded in 2005. Under data-sharing agreements with state and city agencies, it identified people likely to qualify for benefits and contacted them. This is called targeted outreach.
It ran a live call center. It held the City of Philadelphia's benefits contract, called BenePhilly, from 2008, plus state contracts. Its navigators screened each caller for many programs at once. They stayed on the line until the application was submitted.
In 2023 it reported helping more than 120,000 people get about $182 million in benefits. It reported more than $10 billion over its lifetime. Philanthropy paid for roughly two-thirds of its work, and government contracts for about one-third.
How it closed
In June 2024, days after parting with its chief executive, the board voted unanimously to wind the organization down within 60 days. The Philadelphia Inquirer reported the chief executive's departure about two weeks before the announcement. A Technical.ly post-mortem put it at roughly a month.
The board set the 60-day window itself, by its own vote. The board's only official explanation was "a perfect storm of circumstances." At the end of 2023 the organization held about $12 million in reserves free of restrictions. It projected about $32 million in 2024 revenue, and the board had been shown an expectation of breaking even.
A search for a partner to take over the work failed. In words the sources quote, "we had exhausted all avenues." The Philadelphia call center took its last call on July 25, 2024. The organization closed on August 24, 2024.
Its layoff notice to the state, which employers must file before mass layoffs, covered 273 employees. The lowest-paid call-center workers were the most affected.
What each side says
A Technical.ly post-mortem, sourced from staff, blamed a built-in gap between costs and income. Each successful application reportedly cost a few hundred dollars in outreach and call-center time, against roughly $2 for online self-service tools. One insider put monthly expenses at $2.5 million against $1 million in revenue. These figures rest on anonymous staff sources, so they are staff claims, not audited figures.
About 120 current and former employees signed an open letter demanding an explanation. The founder said there was "no malfeasance, misappropriation of funds, or fraud." Former employees contend the 60-day timeline was avoidable. That claim is contested.
The Pennsylvania Attorney General's office said it reviews charity failures for breach of fiduciary duty, the legal duty of a charity's leaders to act in its interest. As one former executive put it, the high-tech nonprofit "died the old fashioned way."
Who was left without a successor
The closure left active government partnerships with no designated successor. For the Pennsylvania Department of Aging, Benefits Data Trust had processed nearly 48,000 applications from 27,018 households in its final year. They included nearly 9,000 applications for PACE, a prescription-subsidy program, and 23,000 Medicare drug-subsidy assists. The department leaned on a subcontractor.
Benefits Data Trust was exceeding the BenePhilly contract's performance requirements through the first half of 2024. That work was redistributed across six partner agencies. A Tennessee SNAP food-assistance contract was reported ending with no replacement.
Help with applications moved to harder-to-use channels. One nonprofit took a state SNAP-enrollment contract the following January and hired former staff. Other providers brought the work in-house. Referral waits were reported at several months.
Kristin Romens of The Pew Charitable Trusts called this a "cascading effect." Pew, a funder of Benefits Data Trust, put the benefits going unclaimed by eligible Philadelphians at more than $450 million a year.
What happened to its data and tools
The fate of Benefits Data Trust's custom screening applications, toolkits, and chatbots was reported unresolved at closure. Pennsylvania's own Compass system had already replaced its Community Prism tool. The sources do not describe Compass.
The case file says the closure orphaned the agency data-sharing links. The sources read for this case do not say what became of the data itself.
What this network is drawn from
This network follows the pattern the case file describes. It does not reconstruct the organization, its call center, or Community Prism. It shows the screening tools, the navigators, the agency data and records, the board, and the partners who relied on the work. The households served are outside the network.
What the available tools can and cannot address
A failure pathway is a link between two parts of the network where a mistake made by one part can be passed on to the other. Closing a pathway means mistakes stop passing along it. The work along it goes on.
This case has a budget of 11 units. Each tool costs the same at every target level.
Explore (No Targets) sets no targets. Service Targets Only asks for two things. The network must be self-correcting, meaning its mistakes are corrected rather than building on each other. The screening tools must also be helping the navigators' work. Before any tool is used, the screening tools are helping, but the network is at a tipping point, not self-correcting.
Five tools meet Service Targets Only on their own at their standard setting: Peer sharing rules, Escalate checks, Mark AI-written records, Store less data, and Gate record entries. The cheapest cost 2 units. Escalate checks raises checking when trouble is flagged. Mark AI-written records labels what the screening tools produced in the records. Gate record entries requires sign-off before anything enters the records. Review on schedule does it at its stronger setting, for 4 units. Lingering effects is a Lab setting in which mistakes and reliance on the system stay after their cause is gone. The Lab starts with it on. Upgrade model does it at its stronger setting, for 5 units, while lingering effects are on. With lingering effects off, Vet connections also does it alone, for 3 units, and Upgrade model does not. More than 400 different sets of tools within the budget meet this level.
Under Service and Safety Targets and All Governance Targets, the targets can be met within the budget. Both levels ask you to close every failure pathway, among other targets. Four are open before any tool is used. They are Screening results to navigators, Agency data and rules used in screening, Navigators record applications and outcomes, and Partners send people to navigators.
The cheapest way costs 9 units. Peer sharing rules closes Partners send people to navigators. Escalate checks closes Screening results to navigators. Mark AI-written records closes Agency data and rules used in screening. Gate record entries or Store less data closes Navigators record applications and outcomes.
Every way to meet either level includes Peer sharing rules and Escalate checks. Service and Safety Targets can be met with 10 different sets of tools. There, Vet connections can close the agency data pathway in place of Mark AI-written records. All Governance Targets can be met with 5 sets, and each includes Mark AI-written records.
Upgrade model is in no set that meets either level. Review on schedule is in none that meets All Governance Targets.
More is not better here. Every tool at its strongest setting costs 36 units, more than three times the budget. It closes every failure pathway, but the screening tools then no longer help the work, so it meets no level's targets.
Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.
What this models
This example runs on the Intermediary-node-deletion of a benefits-navigation nonprofit network: 6 components and 11 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.
Evidence base: 3 assumed · 3 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.
Show all 6 assumptions
- assumed
This network follows the pattern the Benefits Data Trust case file describes. It does not reconstruct the organization, its call center, or its Community Prism screening tool. What matters here is the organization and its links, not a tool inside it. Benefits Data Trust was not an algorithmic decision or risk-scoring system. The sources carry no error, override, or accuracy figures for it.
- baseline
The network assumes the board of directors decided the outcome. It alone could decide whether the organization continued, and no agency partner, funder, or regulator could review that choice. Philanthropy paid for roughly two-thirds of the work. The Technical.ly post-mortem blamed a business model that cost a lot for each application. Its figures, a few hundred dollars per successful application against about two dollars for online self-service, come from anonymous staff. They are staff claims, not audited figures.
- baseline
The harm this network watches is partners losing the organization they depended on. The case file describes their access to benefits as running through this one organization. The closure left government partnerships with no designated successor, and a search for a partner to take over failed. So the network draws a handover to successors, which the closure never had. The work moved to harder-to-use channels, such as other partner agencies and a subcontractor. Referral waits were reported at several months. Former employees contend the 60-day timeline was avoidable. That claim is contested, and the board's only official explanation was "a perfect storm of circumstances."
- baseline
The network draws a pathway where the organization's data and tools could be copied outside. The sources report that the fate of its custom screening applications, toolkits, and chatbots was unresolved at closure. The case file says the closure orphaned the agency data-sharing links. The sources read for this case do not say what became of the data itself. The agencies shared that data under data-sharing agreements, to find and contact people likely to qualify. It identifies people, so the network marks the agency data used in screening, and the caller details navigators enter, as sensitive to privacy.
- assumed
This is an organization removed, not one that failed at its work. It was exceeding the performance requirements of BenePhilly, its City of Philadelphia benefits contract, through the first half of 2024. Then its own board closed it within 60 days. No error rate was ever measured for the screening tools, so any error the network gives them is a modeling choice. The danger here is what happens to the partners and links that depended on the organization. It is also whether the exit follows a dated schedule with an orderly handover, or comes abruptly.
- assumed
The network leaves out the households served and whether they received benefits. It shows the organizations and their links only. Pew put the benefits unclaimed by eligible Philadelphians at more than $450 million a year. That figure covers Philadelphia only, and it comes from an analysis by a funder of Benefits Data Trust. It is not a measured effect of the closure, and no measured drop in enrollment caused by the closure has been published. A call, an application, or a rerouted case in this network stands for work between organizations, never a person.
What this example does not show
Show all 2 limitations
- This example does not show the households served, or the benefits they did or did not receive. It shows the organizations and their links only. Pew's figure of more than $450 million a year in benefits unclaimed by eligible Philadelphians covers Philadelphia only. It comes from an analysis by a funder of Benefits Data Trust. It is not a measured effect of the closure, and no measured drop in enrollment caused by the closure has been published.
- Benefits Data Trust was a navigation service, not an algorithmic decision or risk-scoring system, so no error, override, or accuracy figures were published for it. The per-application cost figures come from anonymous staff in a post-mortem, so they are staff claims, not audited ones. The board's only official explanation was "a perfect storm of circumstances." Former employees contend the 60-day timeline was avoidable. This example carries that as a contested claim, not a settled fact.
Sources and evidence
What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.
In June 2024 the board of Benefits Data Trust, a Philadelphia benefits-navigation nonprofit that reported helping more than 120,000 people access about $182 million in benefits in 2023, voted unanimously to wind the organization down within a self-imposed 60-day window, citing only 'a perfect storm of circumstances'; the organization closed on August 24, 2024, laying off 273 employees, despite roughly $12 million in unrestricted reserves at the end of 2023 and about $32 million in projected 2024 revenue.
empirical- Investigative Brubaker, Benefits Data Trust is shutting down in 60 days (The Philadelphia Inquirer, 2024) https://www.inquirer.com/health/benefits-data-trust-bdt-shutting-down-20240625.html
- Investigative Brubaker, Benefits Data Trust is leaving employees and supporters in the dark over its abrupt closure (The Philadelphia Inquirer, 2024) https://www.inquirer.com/health/benefits-data-trust-bdt-surprise-closure-philadelphia-20240627.html
- Investigative Wink, Why Benefits Data Trust fell apart despite millions from philanthropy and government contracts (Technical.ly, 2024) https://technical.ly/civic-news/benefits-data-trust-shutdown-trooper-sanders/
- Investigative Mosbrucker-Garza, Philly's Benefits Data Trust shutters after 20 years. Laid-off workers say they still want answers (WHYY News, 2024) https://whyy.org/articles/philadelphia-benefits-data-trust-closure-employees-laid-off/
The closure left active government partnerships without a designated successor, including a Pennsylvania Department of Aging workload of nearly 48,000 applications from 27,018 households in the final year and a Philadelphia BenePhilly call-center contract the organization was reported to be exceeding through mid-2024; the navigation function fragmented to higher-friction channels, with the work redistributed across partner agencies and a subcontractor and referral waits reported as several months, which a Pew analyst described as a 'cascading effect.'
empirical- Investigative Brubaker, What the loss of Benefits Data Trust means for two government agencies in Harrisburg and Philly (The Philadelphia Inquirer, 2024) https://www.inquirer.com/health/benefits-data-trust-closing-august-23-20240823.html
- Trade press Burnley, After the abrupt closure of Benefits Data Trust, Philly nonprofits are stepping up to fill in the gaps (Technical.ly and The Philadelphia Citizen, 2024) https://technical.ly/civic-news/philadelphia-senior-care-benefits-navigation/
- Investigative Mosbrucker-Garza, Philly's Benefits Data Trust shutters after 20 years. Laid-off workers say they still want answers (WHYY News, 2024) https://whyy.org/articles/philadelphia-benefits-data-trust-closure-employees-laid-off/
Where this connects
Institutional pressures in this domain
- Workload surge — Demand outruns staffing; per-case attention shrinks and review becomes triage.
- Vendor opacity — The deploying institution cannot inspect the model, data, or update pipeline it is accountable for.
- Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
- Reviewer bottleneck — One fixed-capacity checking stage sits between AI output and consequence; everything queues behind it.
- Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.
All of them in context on the Benefits navigation & public-facing chat domain page.
Levers available here and the patterns behind them
- Peer sharing rules — Peer-edge governance
- Review on schedule — Oversight cadence & retrospectives
- Vet connections — Connection authorization
- Store less data — Data minimization
- Escalate checks — State-feedback vigilance
- Keep skills sharp — Deskilling-arrest mandate
- Keep prompts neutral — Framing and mirroring reduction
- Gate record entries — Human-in-the-loop write gating
- Mark AI-written records — Provenance labeling
- Upgrade model — Improve the model
Documented case histories
- Benefits Data Trust wind-down
- Nava assistive benefits chatbot
- Caddy adviser copilot at Citizens Advice
- GOV.UK Chat
- Mass.gov Virtual Assistant
- Frida (NAV Norway)
- SSA 800-Number Conversational AI Assistant
- EDD Virtual Assistant
- Burokratt
- Singapore's chatbot fleet refresh: eighty scripted engines slated for retirement onto a shared LLM platform
- IRS collection chatbots: expanded and made permanent with no performance measures
- Albert France Services
- Propel in-app SNAP benefits assistant
- GetCalFresh: the nonprofit front door that carried most of California's online SNAP intake
- MyFriendBen benefits screener