Skip to content
ParamergeParamerge
Stephen Lieberman
Through Paramerge
Contact

PAN Lab example

Homebase Risk Assessment Questionnaire

The prevention screener that has to describe itself in public every year

Caseworkers in Homebase, New York City's homelessness-prevention program, ask households fifteen questions. Based on the points total, they offer full services or a brief contact.

See more

The Homebase Risk Assessment Questionnaire is fifteen questions a caseworker asks each household applying to Homebase. Each answer earns points, and the total sets whether the household is offered full services or a brief contact. It is a point score, not a machine-learning system.

Who owns and runs the questionnaire

The Homebase Risk Assessment Questionnaire is fifteen questions that a caseworker asks each household applying to Homebase. Each answer earns points, and the total sets which tier of services the caseworker offers.

New York City's Department of Social Services owns the questionnaire, through the Homebase program office of its Human Resources Administration. The department's Office of Research & Policy Innovation maintains it. Caseworkers at seven contracted nonprofit providers ask the questions.

The city began using it in June 2012. It is a point score, not a machine-learning system. The city's public register of algorithmic tools lists it as computation type "Scoring" and purpose "Resource allocation".

What Homebase is

Homebase is New York City's homelessness-prevention program. It began in 2004. Seven nonprofit providers run its services under contract with the city, from 26 neighborhood offices. The city's Department of Homeless Services says the program serves more than 25,000 at-risk households a year.

This network is drawn from the public record of the program and its questionnaire. It shows the structure that record documents. It is not the city's own system or data.

The questionnaire

Since June 2012, a caseworker at a provider office has asked each applying household fifteen questions. Each answer is worth one to three points, for a total between 0 and 25. Before a 2023 revision, a household with a total of 7 or more was offered full services. Full services are financial assistance, case management, and legal and mediation referrals. A household with a lower total was offered a brief contact of one or two visits.

A caseworker who wants to give a household a different tier needs a supervisor's approval. No public source states the cutoff in use since the 2023 revision.

What the city must publish

Local Law 35 of 2022 requires every city agency to report, by 31 December, every algorithmic tool it used one or more times in the prior calendar year. Each report must include six required items of information.

The city's Office of Technology and Innovation compiles the reports into one public report. It delivers that report to the mayor and the speaker of the City Council each 31 March. This report is the city's public register of algorithmic tools.

The city's Department of Social Services has filed an entry for this questionnaire in all four reports so far, covering 2022 through 2025. In 2023 the department revised the questions. After the report covering 2023 was published, the department amended its entry, on 27 March 2024, to disclose the revision. The entries are the department's own descriptions, and no one audits them.

What the department found about its own tool

The Department of Social Services' Office of Research & Policy Innovation maintains the questionnaire. In 2022 it published a peer-reviewed study in the journal Housing Policy Debate. The study covered 48,450 families with children who applied between 2013 and 2016.

Of the families with totals at or above the cutoff of 7, 13.7 percent applied for shelter within two years. Of the families below it, 5.9 percent did.

The study also reported two problems with the tool. First, intake totals piled up at exactly the cutoff: 4,269 cases scored 6, 10,634 scored 7, and 7,756 scored 8. The researchers wrote that this "suggests that Homebase staff may be focused on getting families to that threshold so they qualify for full services."

Second, in 6.1 percent of 58,674 applications, a supervisor approved a different tier from the one the points total produced. Of the families moved up to full services, 3.7 percent later applied for shelter. Of the families moved down to brief services, 25.8 percent did. The paper concluded that worker judgment was less accurate than the questionnaire at predicting risk of homelessness, on average.

All of these figures come from the department's own study. Publishing them was unusually candid, but peer review is not independent evaluation.

What the audits examined

Three government audits have examined Homebase. In 2013 the office of the New York City Comptroller, the city's auditor, examined how the Department of Homeless Services monitored the program. It found no written monitoring policies and every provider risk assessment announced in advance. The Department of Homeless Services rejected the recommendations for unannounced visits.

In January 2020 the city comptroller examined how the Human Resources Administration, part of the Department of Social Services, oversaw the program. The audit found that the administration's program office had performed 80 of the 240 required reviews of provider case files.

In January 2026 the office of the New York State Comptroller examined CityFHEPS, the city's rental-subsidy program, for Homebase clients. It found no evidence of income verification in 30 of 75 sampled case records. The audit reports CityFHEPS spending rising from 176 million dollars in fiscal year 2019 to 834 million in fiscal year 2024. CityFHEPS serves Homebase clients among others, and the sources do not say what share of that spending went to them.

None of the three audits examined the questionnaire. The questionnaire is described in public every year and evaluated by its own department, and no government audit has examined it.

What changed, and what the record does not say

The department's 2022 paper also simulated a lower cutoff of 5 on the revised questionnaire. It reported that precision would rise from 13.7 percent to 15.2 percent, with a similar number of families enrolled.

The sources do not define precision. Its starting figure, 13.7 percent, matches the share of families at or above the cutoff of 7 who applied for shelter within two years.

The 2023 revision changed the questions. The paper's own comparison found the revised questions separated households about as well as the old ones. No public source says whether the cutoff changed.

Whether the program works

Two studies from outside the department's research office found that the program works. The first was a randomized trial from 2010 to 2013, run by Abt Associates for the Department of Homeless Services. It divided 415 households into groups by chance, so that their outcomes could be compared. It analyzed 295 families with children across eleven sites.

The trial found the share of families spending at least one night in shelter falling from 14.5 percent to 8.0 percent. The share applying for shelter fell from 18.2 percent to 9.3 percent.

The second, an independent study by Goodman, Messeri, and O'Flaherty, compared the city's community districts. It estimated that Homebase reduced family shelter entries by roughly 5 to 11 percent.

What the available tools can and cannot address

A failure pathway is a link between two parts of the network where a mistake made by one part can be passed on to the other. Explore (No Targets) has no targets to meet. Under Service Targets Only, the targets can be met within the case's budget.

Under Service and Safety Targets and under All Governance Targets, the available tools cannot fully address this deployment. Both levels require you to close every failure pathway. Five pathways stay open: a mistake can still be passed along them. These five are the program's own working steps. No lever within the department's documented authority closes them.

Two of the five score the recorded answers and turn the total into the tier a caseworker offers. A third links the total to a supervisor's review when a caseworker proposes a different tier. The fourth is the publication of the annual filing. In the fifth, a household's case record is used to open a rental-subsidy file. The Lab offers only levers that match the department's documented authority.

Every other target can be met. If the requirement to close every pathway were dropped, a set of levers costing 7 of the 12 units would meet all the other targets at both levels.

One of the five pathways is the public filing the law requires. What the department publishes there is a description of the tool, not any household's record. The Lab still counts it the same way as any other pathway out of the network. This is a finding about the deployment, not a gap in your approach.

Stylized model of a documented deploymentHousing & homelessness services

Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.

What this models

This example runs on the Prevention-screener-class shelter-entry triage network: 12 components and 23 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.

Evidence base: 4 assumed · 4 published baseline · 3 measured. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.

Show all 11 assumptions
  • assumed

    Binding framing, applied throughout. The instrument is a regression-derived additive point screener: fifteen items scored 0 to 25, distilled by backward elimination from a proportional-hazards analysis of 2004-2008 program data, and filed under the city register's scoring computation type. It is never described as machine learning on this map. A genuinely machine-learning shelter model does appear in the same registers, but it belongs to a different agency and a different purpose, and the two are never conflated. The documented cutoff of 7 points describes the configuration before the 2023 item revision; no public source states the threshold in force afterwards, and the register entries say only that a certain point threshold applies.

  • measured

    Every performance and distortion figure on this map is agency-reported, and is labelled as such at each use. The discrimination figures (13.7% of households at or above the cutoff applied for shelter within two years against 5.9% below it; area under the curve 0.7387 for the deployed instrument and 0.7389 for the revised one), the score-clustering counts (4,269 cases at a total of 6, 10,634 at exactly 7, 7,756 at 8), the override volume and outcomes (6.1% of 58,674 applications, 4.8% up and 1.4% down, with 3.7% against 25.8% subsequent shelter applications) and the alternative-cutoff simulation (precision 13.7% rising to 15.2% at a cutoff of 5) all come from one peer-reviewed paper written by the deploying agency's own research office. Peer review is not independence, and the paper's own caveats apply on this map: the observed score distribution is confounded by the clustering it documents and by low-scoring households leaving intake, and outcomes by service tier are contaminated by the very overrides being measured.

  • baseline

    Three review steps, because three separately-documented supervisory bodies exist with three different documented behaviours, and each is wired in by an inbound pathway rather than left dangling. The provider supervisor tier receives the second model-to-operator pathway, drawn faint, the strength its own measured 6.1% override volume gives it. The government program auditors receive a record-to-staff read, drawn faint, from the case record, which is what all three audits actually sampled, plus a model-to-operator pathway, empty, that records the documented fact that none of them examined the screener. The Council register oversight receives a record-to-staff read at a substantial level from the annual filing, because the compilation and delivery are statutory and happened four years running. None of the three is decorative and none is a duplicate of another: one approves individual departures from the score, one writes recommendations about program administration, one compels an annual public description.

  • baseline

    The data-leaving pathway runs at a substantial level because the statute requires it, and it is scored here as an open pathway all the same. The corpus idiom draws this pathway as an exposure to close, and every other instance in the catalogue carries client or personal data out of the boundary. Here the statute obliges the agency to report, by 31 December, every algorithmic tool it used one or more times during the prior calendar year, with six mandatory disclosure elements, and obliges the technology office to compile the filings into one public report delivered to the mayor and the Council speaker each 31 March. Four consecutive editions carry this tool. It sits at a substantial level rather than full because the production is annual against a tool the register itself lists as used daily, and because the entries are unaudited agency self-reports in an agency-self-identified regime. What crosses the boundary here is a description of the tool rather than a household record, but the model does not distinguish the two: an open data-leaving pathway of any kind draws on the privacy reading, and neither of the two levers that would close or slow it is offerable on this deployment's evidence. So this map carries a pathway out that the record documents as a duty and the reading scores as an exposure, and both halves of that are stated rather than resolved. Nothing on this map treats the register as verification of anything it describes.

  • measured

    The model-to-operator pathway into the auditors sits empty on a documented absence, not on silence. Three government audits of this program exist and all three are program audits: the shelter agency's monitoring of providers in 2013, the benefits agency's oversight of the program in 2020, and the downstream rental-subsidy channel for this program's clients in January 2026. None examined the risk screener. Drawing the pathway empty rather than omitting it is the honest form, because the auditors exist, hold access and could look; what the record documents is that in thirteen years they have not. Nothing in this file may be cited as an algorithmic audit of this instrument.

  • measured

    The enforcement node grants rather than penalizes, and its reconciliation is drawn faint rather than empty. The downstream channel is documented: rental-assistance vouchers and one-shot emergency grants processed for households this program screened, 57,888 new subsidy cases and 123,762 individuals housed since 2018, spending rising from 176 million dollars in one fiscal year to 834 million five years later. Its reconciliation is measured, not guessed: the state audit sampled 75 case records and found 30 with no evidence of income verification, which puts the check at 45 of 75. That is a present-but-partial check and it is drawn that way, in deliberate contrast with the corpus's replicate-without-recheck shape, which is empty.

  • baseline

    baselineDemand 3 and manualCapacity 2 are both derived. Demand comes from the documented workload and its documented growth: 48,450 deduplicated families with children applying across 2013-2016 for 58,674 family-years, a caseload rising from about 600 cases a month in 2013 to more than 1,500 a month in some 2015 months, a network of 26 offices run by seven contracted nonprofits, a city statement of more than 25,000 at-risk households served a year, and a register entry listing the frequency of use as daily. Capacity is 2 rather than the strained 1 because a full human screening channel is documented and it predates the tool: before June 2012 caseworkers screened every applicant by judgment and deemed 66.5% of them eligible, and a live supervisory gate still runs on every departure from the score. It is not 3 because the same record bounds that channel: the original study found the instrument raising correct targeting by 26% and cutting misses by almost two-thirds at equivalent false-alarm rates, and the city audit found the program office performing 80 of 240 required provider case-file reviews.

  • baseline

    The closed loop is drawn through its documented mechanism rather than asserted. The instrument writes the tier and any override flag into the case record at full strength; the record is read back by the agency's own research office, faintly; that office revised the item set, faintly. So the records the deployed screener has been generating since June 2012 are what produced its 2023 revision, and the agency's own paper says the distortions it found in those records are carried into any later re-examination of them. The 2013-2016 revalidation cohort of 48,450 families is exactly such a re-examination, and the office published that caveat about its own work.

  • assumed

    Served people are not in these dynamics and no served-person outcome is computed anywhere on this map. Applicant households, the children in them and their housing outcomes are boundary-only: a score, a tier, an override or a subsidy file here is an institutional signal, never a person. The causal outcome evidence for this program lives in the case file and is measured outside any diagram like this one: a randomized trial of 295 families across eleven sites found the share spending at least one night in shelter falling from 14.5% to 8.0%, the share applying for shelter falling from 18.2% to 9.3%, and average shelter nights falling by 22.6; a community-level study estimated 5-11% fewer family shelter entries, with a 14.2-million-dollar annual budget avoiding an estimated 20 to 44 million dollars of shelter spending. The city's separate claim of a prevention rate around 90 to 97% is a performance metric with no counterfactual and is used nowhere on this map as an effect size.

  • assumed

    Documented actors this map does not draw, and why. The external evaluators (the randomized trial team and the community-level economists) evaluated the program rather than the instrument, and their documented action is a completed study rather than a recurring pathway into this deployment, so they are narrated in the research office's role and in the case file instead of drawn as a body with an invented inbound flow. There is no vendor node because the current register lists no vendor at all; the earlier filings describe academic researchers contracted to analyse the agency's own data, and the literal vendor-name label appears in only two of the four cycles. No worklist is drawn because the record documents caseload growth and repeat episodes rather than a queue or a backlog, and a worklist here would be decoration. No guardrail is drawn because the record documents no automated screen on the instrument's output; the only check on an assignment is a human supervisor.

  • assumed

    Two figures on this map are upper bounds on flow rather than counts of harm, and are used as such. The 58,674 family-years cover applications across 2013-2016 by 48,450 deduplicated families with children, so a family appears more than once. The 57,888 subsidy cases and 123,762 individuals housed cover the whole downstream channel from 2018 to March 2025, which serves this program's clients among others, and the state audit's scope ran July 2021 to July 2025. The independent provider-affiliated re-analysis of 9,630 participants covers one provider's three catchment areas rather than the citywide caseload, and that scope is kept in every use of its figures.

What this example does not show

Show all 5 limitations
  • Served people are not modeled here; the Lab models institutional propagation only, and the outcomes that land on applicant households and the children in them are documented in the case file and measured outside any diagram like this one. A score, a service tier, an override or a subsidy file on this map is an institutional signal, never a person. The causal evidence for what this program does for households sits deliberately outside these dynamics: a randomized trial of 295 families across eleven sites found the share spending at least one night in shelter falling from 14.5% to 8.0%, the share applying for shelter falling from 18.2% to 9.3%, and average shelter nights falling by 22.6; a community-level study estimated roughly 5-11% fewer family shelter entries, with a 14.2 million dollar annual budget avoiding an estimated 20 to 44 million dollars of shelter spending. The city's separate claim of a prevention rate around 90 to 97% is a performance metric with no counterfactual and is used nowhere on this map as an effect size.
  • This is a point screener, and the threshold on it is a historical fact rather than a current one. The instrument is fifteen questions scored 0 to 25, distilled by backward elimination from a proportional-hazards analysis of 2004-2008 program data, and the public register files it under scoring for resource allocation. It is not machine learning and is never described as such here. A genuinely machine-learning shelter model does appear in the same public registers, but it belongs to a different agency, targets a different population and a different resource, and is never conflated with this one. The cutoff of 7 points describes the documented configuration before the 2023 item revision; no public source states the threshold in force afterwards, and the register entries say only that a certain point threshold applies.
  • Almost every number on this map comes from the deploying agency about itself. The discrimination figures, the score-clustering counts, the override volume and the override outcomes, and the alternative-cutoff simulation all come from one peer-reviewed paper written by the agency's own research office. Publishing them was unusually candid and peer review is not independence. The paper's own caveats travel with the figures: the observed distribution of totals is confounded both by the clustering it documents and by low-scoring households leaving intake before they are counted, and outcomes compared by service tier are contaminated by the overrides being measured. The annual filings are likewise the agency's own unaudited words in a regime that depends on agencies identifying their own tools.
  • None of the three government audits examined the risk screener, and nothing here may be read as an audit of it. The 2013 city audit examined the shelter agency's monitoring of providers, the 2020 city audit examined the benefits agency's oversight of a 53 million dollar a year program, and the January 2026 state audit examined the downstream rental-subsidy channel for this program's clients. All three are program audits. The absence is drawn on the map as a pathway at zero rather than left off it, because the auditors exist and hold access; what the record documents is that in thirteen years they have not looked at the instrument. The standing weakness those audits did find, in monitoring policy and review cycles, is a persistent gap across two audits seven years apart rather than a decline, which is why no decay pressure is applied here.
  • Scale figures on this map are flows, not counts of people or of harm. The 58,674 family-years cover applications by 48,450 deduplicated families with children across four years, so a family can appear more than once. The 57,888 subsidy cases and 123,762 individuals housed cover the whole downstream channel from 2018 to March 2025, which serves this program's clients among others, and the state audit's own scope ran from July 2021 to July 2025. The independent provider-affiliated re-analysis of 9,630 participants covers one provider's three catchment areas, not the citywide caseload, and its finding that more than 40% of the race field carried a do-not-know response is a record-quality signal from that cohort rather than a citywide measurement.

Sources and evidence

What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.

  • New York City's Homebase homelessness-prevention program has since June 2012 routed applicant households on a 15-item Risk Assessment Questionnaire scored 0-25 (each answer worth 1-3 points), distilled by backward elimination from a Cox proportional-hazards model of shelter entry fitted to 11,105 families who applied October 2004 - June 2008 (12.8% entered shelter within three years; decile risk 1% to 37%); a total at or above 7 points routed a household to 'full' services (financial assistance, case management, legal and mediation referrals) rather than a 'brief' one-or-two-visit contact. It is a regression-derived additive point screener, not a machine-learning system: the city's statutory algorithmic-tool register files it under computation type 'Scoring', purpose 'Resource allocation', autonomy 'Monitored', frequency 'Daily', with 'Vendor(s): None' in the CY2025 entry. Against caseworker judgment, which had deemed 66.5% of applicants eligible, the instrument would have increased correct targeting of families entering shelter by 26% and cut misses by almost two-thirds at equivalent false-alarm rates. Services are delivered by seven contracted nonprofit providers across 26 neighborhood offices, and The Department of Homeless Services states the network serves more than 25,000 at-risk households a year. The causal effect evidence for the program comes from outside the deploying agency's own research office: a randomized controlled trial by Abt Associates, commissioned by the Department of Homeless Services (2010-2013, 295 families with children analyzed across eleven sites), found the share spending at least one night in shelter falling from 14.5% to 8.0%, the share applying for shelter falling from 18.2% to 9.3%, and average shelter nights falling by 22.6; an independent community-district difference-in-differences study estimated roughly 5-11% fewer family shelter entries (11.2 log points, 95% CI 3.6-18.8), a $14.2M annual budget avoiding an estimated $20-44M of shelter expenditure. The separately claimed 'prevention rate' of around 90-97% is a city performance metric with no counterfactual and is not an effect size. The threshold of 7 describes the documented pre-2023 configuration; no public source states the threshold in force after the 2023 item revision.

    empirical
    • Academic Shinn M, Greer AL, Bainbridge J, Kwon J, Zuiderveen S, Efficient Targeting of Homelessness Prevention Services for Families, American Journal of Public Health 103 S2 (2013) https://pmc.ncbi.nlm.nih.gov/articles/PMC3969118/
    • Government New York City Office of Technology and Innovation, Agency Compliance Reporting of Algorithmic Tools Calendar Year 2025 final version dated March 27 2026, Department of Social Services entry for the Homebase Risk Assessment Questionnaire (2026) https://www.nyc.gov/assets/oti/downloads/pdf/reports/LL35%20Report%202025%20-%20Final%20-%202026-03-27.pdf
    • Government Rolston H, Geyer J, Locke G, Evaluation of the Homebase Community Prevention Program Final Report, Abt Associates for the New York City Department of Homeless Services (2013) https://www.abtglobal.com/sites/default/files/migrated_files/cf819ade-6613-4664-9ac1-2344225c24d7.pdf
    • Academic Goodman S, Messeri P, O'Flaherty B, Homelessness prevention in New York City on average it works, Journal of Housing Economics 31 (2016) https://pmc.ncbi.nlm.nih.gov/articles/PMC4770906/
    • Reference Institute for Children Poverty and Homelessness, From Local Pilot to National Model HomeBase at 20 Years and Its Impact on Housing Insecure Families in New York City (2024) https://www.icph.org/reports/from-local-pilot-to-national-model-homebase-at-20-years-and-its-impact-on-housing-insecure-families-in-nyc/
    • Reference New York City Department of Homeless Services, Homebase Frequently Asked Questions, program page (2025) https://www.nyc.gov/site/dhs/prevention/homebase/homebase-faq.page

  • The deploying agency's own Office of Research & Policy Innovation published a peer-reviewed re-examination (Housing Policy Debate, 2022) of 48,450 deduplicated families with children applying 2013-2016 (58,674 family-years, over a period in which the caseload rose from about 600 cases a month in 2013 to more than 1,500 a month in some 2015 months), and documented two distortions against itself. First, score clustering at the eligibility cutoff: 4,269 cases scored 6, 10,634 scored exactly 7, and 7,756 scored 8, which its researchers wrote 'suggests that Homebase staff may be focused on getting families to that threshold so they qualify for full services' — while the agency separately asserts that the override valve reduces incentives for workers to misreport data to ensure eligibility. Second, override outcomes: 6.1% of the 58,674 applications departed from the score with mandatory supervisor approval (4.8% up to full services, 1.4% down to brief), and only 3.7% of the families moved up later applied to shelter against 25.8% of the families moved down, the paper concluding that worker judgment is less accurate than the RAQ on average; a ten-case note review attributed many downward decisions to needs beyond the program's capacity, such as families needing an apartment immediately with no funds, rather than to a judgment about risk. The same paper reported 73.9% of applications at or above the threshold, shelter application within two years at 13.7% above the cutoff against 5.9% below it (chi-square 699.98, p<.001), an area under the curve of 0.7387 for the deployed instrument against 0.7389 for the revised one, and a simulated alternative threshold of 5 raising precision from 13.7% to 15.2% at similar enrollment volume. These are agency-authored figures, published in a peer-reviewed venue; the paper's own caveats are that the observed score distribution is confounded by the clustering it documents and by low-scoring self-selection out of intake, and that outcomes compared by service tier are contaminated by the overrides being measured.

    empirical
    • Academic Mullen EJ, Ghesquiere A, Dinan K, Richard K, Periodic Evaluations of Risk Assessments Identifying Families for Homelessness Prevention Services, Housing Policy Debate 32 6, accepted manuscript hosted by the New York City Department of Social Services Office of Research and Policy Innovation (2022) https://www.nyc.gov/assets/hra/downloads/pdf/about/DSS-Resource-Corner/NYCDSS-ORPI-RAQ-Housing-Policy-Debate-Accepted-Manuscript-2022.pdf
    • Government New York City Office of Technology and Innovation, Algorithmic Tools Calendar Year 2023 updated version effective March 2024, Department of Social Services entry marked Updated in 2023 Yes with the register amendment note (2024) https://www.nyc.gov/assets/oti/downloads/pdf/reports/2023-algorithmic-tools-reporting-updated.pdf
    • Academic Farrell DC, Kuebris A, Parulkar A, Preda M, Toledo M, Reassessing Measures of Risk for Homelessness Among Families with Children in New York City, Cities, provider hosted manuscript (2023) https://www.helpusa.org/wp-content/uploads/2023/01/Reassessing-Measures-of-Risk-for-Homelessness-Among-Families-with-Children-in-New-York-City.pdf

  • Local Law 35 of 2022 (passed by the NYC Council 2021-12-15, lapsed into law unsigned 2022-01-14) added Admin. Code sec. 3-119.5, requiring every city agency to report by 31 December every algorithmic tool it used one or more times during the prior calendar year — expressly including tools that 'generate risk scores' or 'determine what resources are allocated to particular groups or individuals' — with six mandatory disclosure elements, compiled by the Office of Technology and Innovation into a public report delivered to the mayor and Council speaker each 31 March. DSS filed the Homebase RAQ in all four cycles to date (CY2022-CY2025), each entry disclosing in the agency's own words the June 2012 start date, the 2004-2008 training data analyzed with academic researchers, the input factors, the points-and-threshold eligibility mechanism, and the worker-override-with-supervisor-permission rule. The loop demonstrably fired: after initial publication of the CY2023 report the register carries the note 'Update 3/27/2024 - The Department of Social Services updated their reporting to include changes made to the Homebase Risk Assessment Questionnaire after initial publication of the report', disclosing the 2023 item revision and adding its peer-reviewed citation; the Council's Committee on Technology held an oversight hearing on the regime on 2024-10-28, at which OTI described coordinating 45 agencies plus 24 further offices. Register entries are unaudited agency self-reports. Separately, three government audits have examined this program and none examined the risk model: the city comptroller's MG12-125A (2013-06-27) found no written monitoring policies, no records of initial ineligibility determinations, and all provider risk assessments announced in advance, with the unannounced-visit recommendations rejected; the city comptroller's January 2020 audit of HRA's oversight of the then-$53M-a-year program found 80 of 240 required provider case-file reviews performed, 2,661 of 24,938 FY2018 households (11%) returning one to four times within twelve months, $2,271,797 in provider advances unrecouped some sixteen months after closeout, and 5 of 28 visited client homes not habitable (4 never fixed), issuing 19 recommendations; and the state comptroller's audit 2023-N-8 (issued 2026-01-07, scope July 2021 - July 2025) examined the downstream CityFHEPS rental-subsidy channel for DSS Homebase clients (57,888 new cases and 123,762 individuals housed since 2018 through March 2025; spending $176M in FY2019 rising to $834M in FY2024) and found units with hazardous violations approved, 30 of 75 sampled case records without evidence of income verification, and rents averaging $525 a month above comparables in eleven of thirty sampled statewide cases. None of the three may be cited as an audit of the algorithm.

    empirical
    • Government New York City Council, Local Law 35 of 2022 (Administrative Code section 3-119.5, annual agency reporting on algorithmic tools), certified City Clerk text (2022) https://intro.nyc/local-laws/2022-35
    • Government New York City Office of Technology and Innovation, Algorithmic Tools Calendar Year 2023 updated version effective March 2024, Department of Social Services entry marked Updated in 2023 Yes with the register amendment note (2024) https://www.nyc.gov/assets/oti/downloads/pdf/reports/2023-algorithmic-tools-reporting-updated.pdf
    • Government New York City Council Committee on Technology, Local Law 35 reporting on algorithmic tools, oversight hearing transcript of October 28 2024 (2024) https://citymeetings.nyc/city-council/2024-10-28-0100-pm-committee-on-technology/chapter/local-law-35-reporting-on-algorithmic-tools
    • Government Office of the New York City Comptroller, Audit Report on the Department of Homeless Services Monitoring of the Homebase Program MG12-125A (2013) https://comptroller.nyc.gov/reports/audit-report-on-the-department-of-homeless-services-monitoring-of-the-homebase-program/
    • Government Office of the New York City Comptroller, Comptroller Stringer Audit Reveals Weak City Oversight of the 53 Million Dollar Homebase Homelessness Prevention Program, audit of the Human Resources Administration (2020) https://comptroller.nyc.gov/newsroom/comptroller-stringer-audit-reveals-weak-city-oversight-of-53-million-homebase-homelessness-prevention-program/
    • Government Office of the New York State Comptroller, Administration of the CityFHEPS Program for Department of Social Services Homebase Clients, Audit 2023-N-8 (2026) https://www.osc.ny.gov/state-agencies/audits/2026/01/07/administration-cityfheps-program-department-social-services-homebase-clients

Where this connects

Institutional pressures in this domain

  • Workload surge — Demand outruns staffing; per-case attention shrinks and review becomes triage.
  • Austerity & recovery incentives — Cost-cutting and overpayment-recovery targets tilt the system toward denial and enforcement errors.
  • Vendor opacity — The deploying institution cannot inspect the model, data, or update pipeline it is accountable for.
  • Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
  • Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.

All of them in context on the Housing & homelessness services domain page.

Levers available here and the patterns behind them

Documented case histories