PAN Lab example
StopNCII & Take It Down
Two ways to run a hash bank: the intimate-image removal pair
StopNCII and Take It Down fingerprint a person's intimate images on their own device, so partner platforms can find copies shared without consent.
See more
StopNCII.org and Take It Down compute a hash, a digital fingerprint, of an intimate image or video on the person's own device. Only the hash leaves the device, and participating platforms compare uploads against it. SWGfL's Revenge Porn Helpline runs StopNCII for adults, and the US National Center for Missing & Exploited Children runs Take It Down.
Who each service serves
StopNCII.org launched in December 2021. SWGfL is a UK charity, and StopNCII was developed with Meta. It serves adults who hold intimate images or videos of themselves.
The National Center for Missing & Exploited Children, known as NCMEC, launched Take It Down in February 2023. It serves people who were under 18 when the material was made.
In both services, the person shown in the image or video makes the submission themselves. Nobody may submit for someone else, and the person states their own eligibility.
How it is used
The person opens the web tool and selects the image or video. The tool computes the hash on their device. The original is never uploaded, never stored, and never seen by either operator.
The tool uploads the hash to the operator's hash bank, and the operator shares it with participating platforms. Each platform's matching system, self-hosted or from a vendor such as Thorn's Safer, compares the hashes with uploads on public or unencrypted services.
A match puts the upload in front of the platform's own trust and safety reviewers. They decide under their own platform's policy whether to act. A match obliges nobody, and every removal is a decision by people at a platform.
Which hash functions it uses
StopNCII states its functions: PDQ and PhotoDNA for photos, and MD5 for videos. Independent researchers confirmed that Take It Down runs PDQ by inspecting its web tool.
PDQ is Facebook's function, released as open source in 2019. PhotoDNA is Microsoft's proprietary function.
Two designs for the person reporting
The two services run the same mechanism and differ, by design, in what they give the person reporting.
StopNCII gives the reporter a case number, a PIN, and a status page. A status update can take three to five days. StopNCII also carries out withdrawals of a whole case, against a case number and PIN that it states it does not store and cannot reset.
Take It Down is anonymous by design. It has no status page, sends no notice of matches, and offers no way to withdraw. So each service shows what the other lacks.
What the design gives and what it costs
The choice that protects the person also removes any way to check the hash bank. Keeping the image on the device means the material never leaves the person's control.
It also means nobody can check any entry against the image it came from. Nobody assesses a submission in either service, and the image exists nowhere the operator has access to.
The one place an entry can be checked against the material is a platform that already holds it. There, the exposure the design exists to prevent has already happened.
How much it is used
The figures are the operators' own. SWGfL reported on 25 November 2025 that StopNCII hashes were protecting 2 million images across more than 785,000 cases. That was a 97 percent increase over 2024, with 17 industry partners.
NCMEC reports Take It Down submissions rising from over 83,000, with over 166,000 hashes, in 2024. In 2025 there were over 130,000 submissions, covering over 273,000 images and videos.
These figures count people reporting, never how often abuse happens. No operator of this kind of service publishes an audited error rate for its system.
Who takes part
Taking part is voluntary for platforms. Coverage is a patchwork the person reporting cannot see around. As of 27 August 2026, YouTube matches Take It Down's list but not StopNCII's. X matches StopNCII's list but not Take It Down's, and Discord matches neither. Encrypted messaging is outside both by design.
Google announced on 17 September 2025 that it would begin using StopNCII hashes to remove such images from its Search results. The announcement covers Search alone, rolling out over months. Google was not on StopNCII's partner page as fetched on the same day as the lists above.
Where people stay unprotected
When this service fails, nobody is falsely accused. A person who did everything right stays unprotected somewhere the hashes do not cover.
That may be a platform that does not take part, or an encrypted service. It may be a copy that was cropped, filtered, or clipped, and no longer matches. It may be a partner's copy of the hash, which a withdrawal does not remove. And a Take It Down reporter can never be told whether a match was found.
What outside researchers found
In 2023 and 2024, researchers at Royal Holloway and partner institutions rebuilt recognizable images from PDQ, PhotoDNA, NeuralHash, and aHash hashes. They recovered hair colour and length, face shape, other facial features, and some background.
They used an off-the-shelf image generator trained on 1,000 public celebrity-face images, on 2015-era consumer hardware. On a score of how closely a rebuilt image resembled the original, the average was 60.10 percent for PDQ and 74.04 percent for PhotoDNA. They measured this on a public benchmark, never on any reporter's material. They concluded the hashes should be treated as sensitive in the same way as the original images.
A separate study at the USENIX Security conference in 2023 showed efficient attacks on PhotoDNA and PDQ. An attacker can alter an image so it matches a chosen hash, or so a listed image escapes detection.
The warning that went unanswered
The Royal Holloway team quoted Take It Down's FAQ: "No, images and videos cannot be reverse engineered or created from the hash values shared with NCMEC." They reported that answer to be wrong.
They wrote to the operator, NCMEC, twice between August and December 2023. They received no reply, and they recorded that the website did not change. The sentence was still on the FAQ page on 27 August 2026.
The new legal duties
Lawmakers in the US and the UK are adding legal duties around this voluntary service. The US TAKE IT DOWN Act's notice-and-removal duty took effect on 19 May 2026. Covered platforms must remove reported non-consensual intimate images, and known identical copies, within 48 hours of a valid request.
The Federal Trade Commission enforces the duty. It sent compliance letters to fifteen major companies, including Alphabet, Discord, and X. Not all of them take part in either hash list.
The UK made sharing such images a priority offence under the Online Safety Act in 2024. Creating them, deepfakes included, became an offence on 6 February 2026. Ofcom's hash-matching code measures were expected in force from summer 2026. The UK has also announced a 48-hour takedown amendment, with penalties up to 10 percent of worldwide turnover.
The laws require platforms to answer removal requests. They do not require joining either hash list, and they add no way to audit one.
What the available tools can and cannot address
A failure pathway is a link between two parts of the network, where a mistake made by one part can be passed on to the other. A tool closes a pathway when mistakes stop passing along it.
This case has a budget of 8 units. Explore (No Targets) sets no targets. There, and under Service Targets Only, two tools costing 5 of the 8 units are enough to keep the mistakes on this network contained. Under Service Targets Only, the same two also meet the service target. One such pair is Incident loop with Vet connections.
Under Service and Safety Targets and All Governance Targets, the targets are not fully addressable with the available tools. Within the budget, the Lab tried all 2,422 allowed combinations of the twelve tools and their settings under each of those two levels. None meets every target.
Money is not what stands in the way. With the budget set aside and every tool applied at its stronger setting, two failure pathways stay open.
One is matches sent for platform review, the one route by which anything is ever removed. The other is the researchers' examination of the public web tool, the one outside check the sources show working.
No tool offered here closes either one. Both are how the service does its work: the first is how anything gets removed, and the second is how outside testing happens. This is a finding about the deployment, not a gap in your approach.
Open this example in PAN Lab v0.1 to apply pressures and levers and watch what the system does.
What this models
This example runs on the StopNCII-class on-device hash-removal index network: 8 components and 14 pathways between them. Every context in the Lab is a stylized model, never a reconstruction of any actual deployment, and each assumption behind it carries a provenance label.
Evidence base: 3 assumed · 9 published baseline. In the Lab, the shaded evidence band behind each headline readout draws its width from the least-established class below.
Show all 12 assumptions
- assumed
This network models the kind of service that StopNCII and Take It Down both run. It models the two together, because together they give enough evidence to build the network. StopNCII, run by SWGfL's Revenge Porn Helpline, has cases, PINs, status pages, and withdrawals. Take It Down, run by NCMEC, is anonymous, with no status page and no withdrawal. The published counts are fullest on NCMEC's side. This network is not a copy of either real system.
- assumed
This is a help system, and its failures are gaps in protection. Each documented failure leaves someone unprotected. It may be a platform off the partner list, an encrypted service, or a re-encoded copy that no longer matches. It may also be a partner's copy that a withdrawal does not remove, or a status question the anonymous service never answers. None is a false accusation against anyone. A match obliges nobody, because the platform's own policy review stands between a match and any action. Nothing in this network works out an outcome for anyone the service protects, or for anyone whose upload is matched.
- baseline
The workload in this network comes from the operators' own counts. SWGfL reported 2 million images protected across more than 785,000 StopNCII cases by November 2025, a 97 percent increase over 2024. NCMEC reported Take It Down submissions rising from over 83,000 in 2024 to over 130,000 in 2025. The 2025 submissions covered over 273,000 images and videos. These are the operators' own counts of people reporting. They do not measure how often abuse happens, and nothing in this case reads them that way.
- baseline
The manual alternative to hash matching is reporting each item to each platform by hand. One independent study measured such a channel on a participating platform, X. X removed none of the images reported through its non-consensual nudity channel in 21 days. It removed every image reported as a copyright violation, in about 25 hours. That study tested X's reporting channel for victims, not hash matching, and this network uses it for that difference alone. The operators' own manual channel is slow as well. A StopNCII case status can take three to five days to change. Take It Down gives a reporter no such channel at all.
- baseline
The network draws two stores of hashes, because StopNCII's own words describe two. Hashes are shared with participating companies, and with new partners as they join. Participating companies "reserve the right to continue enforcing their policies once they've acquired knowledge of the hash." A withdrawal exists at StopNCII alone. It removes a whole case rather than an entry, and it needs credentials the operator does not store and cannot reset. It removes the bank's copy and not the partners' copies. The sources document no step that brings partners' copies back in line with the bank. No operator publishes how many partners hold a copy, how long they keep it, or what a partner does with a withdrawn hash.
- baseline
Hashing on the reporter's device keeps the image in their hands. The same choice means nobody can check any entry in the bank against the image it came from, because that image exists nowhere the operator has access to. The one place an entry can be checked against the material is a platform that already holds it. There, the exposure the design exists to prevent has already happened. So the network puts no substantive check on a submission, only the mechanical screen for format and errors. The platform's own review is the one substantive check anywhere in the system.
- baseline
The network marks the pathways that carry hashes as sensitive for privacy, following a peer-reviewed independent evaluation. Researchers rebuilt recognizable images from hashes of the kinds this service uses. They recovered hair colour and length, face shape, other facial features, and some background. On a score of how closely a rebuilt image resembled the original, the average was 60.10 percent for PDQ and 74.04 percent for PhotoDNA. They concluded the hashes should be treated as sensitive in the same way as the original images. They measured this on a public benchmark of celebrity faces with an off-the-shelf image generator, never on any reporter's material. The figures describe how far this kind of hash can be reversed, not any person's exposure.
- baseline
The researchers' access to the tool, and their channel into the operator, are drawn from what the sources show. Their whole access was inspecting Take It Down's public web tool. They wrote to the operator twice between August and December 2023, received no reply, and recorded that the website did not change. The FAQ sentence their result contradicts says material cannot be reverse engineered from the hash values. It was still on the page when it was checked again on 27 August 2026.
- baseline
The network links the tool on the device with every partner's matching system, because a match needs the same hash functions at both ends. So a documented weakness of those functions holds everywhere at once, not at one desk. Two security evaluations document such weaknesses. One showed attacks on PhotoDNA and PDQ that make an image match a chosen hash, or escape detection. The other showed the inversion result: hashes turned back into recognizable images. These are ways matching can go wrong or be tampered with. Nothing about them accuses any person.
- baseline
Partner lists change, so every statement about them carries its date. As fetched on 27 August 2026, StopNCII listed 17 partners and Take It Down listed 12. The two lists overlap and differ. YouTube matches Take It Down's list and not StopNCII's. X matches StopNCII's list and not Take It Down's, and Discord matches neither. StopNCII's own pages disagree with its own milestone article on four names. Google announced in September 2025 that it would use StopNCII's hashes in Search. That announcement covers Search alone, and Google was not on the partner page as fetched. This network treats Google as an announcement, never as a listed partner.
- baseline
The 2026 legal duties are left out of the network on purpose. In the US, the TAKE IT DOWN Act's notice-and-removal duty took effect on 19 May 2026, and the Federal Trade Commission enforces it. Covered platforms must remove reported images within 48 hours of a valid request. That duty answers one removal request at a time. In the UK, Ofcom's hash-matching code measures were expected in force from summer 2026. These mandatory duties work independently of the voluntary hash lists, and they do not require joining one. Treating the legal duty as if it were hash matching is an error the sources specifically warn against. The duties are described in the case file. They surround this voluntary service with mandatory legal duties, and they leave the hash lists as impossible to audit as before.
- assumed
The people the service exists to protect are outside this network's workings. That covers people who hold intimate material of themselves and submit hashes of it. It also covers people whose uploads are matched. Nothing drawn here works out an outcome for either group. The two reconstruction figures for the two image functions are outside measurements, kept in the case file with their benchmark warning. What this network traces is how mistakes pass between the case desks, the matching systems, the platform reviewers, and the researchers.
What this example does not show
Show all 7 limitations
- Every volume figure here is the operators' own report. SWGfL reported 2 million images protected and over 785,000 StopNCII cases by November 2025. NCMEC reported over 130,000 Take It Down submissions covering over 273,000 images and videos in 2025. These count people reporting, never how often abuse happens, and no prevalence figure is taken from them. No measure of how well the service works exists anywhere in the sources. No operator of this kind of service publishes an audited error rate for its system. StopNCII's site also carries a separate removal claim of "over 90 percent". That figure describes its parent helpline's casework, not the hash service, and the Lab does not use it anywhere.
- The hash reversal figures come from a public benchmark of celebrity faces, never from any reporter's material. The average similarity was 60.10 percent for PDQ and 74.04 percent for PhotoDNA. The researchers used an off-the-shelf image generator. The figures describe how far this kind of hash can be reversed, not any person's exposure. They always appear here with that warning.
- The study cited for manual reporting tested a participating platform's reporting channels for victims. It did not test hash matching. On X, copyright reports led to removal of every test image in about 25 hours. Reports through X's non-consensual nudity channel led to no removals in 21 days. This case uses the study for one point alone: a platform can take part in hash matching while its channel for victims does not work. It is never a measure of the hash matching.
- This is a help system, and every documented failure is a gap in protection. It may be a platform that does not take part, an encrypted service, or a re-encoded copy. It may be a partner's copy that a withdrawal does not remove, or a status question the anonymous service never answers. Nothing in the sources is a false accusation. A hash match leads to the platform's own policy review, not to an automatic penalty.
- Partner lists change, and the record shows StopNCII's own pages disagreeing with each other. Every partner statement here is as fetched on 27 August 2026, and none should be read as stable. Google announced in September 2025 that it would use StopNCII's hashes in Search. That use is limited to Search, and it was announced as rolling out over months. Google was not on the partner page as fetched. It is an announcement, not a listed partner.
- The 2026 legal duties are a separate, mandatory legal channel. In the US, the TAKE IT DOWN Act's notice-and-removal duty took effect on 19 May 2026, and the Federal Trade Commission enforces it. It answers one removal request at a time. In the UK, sharing intimate images without consent is a priority offence, and creating them is an offence. Ofcom's hash-matching code measures were expected in force from summer 2026. These duties work independently of the voluntary hash lists, and they do not require joining them. Nothing here treats the Act's 48-hour duty as hash matching.
- The people served are not modeled. That covers the people the two services protect, and the people whose uploads are matched. The Lab traces how mistakes pass between the organisations and teams that run the service: the case desks, the matching systems, the platform reviewers, and the researchers. It works out no outcome for anyone outside them.
Sources and evidence
What this example rests on, claim by claim. Every entry resolves to the same ledger the Evidence Registry publishes.
StopNCII.org (operated by the Revenge Porn Helpline within the UK charity SWGfL, developed with Meta, launched December 2021) and Take It Down (operated by the US National Center for Missing & Exploited Children, launched February 2023) run one on-device hash-removal mechanism in two configurations: the person who holds the material generates a hash on their own device, only the hash leaves the device, the original is never uploaded or stored, third-party submission is refused and eligibility is self-attested, and participating platforms match the hash against uploads on public or unencrypted surfaces, reviewing any match under their own policies. StopNCII states its algorithms as PDQ and PhotoDNA for photos and MD5 for videos; independent researchers verified by inspecting the Take It Down web client that it runs PDQ. The operators' own volume reports — advocacy-tier counts of reporting behaviour, never incidence — were 2 million images protected across more than 785,000 cases by November 25, 2025 on the StopNCII side (a reported 97 percent increase over 2024, 17 industry partners) and 130,000+ submissions covering 273,000+ images and videos in 2025 on the Take It Down side, up from 83,000+ submissions in 2024. As fetched 2026-08-27 the two partner rosters overlap and diverge (YouTube on the minor index only, X on the adult index only, Discord on neither), encrypted surfaces are outside both by design, and Google announced on September 17, 2025 that it will use StopNCII hashes in Search — an announcement scoped to Search results, with Google absent from the StopNCII partners page as fetched the same day.
empirical- Advocacy StopNCII.org (Revenge Porn Helpline / SWGfL). Service pages: How It Works, Industry Partners, Frequently Asked Questions https://stopncii.org/
- Advocacy National Center for Missing & Exploited Children, "Take It Down" (minor-focused hash removal service); with the NCMEC service page. https://takeitdown.ncmec.org/
- Advocacy National Center for Missing & Exploited Children. CyberTipline Data (2025 report overview, including Take It Down volumes) https://www.missingkids.org/gethelpnow/cybertipline/cybertiplinedata
- Academic Hawkes, S., Weinert, C., Almeida, T., & Mehrnezhad, M. (2024). Perceptual Hash Inversion Attacks on Image-Based Sexual Abuse Removal Tools. IEEE Security & Privacy Magazine https://pure.royalholloway.ac.uk/ws/portalfiles/portal/63677133/TiD_OA.pdf
- Advocacy SWGfL (2025, November 25). StopNCII.org Being Used to Protect 2,000,000 Images Online in the Fight Against Intimate Image Abuse https://swgfl.org.uk/magazine/stopncii-org-being-used-to-protect-2-000-000-images-online-in-the-fight-against-intimate-image-abuse/
- Vendor Google (2025, September 17). Developing a new partnership to combat non-consensual intimate imagery on Search (The Keyword blog) https://blog.google/products/search/stopncii-program-partnership/
A peer-reviewed independent evaluation (IEEE Security & Privacy Magazine 2024) reconstructed recognizable pre-images — hair colour and length, face shape, other facial features, some background — from PDQ, PhotoDNA, NeuralHash, and aHash hashes using an off-the-shelf conditional image-to-image generative network trained on 1,000 public celebrity-face images on 2015-era consumer hardware, with mean perceptual similarity of 60.10 percent for PDQ and 74.04 percent for PhotoDNA, measured on a public celebrity-face benchmark and never on any reporter's material; the authors concluded the hashes should be treated as sensitive in the same way as the original images. The same team quoted Take It Down's FAQ answer that material 'cannot be reverse engineered or created from the hash values shared with NCMEC', reported that answer to be wrong, wrote to the operator twice between August and December 2023, received no reply, and recorded that the website did not change; the sentence was still on the FAQ page on 2026-08-27. A separate USENIX Security 2023 evaluation demonstrated efficient targeted second-preimage and detection-avoidance attacks against PhotoDNA and PDQ, concluding existing perceptual hash functions are likely insufficiently robust for adversarial settings. No operator in this class publishes an audited error rate for its deployed system.
empirical- Academic Hawkes, S., Weinert, C., Almeida, T., & Mehrnezhad, M. (2024). Perceptual Hash Inversion Attacks on Image-Based Sexual Abuse Removal Tools. IEEE Security & Privacy Magazine https://pure.royalholloway.ac.uk/ws/portalfiles/portal/63677133/TiD_OA.pdf
- Academic Prokos, J., Fendley, N., Green, M., Schuster, R., Tromer, E., Jois, T., & Cao, Y. (2023). Squint Hard Enough: Attacking Perceptual Hashing with Adversarial Machine Learning. 32nd USENIX Security Symposium https://www.usenix.org/conference/usenixsecurity23/presentation/prokos
- Advocacy National Center for Missing & Exploited Children, "Take It Down" (minor-focused hash removal service); with the NCMEC service page. https://takeitdown.ncmec.org/
The two configurations diverge on the victim channel by design, and both operators document the propagation surface a withdrawal does not reach. StopNCII gives a reporter a case number, PIN, and status page (updates can take 3 to 5 days) and services whole-case withdrawal against credentials the operator states it does not store and cannot reset; Take It Down is anonymous by construction, with no status channel, no notification of matches, and no withdrawal mechanism at all. StopNCII states that hashes persist after the reporter deletes the image and are shared with new partners as they join, and its FAQ states that participating companies 'reserve the right to continue enforcing their policies once they've acquired knowledge of the hash' — so an entry's reach grows after submission and a withdrawal retracts the bank's copy but not partner-held copies. A hash match obliges nobody: matching platforms review the content against their own policies before any action, platform-side matching is deployed self-hosted or by vendor API, and every documented failure of this protective system is a protection gap — a non-participating platform, an encrypted surface, a re-encoded copy that no longer matches, a partner copy a withdrawal cannot reach — never a false accusation.
empirical- Advocacy StopNCII.org (Revenge Porn Helpline / SWGfL). Service pages: How It Works, Industry Partners, Frequently Asked Questions https://stopncii.org/
- Advocacy National Center for Missing & Exploited Children, "Take It Down" (minor-focused hash removal service); with the NCMEC service page. https://takeitdown.ncmec.org/
- Vendor Thorn, "Safer" (CSAM detection service for platforms). https://safer.io/
A University of Michigan audit study (ACM CSCW 2026) posted 50 synthetic-persona deepfake nude images to X and reported half through X's non-consensual-nudity mechanism and half as DMCA copyright violations: the DMCA reports achieved 100 percent removal within about 25 hours (mean 20.3 hours), while the non-consensual-nudity reports achieved zero removals over 21 days. The study tested X's victim-facing report channels, not hash matching, and X is a StopNCII partner — independent evidence that a platform's participation in a hash program can coexist with a non-functioning victim-facing report channel, which is why the match channel and the platform report channel are modeled separately.
empirical- Academic Zhang and colleagues, Reporting Non-Consensual Intimate Media: An Audit Study of Deepfakes (ACM CSCW 2026; read in the arXiv 2409.12138 preprint version) https://arxiv.org/html/2409.12138v1
The voluntary hash indexes acquired a mandatory legal overlay during 2024-2026, on separate tracks that operate independently of hash-program membership. In the US, the TAKE IT DOWN Act's Section 3 notice-and-removal duty took effect May 19, 2026: covered platforms must remove reported non-consensual intimate imagery and known identical copies within 48 hours of a valid request, enforced by the FTC, which opened a complaint portal and sent compliance letters to fifteen major companies including Alphabet, Discord, and X — companies that do not all participate in either voluntary hash index. In the UK, NCII sharing became a priority offence under the Online Safety Act via 2024 regulations, a creation offence covering purported intimate images including deepfakes came into force February 6, 2026 under the Data (Use and Access) Act 2025, Ofcom's hash-matching code measures were expected in force from summer 2026, and a Crime and Policing Bill amendment announced February 19, 2026 would require 48-hour takedown with penalties up to 10 percent of worldwide turnover. The statutes mandate responding to removal requests; they do not require joining either hash index, and they leave the indexes' unauditability untouched.
empirical- Government Federal Trade Commission (2026, May). FTC Begins Enforcing the TAKE IT DOWN Act (press release) https://www.ftc.gov/news-events/news/press-releases/2026/05/ftc-begins-enforcing-take-it-down-act
- Reference Fladgate LLP (2024). UK Government set to strengthen Online Safety Act to combat non-consensual intimate image sharing https://www.fladgate.com/insights/uk-government-set-to-strengthen-online-safety-act-to-combat-non-consensual-intimate-image-sharing
- Reference Osborne Clarke (2026, February). Digital regulation | UK Regulatory Outlook February 2026 https://www.osborneclarke.com/insights/regulatory-outlook-february-2026-digital-regulation
Where this connects
Institutional pressures in this domain
- Reviewer bottleneck — One fixed-capacity checking stage sits between AI output and consequence; everything queues behind it.
- Austerity & recovery incentives — Cost-cutting and overpayment-recovery targets tilt the system toward denial and enforcement errors.
- Compliance over substance — Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.
- Vendor opacity — The deploying institution cannot inspect the model, data, or update pipeline it is accountable for.
- Data & policy drift — The world, the intake process, and the rules change under a system trained on how things used to be — two mechanisms with different remedies: the statistical properties of what the system processes move (concept drift), or the mixture of inputs arriving in deployment differs from the mixture it was trained on (covariate shift).
All of them in context on the Content moderation & editorial AI domain page.
Levers available here and the patterns behind them
- Upgrade model — Improve the model
- Check with a second model — Cross-model verification
- Gate record entries — Human-in-the-loop write gating
- Mark AI-written records — Provenance labeling
- Store less data — Data minimization
- Vet connections — Connection authorization
- Check copied records — Reconcile copied records
- Gate vendor updates — Vendor quality gate
- Review on schedule — Oversight cadence & retrospectives
- Understand the system — Understand the system
Documented case histories
- StopNCII & Take It Down
- The errors that became visible when the reviewers went home
- The most built-out correction structure and the reach it doesn't have
- The byline nobody was behind
- A staff byline the AI wrote and the review it implied
- X Multilingual Hate-Speech Enforcement
- X Community Notes (crowd annotation)
- GIFCT hash-sharing database
- Google CSAM detection and total account closure
- Meta cross-check: the enforcement-exemption tier
- The CyberTipline: triage under a rule against looking
- Sama Nairobi: the review workforce as the governed subsystem
- TikTok EU and UK trust-and-safety staffing substitution
- The score is published and the service cannot act on it
- YouTube Content ID