The Knock at the Door: How Predictive Risk Models Shape American Child Welfare
An investigative look into algorithmic risk-scoring systems used by child protective services in New York and Pennsylvania, exposing opaque surveillance and a lack of due process.
Stock photo for illustration only, not from the actual event
- Child welfare caseworker visits in America often begin with automated risk scores generated by predictive algorithms.
- Flagged families lack the legal right to view their scores or challenge the specific factors triggering an investigation.
- Investigations reveal that screening systems disproportionately target Black and low-income families.
- Predictive risk tools have expanded from early pilots in Pennsylvania to major urban agencies like New York City's ACS.
The knock comes on a Tuesday, late afternoon, when the rice is still on the hob and the youngest is doing homework at the kitchen table. A caseworker in a thin coat introduces herself, asks if she can come in, and explains that the city has received a report and is required to follow up. The mother, who has lived in the same flat for nine years and has never had a child welfare investigation in her life, asks who made the report. The caseworker hesitates. It is not exactly a report, she says. It is a flag.
This is the moment, repeated thousands of times a year across American cities, when a family discovers that they have been the subject of attention they did not know was possible. The flag did not come from a neighbour or a teacher or a paediatrician. It came from a model. A risk-scoring system, fed on years of administrative data, generated a number that crossed a threshold inside a software dashboard at the local child protective services office. A screener saw the number. A supervisor signed off. A caseworker was dispatched. Somewhere along that chain, a human being had to make a final decision, but the decision was anchored, framed, and quietly shaped by an output that nobody in the home would ever see.
Stock photo for illustration only, not from the actual event
The mother in the flat has no right to see the score. She has no right to know which features pushed her family above the threshold. She has no right to challenge any one of those features in front of a neutral reviewer. She has no right, in any meaningful sense, to know that the algorithm exists.
The deployment of predictive algorithms in social welfare highlights deep structural challenges. When administrative data such as postal codes or local service density serve as proxies for race and class, automated systems end up reinforcing existing inequalities under the guise of technical objectivity, all while bypassing transparent oversight mechanisms.
That is where American child welfare sits in the spring of 2026: an expanding lattice of predictive systems, deployed inside agencies whose decisions can place a family under state surveillance and, in the worst cases, separate parents from their children, operating almost entirely outside the procedural rights that any other consequential decision in modern life would attract. A family flagged by a credit-scoring algorithm has more statutory recourse than a family flagged by a child welfare risk model. A driver flagged by a parking enforcement camera has more transparency. A tenant flagged by an algorithmic landlord screen has more legal scaffolding to push back. The state has built one of the most invasive deployments of pattern-matching in American public administration, and it has done so on top of the thinnest possible layer of due process.
In 2025, The Markup published an investigation into the Administration for Children's Services in New York City, the agency that handles child abuse and neglect reports for roughly 1.6 million children. The investigation, drawing on internal documents and interviews with agency staff, established that ACS had been using an algorithmic risk-scoring tool to help decide which families warranted heightened scrutiny, surveillance and investigation following a hotline call. The tool, which the agency had introduced years earlier with limited public discussion, generated a score for every family entering the system, and that score informed which cases were elevated for what staff called “high-priority” review.
The Markup's reporters, working with academic researchers, found that the system disproportionately flagged Black and low-income families at rates higher than would be expected from the underlying base rates of confirmed maltreatment in those populations. The disparity was not fully explained by the data the agency claimed to be using. There were other variables, less obvious ones, that appeared to be doing meaningful work inside the model. Postcodes. Prior contact with public assistance programmes. Density of services in a neighbourhood. Each was, on its face, a non-racial input. Each, in practice, served as a proxy for race and class, because race and class in New York are written into the geography and the administrative trail of poverty.
"The tool was not deployed in secret. There were public meetings, advisory committees, journalist briefings..."
Allegheny Family Screening Tool Developers
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment