The Letter That Changes Everything
Imagine receiving a notice in the mail informing you that your Medicaid coverage has been terminated — not because a caseworker reviewed your file, not because your income changed, but because a computer system flagged an anomaly in your data profile and generated a termination without human review. You have thirty days to appeal. The appeal process requires documentation you may not have, submitted through a system you may not be able to navigate, adjudicated by an agency operating under caseload pressures that make thorough review unlikely. In the meantime, you are uninsured.
This is not a hypothetical. It is the lived experience of hundreds of thousands of Americans in states that have adopted automated eligibility determination and fraud-detection systems for public benefits programs — and it is accelerating.
The Arkansas Experiment
The most extensively documented case of algorithmic benefits administration failure involves Arkansas's implementation of a computer algorithm to determine Medicaid eligibility for home- and community-based care recipients. Beginning in 2016, the state deployed a scoring system developed by a private contractor to assess the care needs of elderly and disabled Medicaid beneficiaries. The algorithm produced results that defied comprehension to the people affected: patients with serious, documented medical conditions saw their approved care hours slashed — in some cases by more than half — with no explanation beyond a number generated by a process neither they nor their caseworkers could interpret.
A federal class-action lawsuit, Ledgerwood v. Juster, resulted in a landmark 2019 ruling by the Eighth Circuit Court of Appeals affirming that Arkansas had violated recipients' due process rights by failing to provide adequate notice of why the algorithm had cut their benefits. The court found that the state's notices were constitutionally insufficient because they could not explain the reasoning behind algorithmic decisions that the system itself was not designed to make transparent. Arkansas was ordered to reform its process. The algorithm, in various forms, remained in use.
Arkansas was not an isolated experiment. It was a preview.
Idaho, Michigan, and the National Pattern
In Idaho, an automated Medicaid eligibility system implemented as part of the state's transition to managed care generated a wave of erroneous termination notices in 2021 and 2022, cutting off coverage for recipients who had not experienced any change in circumstances. Advocacy organizations including Disability Rights Idaho documented cases of individuals with developmental disabilities and chronic conditions losing access to care as a direct result of system errors. State officials acknowledged the problem — and pointed to the vendor.
In Michigan, a decade-long saga involving the state's MiDAS unemployment fraud detection system offers perhaps the starkest illustration of what algorithmic governance without accountability looks like at scale. Between 2013 and 2015, the system — which operated without human review of its fraud determinations — falsely accused more than 40,000 Michigan residents of unemployment insurance fraud, triggering automatic benefit termination, financial penalties, and in some cases, criminal referrals. The system's false positive rate was later determined to be approximately 93 percent. The state eventually paid out more than $20 million in settlements. The damage to workers who had their wages garnished, their tax refunds seized, and their credit destroyed in the interim was not so easily remedied.
These are not isolated technical failures. They are predictable consequences of deploying systems optimized for cost reduction and fraud detection in contexts where the cost of error falls entirely on people who cannot afford to absorb it.
The Racial Architecture of Algorithmic Denial
Algorithmic bias in public benefits systems is not merely a technical problem — it is a political one, because the data on which these systems are trained reflects the historical inequities of the systems they are designed to administer. When an automated eligibility system is trained on decades of benefits data generated by agencies that administered programs unequally along racial lines, the algorithm learns to replicate those inequalities at scale and at speed.
Researchers at the AI Now Institute and the Georgetown Center on Poverty and Inequality have documented how automated decision systems in public benefits contexts consistently produce disparate outcomes for Black and Latino recipients — higher rates of flagging for fraud review, higher rates of benefit termination, lower rates of successful appeal — even when controlling for income and benefit type. These disparities are not incidental. They are structural, embedded in training data that reflects decades of discriminatory administration, and amplified by the opacity of systems whose reasoning cannot be examined, challenged, or corrected through ordinary administrative processes.
The political context matters here. The expansion of algorithmic surveillance in welfare administration has been most aggressive in states with Republican-controlled legislatures and governors' offices — states where reducing the beneficiary rolls is an explicit policy objective, and where the algorithm provides a politically convenient mechanism for achieving that objective while maintaining the fiction of neutral, technocratic administration. The machine did not cut your benefits. The data did. The system did. No one is responsible.
Efficiency as Cover Story
The public rationale for automated benefits administration is fraud prevention and administrative efficiency. Both arguments deserve scrutiny. On fraud: the Government Accountability Office has consistently found that improper payments in SNAP and Medicaid — the category that includes both fraud and administrative error — are dominated by administrative error, not intentional fraud. The overpayment rate in SNAP attributable to recipient fraud is estimated at less than 1 percent of total program expenditure. The algorithms deployed to combat this fraction of a percent generate error rates that, in documented cases, have exceeded 90 percent — meaning that for every genuine fraud case identified, dozens of eligible recipients are wrongly denied.
On efficiency: automated systems do reduce the cost of human caseworker time. They transfer that cost — in the form of wrongful termination, administrative appeals, emergency medical care, food insecurity, and legal proceedings — onto recipients and, ultimately, onto the public. The efficiency is real. The accounting is selective.
Due Process in the Age of the Algorithm
The constitutional framework governing public benefits was established in Goldberg v. Kelly (1970), in which the Supreme Court held that welfare recipients have a due process right to an evidentiary hearing before benefits are terminated. What the Court could not have anticipated was a world in which the termination decision is made by a system whose reasoning is proprietary, whose error rate is undisclosed, and whose output arrives as a fait accompli with a thirty-day appeal window.
The due process problem with algorithmic benefits administration is not merely procedural — it is epistemological. You cannot meaningfully challenge a decision whose basis you cannot access. When a caseworker denies your claim, you can ask why. When an algorithm denies your claim, the state can tell you what the system produced but often cannot tell you why it produced it — because the model's internal logic is either proprietary, technically opaque, or both. This is not a minor administrative inconvenience. It is a structural denial of the right to a meaningful hearing that Goldberg was designed to protect.
What Accountability Actually Requires
The Center on Budget and Policy Priorities, the Electronic Privacy Information Center, and a growing coalition of civil rights and legal aid organizations have called for a set of minimum standards for algorithmic decision-making in public benefits contexts: mandatory human review before any adverse action, plain-language explanations of all eligibility determinations, public disclosure of algorithm error rates and demographic impact data, independent auditing of vendor systems, and robust legal aid support for recipients navigating the appeals process.
These are not radical demands. They are the minimum conditions under which a system that makes life-altering decisions about the country's most economically vulnerable residents can claim to operate with even basic democratic legitimacy.
States have been permitted to outsource the administration of constitutional rights to private vendors operating proprietary systems with no public accountability. The federal government — through the Centers for Medicare and Medicaid Services and the USDA's Food and Nutrition Service — has the authority to condition program funding on algorithmic accountability standards. It has not used that authority with anything approaching the urgency the situation demands.
When a government deploys a machine to decide who eats and who receives medical care, and then hides behind the machine's opacity when it gets it catastrophically wrong, that is not a technology problem. That is a justice problem — and the people paying the price are the ones who can least afford to.
