There's a moment in every industry where the thing everyone privately worries about becomes the thing everyone has to publicly reckon with. For AI data labeling, that moment arrived in December 2025 when Business Insider published an investigation into what they called "the shadow market for AI training accounts."

The details were specific enough to make you shift in your chair. Over a hundred groups on Facebook and Telegram, openly buying and selling verified accounts for annotation platforms. US-based accounts going for three hundred dollars. Full fraud kits, complete with residential proxy setup guides and screening test answers, selling for a thousand. YouTube channels offering tutorials on how to bypass geo-restrictions. One WhatsApp ad seeking to purchase accounts for the two biggest annotation platforms in the industry by name.

AlgorithmWatch went further. They sent a reporter undercover into one of these groups and watched, in real time, as someone tried to open a fraudulent account using a borrowed phone number and a residential proxy. The process took minutes. The platform's fraud detection systems caught nothing.

And here's where it starts to really matter: these workers are competent. They pass the screening tests because the screening tests measure skill, and they have the skill. They can write well. They can evaluate model outputs. They can do the RLHF annotation work at a high level. The problem is that one competent person is doing it across five, ten, twenty accounts. Which means the "diverse human feedback" training your model is coming from a much smaller pool of humans than you think.

The data quality crisis that already happened

This played out in public last year, in a way the industry mostly treated as a corporate drama rather than a structural warning.

In June 2025, Meta invested fourteen billion dollars in Scale AI. Within weeks, Google and OpenAI both cut ties with the company. By August, TechCrunch reported that Meta's own researchers were describing Scale AI's data as low quality and quietly shifting work to competitors like Surge and Mercor. Scale AI's internal documents, some of which had been accidentally left public in Google Docs, showed thousands of flagged accounts on a single project for Google in 2024. Spreadsheets titled things like "suspicious non-US taskers" and "Good and Bad Folks."

TIME reported that competitor CEOs described Meta's investment as the equivalent of "an oil pipeline exploding." Demand for alternative data providers tripled overnight. One competitor added fifty million dollars in potential contracts in two weeks.

The conventional reading of this story is that it was about competitive dynamics. Meta's investment compromised Scale AI's neutrality and their clients jumped ship. That's true. But there's a deeper story underneath: the entire industry is built on the assumption that each account represents a unique human, and that assumption is breaking down.

Why the current tools miss it

The platforms fighting this problem deploy serious technology. Device fingerprinting. VPN detection. IP analysis. Behavioral monitoring. Screening tests that take days to complete. All of these tools do real work. They catch bots. They catch someone logging in from a data center IP. They catch the obvious shortcuts.

What they miss is the competent human using a residential proxy. Residential proxies are grey-market services that route internet traffic through actual home IP addresses, making the user indistinguishable from a legitimate connection. VPN detection, the primary tool most platforms rely on, catches commercial VPNs. It does nothing against residential proxies, because the traffic genuinely looks like it's coming from someone's apartment.

They also miss the same person passing the qualification exam five times on five different accounts. The screening tests prove competence. They prove you can do the work. What they can't prove is whether you've already done this exact proof on four other accounts this week.

And then there's the compliance layer. OFAC prohibits payments to sanctioned regions. When someone masks their real location with a residential proxy and a platform pays them for annotation work, that creates a compliance liability. Every platform relying on location verification through IP analysis alone is exposed. The fines can reach millions per violation.

What RLHF actually needs

The whole point of RLHF, the process that turns a language model's raw capabilities into something that feels useful and aligned, is that it draws on diverse human judgment. Different people with different backgrounds and different expertise, ranking model outputs based on quality, safety, and helpfulness. The reward model that emerges from this feedback is only as good as the diversity and authenticity of the humans providing it.

When one person is operating forty accounts, you get forty copies of the same judgment. The same biases, the same blind spots, the same stylistic preferences, amplified across what your platform thinks is forty independent data points. Your model trains on what it believes is broad human consensus. It's actually training on one person's opinion, forty times over.

Research on Sybil attacks in federated learning has shown that duplicate participants can poison model training through label manipulation. The academic term is "Sybil attack," named after the case study of a woman with multiple personalities. In AI training, the damage is more subtle. The model doesn't break in obvious ways. It just becomes less robust, less diverse in its capabilities, more reflective of a narrow set of human preferences masquerading as consensus.

Data labeling costs surged 88x between 2023 and 2024 while compute costs grew only 1.3x. Each major AI lab now spends roughly a billion dollars a year on human data. When you're spending that much on human feedback, the integrity of "human" is the entire value proposition. And right now, there's a shadow market actively undermining it for three hundred dollars a pop.

The question the industry keeps avoiding

Every platform in this space is playing defense. They detect, they flag, they ban. The flagged accounts get replaced. The Telegram groups keep selling. The residential proxy services keep routing. It's the same arms race that played out in market research, in ticketing, in every industry where the value of being "a real person" exceeds the cost of faking it.

The question that needs asking is the same one we keep arriving at across every vertical we work in: why are we still trying to detect fraud after the fact instead of proving humanness at the front door?

When account creation requires verification of unique humanness, the economics of fraud collapse. You can buy an account, but you can't buy an additional face. You can pass a screening test five times, but you can't create five unique anonymous credentials. The same human, across any number of accounts, resolves to one verified person.

The internet has a bot problem. AI training has a duplicate human problem. And the longer the industry waits to solve it, the more models get trained on a fiction: that their human feedback came from humans, plural.