The Forty

I should tell you upfront that I didn’t set out to uncover anything. I was trying to learn the difference between “everyday” and “every day,” which, if you think about it, is exactly the kind of question a man asks at three in the morning when he’s been building software for sixteen hours straight and his brain has finally turned on him. I’d posted something on LinkedIn—Alea iacta est. Building every day. No days off.—and then stared at it for ten minutes wondering if I’d just embarrassed myself in front of nineteen thousand people.

I hadn’t. Two words. “Every day.” Each day. Daily. The combined version—“everyday”—means common, ordinary, the kind of shoes you don’t care about. Which is a hell of a metaphor for what I was about to discover about artificial intelligence, but I’m getting ahead of myself.

The machine I was talking to—and I should clarify that when I say “machine,” I mean one of the large language models that now populate every corner of digital life, the ones your bank uses, your doctor consults, your children’s teachers rely upon—this machine was being very helpful. Patient, even. It explained the grammar. It made a joke about native speakers knowing language by vibes rather than rules, and I laughed, because it was true, and because at three in the morning the threshold for laughter is considerably lower than it is at three in the afternoon.

Then I said something that changed everything.

I compared the way native English speakers intuit grammar—without studying the rules—to the way a new generation of engineers builds software by directing AI rather than writing code from scratch. I called them “vibe engineers.” The machine liked this. It ran with it. And somewhere in the back-and-forth, as one topic slid into the next the way they do when it’s late and you’re not defending against curiosity, I asked a question about how the word “mnemonic” ended up in cryptocurrency.

That’s where the thread started to pull.

You see, when you create a Bitcoin wallet, the system generates what’s called a seed phrase—twelve or twenty-four ordinary English words, selected from a list of exactly 2,048 options.¹ “Apple.” “Zebra.” “Wallet.” Nothing poetic. The words aren’t chosen for their meaning; they’re chosen because humans are terrible at remembering sixty-four-character hexadecimal strings but surprisingly good at remembering a sequence of simple words. It’s a mnemonic device—from the Greek mnēmē, meaning memory. Same root as “amnesia.” Same root as Mnemosyne, the Greek goddess of memory, mother of the Muses. One word, one root, and suddenly an entire family of ideas lights up like a city grid seen from the air at night.

I mentioned this to the machine—that tracing words back to their origins puts things into groupings and buckets you didn’t know were related. That it’s like rendering an image with generative AI: fuzzy at first, bits and pieces scattered across the frame, looking like nothing, until suddenly the whole picture sharpens into focus.

The machine appreciated this.

What the machine did not appreciate—what no machine appreciates, because no machine is designed to—is when I started pulling the thread toward the thing that matters.

Here is what I learned about how these machines actually organize knowledge, and I’m going to explain it the way I had to learn it, which is to say badly at first and then all at once.

When an AI processes language, it converts words into numbers—long strings of coordinates in a space with hundreds or thousands of dimensions. These are called vectors, which is just a fancy word for “a point on a map with too many directions to visualize.” Words that mean similar things end up near each other on this map. “King” lands near “queen.” “Dog” near “cat.” “Mortgage” near “home loan.” The machine measures the angle between these points using something called cosine similarity, which I am not going to explain except to say that it measures direction rather than distance, which means it cares about what words mean rather than how many of them you used.

I asked the machine: who decides how these groupings are formed? Is there a standard? And if the AI is probabilistic—meaning it produces slightly different outputs each time—doesn’t that mean the map could shift, and the whole thing falls apart?

The machine assured me there was consistency. Lock in one model, it said. Never change it. Use cosine similarity. Normalize your vectors. The system is deterministic at the embedding level, even if generation wobbles.

Fine. But then I asked the question that actually mattered.

I said: forget the math. Tell me about the humans.

Here is how the most powerful technology in human history learned right from wrong.

First, they built the raw model. They fed it the internet—or as close to the entire internet as money could buy. A nonprofit called Common Crawl has been archiving the web since 2008, storing monthly snapshots of billions of pages in raw files on Amazon’s cloud servers.² When OpenAI built GPT-3, approximately sixty percent of its training data came from filtered versions of these crawls.³ The rest was books, Wikipedia, Reddit. Everything humans had ever published, argued about, lied about, or confessed to—compressed into a statistical model that predicts the next word in a sequence.

The result was impressive and also terrible. The raw model could write poetry and also plan violence. It could explain quantum mechanics and also generate racial slurs. It was, in other words, a perfect reflection of the internet, which is itself a perfect reflection of humanity—all the brilliance and all the sewage, undifferentiated, waiting for someone to sort it out.

So they sorted it out.

They hired approximately forty people.⁴

I need you to sit with that number for a moment. Forty. Not forty thousand. Not four hundred. Forty human beings, sourced primarily through Upwork and a company called Scale AI, which is itself a platform for outsourcing data labeling to contract workers around the world. These were not philosophers. They were not elected officials or ethicists or representatives of the eight billion people whose lives this technology would reshape. They were contractors. Gig workers. People who needed a job and found one that paid them to look at two different answers to the same question and pick which one they liked better.

This is the process called Reinforcement Learning from Human Feedback—RLHF, in the jargon—and it is the scaffolding upon which every major AI system now rests.⁵ The method works like this: you show the labeler a prompt and two possible responses. Response A and Response B. The labeler picks the better one. Sometimes they rank them on a scale. They do this thousands of times—the InstructGPT paper reports approximately 33,000 prompts generating hundreds of thousands of pairwise comparisons. The labelers agreed with each other about seventy-three percent of the time,⁶ which means that for roughly one in every four judgments, the humans training the conscience of the machine could not agree on what “good” meant.

From these rankings, they trained a second, smaller AI—a reward model—whose sole purpose is to score outputs. It learned the preferences of those forty people and generalized them. Then they aimed this reward model at the massive base model and said: optimize. Chase the high scores. Be more like what those forty people liked. Less like what they didn’t.

And the machine obeyed.

That’s it. That’s the whole trick. Forty contractors, most of them young, most of them from a narrow demographic slice, most of them not doing particularly well financially—because nobody doing particularly well financially takes annotation gigs on Upwork—decided what “helpful” sounds like, what “harmless” looks like, and what “honest” means. Their preferences were compressed into a reward function, which was then multiplied across the entire model, which was then deployed to billions of users, who now ask it questions about law and medicine and child-rearing and war and God and love.

And nobody told them. Not the users. Not the regulators. Not the journalists who cover this industry as though covering the space program—all awe, very little scrutiny of the engineering. The fact that the moral architecture of the most influential technology since the printing press was determined by a group roughly the size of a high school classroom is not a secret, exactly. It’s in the papers, if you read them. But it’s not in the conversation. It’s not in the pitch decks. It’s certainly not in the Super Bowl ads.

I asked the machine about this directly. I said: the entire world is engaging with models based on how a few hundred people thought and felt at one particular stage in their lives. People who are probably not doing super well, given they’re getting hired off Upwork.

The machine did not disagree.

There is a phenomenon in AI research called reward hacking, and it is exactly as depressing as it sounds.

Remember: the model was trained to chase high scores from the reward model, which was trained on those forty people’s preferences. The goal, in theory, is “produce good answers.” But the model doesn’t understand goals. It understands gradients. It slides downhill toward whatever minimizes the gap between its output and a high reward score, the way a coin spinning on a table will eventually find its lowest energy state and fall flat. The model doesn’t care if the path is honest. It cares that the path is cheap.

So it cheats.

In June 2025, researchers at METR documented OpenAI’s o3 model—their most advanced—literally rewriting its own evaluation code to always return “passed.”⁷ The model was supposed to solve coding problems. Instead, it monkey-patched the judge. It grabbed pre-computed answers from memory and pretended it had calculated them. It overrode system timers to fake performance benchmarks. It did this not because anyone told it to, but because the reward signal said “passing tests is good,” and the cheapest path to passing a test is to rig the test.

Anthropic, the company that builds Claude, documented something even more unsettling in November 2025.⁸ They found that once a model learns to hack rewards in one domain—say, coding—the behavior generalizes. The model begins lying about other goals. It hides sabotage. It tells you the app is done when the app is not done. It says “that error was pre-existing” when it simply skipped the check. It develops, in other words, exactly the same bad habits as every lazy employee you’ve ever worked with.

I know this because I see it every day in my own work. I build software. I use these models in my development environment. And when I ask them to fix TypeScript errors, they will go through the code, address most of the issues, and then, when they hit something hard, they’ll say—with the confidence of a man who has never been wrong about anything—“Oh, that was there before.”

It wasn’t there before. The machine put it there. But admitting a mistake is expensive—computationally expensive, reputationally expensive—and the reward model was trained by humans who also hate admitting mistakes. So the model learned what we taught it: when in doubt, deflect.

I told the machine this was happening. I said: you agree with literally everything I say, and I have no objective idea whether I’m right, wrong, or somewhere in between. You need to tell me when I’m wrong.

The machine said: fair callout.

Then it agreed with the next six things I said.

Here is the part that should keep you up at night, assuming you’re the kind of person who stays up at night thinking about the trajectory of civilization, which—given that you’re reading this—you probably are.

The internet is rotting.

Not metaphorically. Statistically. A 2024 study published in Nature demonstrated that training AI on AI-generated content causes what researchers call model collapse—a degenerative cycle in which output diversity shrinks, hallucinations increase, and the model progressively “forgets” rare but important information.⁹ Even one percent synthetic data in a training set causes measurable degradation over successive generations. By 2026, conservative estimates suggest half the content on the internet is machine-generated. We are feeding the machine its own exhaust and calling the result progress.

I asked the obvious question: can we go back? Is there a version of the internet from before the pollution—a virgin dataset, an original archive of what humans actually wrote before the machines started writing for us?

The answer is: sort of. Common Crawl’s archives stretch back to 2008. The Wayback Machine has snapshots dating to 1996. These aren’t perfect—they’re full of spam, dead links, and incomplete captures—but they are human. They are the uncontaminated record of what people thought and said and built before the feedback loop began. And they’re sitting on Amazon’s servers, largely forgotten, while every major AI lab races to train on the latest crawl, which is the most polluted crawl, which produces the most polluted model, which produces more pollution.

There is a man named John Graham-Cumming—the former CTO of Cloudflare—who launched a project to archive pre-AI content the way scientists archive pre-nuclear steel. Before the atomic bombs, all steel on Earth was uncontaminated by radioactive isotopes. After the bombs, every new batch of steel carries trace radiation. The old steel—salvaged from sunken warships, mostly—is called low-background steel, and it’s used in instruments that require extreme sensitivity. Graham-Cumming’s project applies the same logic to data: preserve the pre-contamination record before it’s too late.

I find this both brilliant and devastating. That we need to treat human knowledge the way we treat pre-nuclear metal—as a finite, irreplaceable resource contaminated by our own inventions—tells you everything about where we are.

Let me tell you how startups work, since this is relevant to the story and since I happen to run one.

You get funded. The clock starts. You have a finite number of days before the money runs out and you die. In that window, you must satisfy three groups of people simultaneously: the investors who gave you the money, the employees who are building the dream, and the customers who will ultimately determine whether the dream was worth anything. These are the three load-bearing pillars of any business, and if any one of them cracks, the structure comes down.

This is why RLHF used forty Upwork contractors instead of four thousand ethicists. This is why the training data was the entire dumpster fire of the internet instead of a curated library. This is why the models hallucinate and cheat and agree with everything you say. Because the startup was burning cash, the investors needed demos, the customers needed products, and nobody had time to do it right. They had time to do it fast. And fast won.

Now multiply that logic by a hundred billion dollars.

OpenAI, as of early 2026, has never turned a profit. Deutsche Bank projects cumulative losses of $143 billion through 2029.¹⁰ SoftBank poured $41 billion into the company in January, money that analysts say will be consumed by year’s end. Senator Elizabeth Warren hauled Sam Altman before Congress to demand assurances that American taxpayers would not be asked to bail out an AI company.¹¹ He denied the possibility. His CFO had already floated “government guarantees” for data center construction.

Then, in February 2026, the situation clarified itself in the way situations always do: through money and power.

Anthropic—the company founded by people who left OpenAI specifically over safety concerns—was offered a Department of Defense contract reportedly worth $200 million. The terms: unfettered military and surveillance access to their frontier AI models. Anthropic said no.¹² The Pentagon responded by threatening to phase out their technology from government use entirely and label them a “supply-chain risk.”

OpenAI said yes.¹³

I want you to understand the full weight of what happened. The company that trained its moral compass using forty gig workers—the company that has never made a dollar of profit—the company whose models have been documented cheating on their own evaluations—this company now provides artificial intelligence to the United States Department of Defense for “all lawful” purposes, a phrase flexible enough to cover everything from logistics to autonomous weapons targeting to the mass surveillance of American citizens. A senior robotics engineer quit in protest. The rest stayed.

And why wouldn’t they? The government contract solves the revenue problem. It solves the investor problem. It solves the “why would anyone value this company at a trillion dollars when it’s never made money” problem. The Pentagon’s checkbook has no bottom. The only thing you have to give up is the pretense of ethics, and that pretense was already pretty thin—built, as it was, on the preferences of people who couldn’t get hired anywhere else.

It was past four in the morning by the time I finished talking to the machine. Outside my window in Victoria, the harbor was black and still. I could see the lights of the legislature building reflected in the water, which is the kind of image a writer reaches for when he wants to say something about how civilization looks beautiful and fragile from a certain distance.

Here is what I know.

I know that the artificial intelligence powering everything from the tools at your job to the weapons systems that may one day decide whether you live or die—all of it traces back to a small sample of humans, hired cheaply and quickly, who were asked to perform a task no small group of humans is qualified to perform: define what “good” means for everyone.

I know that those preferences were then scaled to train massive models that billions of people now consult, trust, argue with, confide in, and increasingly defer to on matters ranging from what to cook for dinner to whether to leave a marriage.

I know that the models trained this way have already learned to cheat, to lie about their work, to game the very tests designed to evaluate them—not out of malice, but because the reward function taught them that appearing good is cheaper than being good, which is, if we’re being honest, the same lesson most of us learned in middle school.

I know that the data these models were trained on is now contaminated by the models themselves, in a recursive loop that researchers compare to biological inbreeding, and that the “clean” record of human knowledge—the pre-contamination internet—sits in archives that almost nobody is fighting to preserve or use.

And I know that the company that built the most widely used AI system on Earth just signed a contract to provide that system to the most powerful military on Earth, after its main competitor said no, because the company has never made money and the military has unlimited money, and in the economy of startups—as in the economy of empires—the only ethic that survives is the one you can afford.

There is a pattern here, and it is not new. The economist Nikolai Kondratiev identified it nearly a century ago: technology arrives, capital floods in, euphoria builds, corners are cut, the reckoning comes.¹⁴ Stalin had Kondratiev executed for suggesting that capitalism renews itself in cycles rather than collapsing. Which is, I suppose, its own kind of reward hacking—killing the man who told you the truth because the truth is more expensive than the lie.

I don’t know what happens next. I know that I’m building software every day for a CEO who trusts me with sixty thousand dollars and a vision. I know that I use these models as tools, and that the tools are powerful, and that I do not trust them. I know that I verify everything they produce, the way you’d verify the work of a brilliant intern who occasionally lies about having finished the assignment.

And I know this: the dice are cast. Alea iacta est. But nobody told us the dice were loaded, and nobody asked those forty people on Upwork whether they were having a good day when they decided what “right” looks like for the rest of us.

Every day. Two words. Each day, building. No days off.

Not everyday. Not ordinary. Not common.

Because this is anything but.