The founders of Cloudflare, Matthew Prince and Michelle Zatlyn, publish an annual letter, and this year they wrote about building a better open web with affordances for AI. I like Cloudflare as a company, and they’ve written a hopeful document that proposes some interesting ideas, but I don’t agree with their proposals.

Some background: Cloudflare is a huge internet hosting company. A large portion of the world’s traffic flows through them, and they have a tenuous relationship with AI. For example, when they saw that AI companies were downloading everyone’s content and driving up hosting bills, Cloudflare built a tool that let their customers block AI spiders. Cool!

Simultaneously, writers and publishers are pushing back against AI companies. This is why we see Anthropic on the hook for a $1.5 billion class-action lawsuit paid to authors whose works they indexed (the actual compensation numbers are still being worked out). Other producers like Reddit are cutting increasingly aggressive licensing deals (not that this will trickle down to the posters or moderators), and in general the “we’ll just help ourselves” era of LLM development is slowing. Meanwhile, the ad-funded model of Internet publishing could be under threat, because search traffic is expected to be replaced by AI summaries. 

For its part, Cloudflare is pretty sure the genie is out of the bottle. They write:

It’s already clear that the Internet’s discovery system for the next 15 years will be something different: Answer Engines….[i]n the short term, this is going to be extremely painful for some industries that are built based on monetizing traffic….the loss of traffic for media companies has already been dramatic. It’s not just traditional media. Research groups at investment banks, industry analysts, major consulting firms — they’re all seeing major drops in people finding their content because we are increasingly getting answers not search treasure maps…. Without a change they will still kill content creators’ businesses. If you ask your [AI] agent to summarize twenty different news sources but never actually visit any of them you’re still undermining the business model of those news sources. 

It’s interesting to see all of this plainly stated by an organization that has deep insight into overall online traffic patterns (and a business interest in understanding those patterns). Organic, human traffic is dropping!

That said, I was at a conference last week and saw other stats from a major publisher that say traffic isn’t really dropping much, just a little, and that search is doing fine. It’s a very vague time—but that said, if you’re using AI, it’s reading the web, not you. That’s facts, and I think their vision of the future is logical.

The Cloudflare letter continues:

So how will the business model work? Well, for the first time in history, we have a pretty good mathematical representation of human knowledge. Sum up all the LLMs and that’s what you get. It’s not perfect, but it’s pretty good. Inherently, the same mathematical model serves as a map for the gaps in human knowledge. Like a block of Swiss Cheese—there’s a lot of cheese, but there’s also a lot of holes.

Imagine a future business model of the Internet that doesn’t reward traffic-generating ragebait but instead rewards those content creators that help fill in the holes in our collective metaphorical cheese. That will involve some portion of the subscription fees AI companies collect, and some portion of the revenue from the ads they’ll inevitably serve, going back to content creators who most enrich the collective knowledge….

You could imagine an AI company suggesting back to creators that they need more created about topics they may not have enough content about. Say, for example, the carrying capacity of unladened swallows because they know their subscribers of a certain age and proclivity are always looking for answers about that topic. The very pruning algorithms the AI companies use today form a roadmap for what content is worth enough to not be pruned but paid for.

I gotta say, as a vision of the future open web, this is…bleak. Write new thoughts to please ChatBot, so ChatBot can digest new thoughts and dispense happiness token! Now all happy! I’m not going to sit here and beat up on it. You, reader, are either thinking, “What’s so bad about turning your thoughts into spider food? We’re all consenting adults.” Or you’re on Bluesky, posting angrily about this already. Not gonna change any minds in 2025.

But I have a few thoughts, and then a counter-proposal:

First, I think the choice offered between “make ragebait” and “feed the robot info nuggets” is a little reductive. If those are my choices as a person with thoughts and ideas, I’ll just learn to quilt. There are many other ways to use words. You can even write newsletters.

Second, the market they’re describing already exists, and it’s rough stuff. AI companies already outsource their content classification and creation overseas for very low rates. That’s pretty thoroughly outlined in Karen Hao’s Empire of AI. The industry is dependent on cheap, compliant overseas labor for classification and content creation, and that model is already working for them. I don’t know if broader incentives to get more people into the space would be preferable to them, frankly.

Third, you can’t get writers to do anything without threatening them. I’ve watched countless times over decades as large tech firms tried to “fix the publishing ecosystem” after they broke it, with everything from custom ad-network-friendly CMSes to blockchain payments to aggregation platforms with subscriptions. I’ve participated with hope, but over the years, I’ve realized it’s an impossible challenge, because the scales are so mismatched. When tech giants interact with publishers and writers, it’s a lot like when Lennie pets the puppy in Of Mice and Men. My advice today is basically, “Just put the puppy down.” Then, if you’re feeling virtuous, feed the puppy and leave the puppy alone. It’s messy and bites.

People make high-quality stuff online, the really good stuff, because either (1) someone pays them, and they contribute to a publication, and the publication sells subscriptions; or (2) they want to be a valuable part of an authentic community where they have some control. That’s Reddit, or Wikipedia, or message boards. Do you remember Knol? It was Google’s vetted version of Wikipedia. They could have supported Wikipedia in a truly meaningful way with that money, but they built Knol, and it died.

When I look around today’s web, I can see that a few not-for-profits like Wikipedia and the Internet Archive provide ad-free outposts, and community-funded efforts like Archive of Our Own can find long-term models of sustainability. Tons of good stuff is being published all over the place, and there are huge ad-funded networks that publish consumer content, reviews, recipes and so forth that are still driving billions of views. But my gut tells me that version of the web is fading, at least a little, and potentially a lot. I agree with Cloudflare on this. The web is always dying, and always being reborn.

You know what has staying power, though? Smaller publications with paywalls. The Baffler, n+1, Nautilus, stuff like that—and I’m also watching the new crop of cooperative publications like Defector, 404 Media, Flaming Hydra, and Hell Gate. They run hot, and have extremely low opex. I love them. I have subscriptions to so many things. I believe in the model enough that I’ve helped fund some organizations like these—and then I don’t ask where the money went. You don’t have to wait. You can give them money today.

I think this is the hardest part of all of it for people to understand: You can’t “fix” the commons because it’s humans, and we’re perverse, and we want to do what we want. People don’t make things on the Internet because they want to make LLMs smarter; they make them so they can be more powerful and have more control over their lives, and to meet other people who are into the same stuff. So after many years of loving and appreciating the open web I’ve come to the paradoxical conclusion that the healthiest thing for encouraging new, novel human thought—and the best way to support writers—is to fund their publications and build paywalls.

If the LLM companies want to get access to what’s inside these publications, they should actually send emails and license the work. I’d love to see lots of smaller publishers get together and set their own prices and terms and collectively negotiate some payouts. Then again, they all hate AI so it might be a disaster. That’s fine! As it should be. Our job in life is not to make things easier for big companies. Their job is to make things better for us.

I hate the friction of subscribing to everything, but a nice paywall is a limit even a robot can understand. It means that publications have power against over-eager spiders—they can say “no,” and they can say “pay me.” Does OpenAI want The Baffler? Actually, I think it does.

Is there a role for AI in all of this? Sure! Small publishing cooperatives need help understanding legal issues, governance issues, taxes, fiscal sponsorship, contracts—all baseline stuff where professional help is better but expensive, and the chat is frequently better than nothing. They also need software help, because those tote bags don’t give themselves away, and AI could really help here—custom CMSes! Data import! Customer-relationship management! All far more tractable with the help of a friendly nerd and more adaptable software. Empower them and stand back.

So that’s my counterprosal: AI should pay for the stuff it already uses, and people should keep putting up paywalls. If you really want to help build an ecosystem, subscribe to everything cool and donate if you can. Don’t wait for a class-action lawsuit; actually help! If people make cool stuff and a giant AI company ingests it, with permission, into their LLM, the result will be much, much cooler and more interesting than asking people to write a bunch of stuff to reactive specifications for AltmanKoinz. One approach gets you a museum filled with art, and the other gets you Thomas Kinkade stores at the mall. Your call.

Does a world of defensive paywalls suck? A lot! I hope better models emerge, but ultimately I want to read and look at very good stuff with my one wild and precious online life, and maybe the best way to create a marketplace of ideas is to actually, you know, pay for the goods.