AI Elites are running into a COVID Trap

Boosters yell past opponents and risk alienating (and maybe killing) a lot of people

AI is going to be what it is. I offer no predictions or explanations of what it will look like in 1, 3, 10, or 100 years. I remember watching Twitch streams in November 2022 with ridiculous, instantaneous songs and interactions generated by ChatGPT. I remember prompting images in Hugging Face and being entertained by the incredible capability to get close to a concept but still being distinctly wrong.

Things have changed. At work, I have access to Claude Code and Codex with few limitations on usage. I have worked with Claude to build a report in two hours that typically required months of back and forth with co-workers and customers. A week later, it recommended making a change that every person in my role is taught on our first day of training never to do. I have seen impressive, stupid, and disappointing in the same prompt. I’m not even that good at using AI. Boosters will probably say “get better at prompts” and opponents will stop reading this article because I have at one point purchased access to AI for personal use. I don’t care; right now my issues are with how the elites of AI are communicating (and not communicating) what’s possible to the masses.

Myopinion

Noah Smith reminded hundreds of thousands this week that Americans can be weird about technology. We will use poorly designed tools like pizza wheels and ignore convenient tools like washlets/bidets. I own both and I get way more questions about the bidet. I don’t think I’ve successfully persuaded any of my friends to get one, but I’m also not that passionate about them.

I’m extremely passionate about local government. Last Wednesday, I had the opportunity to speak to Bill Tishler, the District 11 Alder in Madison, WI, about the first Strong Towns book. As he puts it, the book club is an incredible excuse to make him read books and also engage on controversial-ish ideas to see how his constituents feel about them. I commend him for putting this together and it was great to talk to a few dozen District 11 residents about Strong Towns.

If you aren’t familiar with the area, the west side of Madison is about as close of a model of the American Experiment as you can get. And “Strong Towns: A Bottom-Up Revolution to Rebuilt American Prosperity” spends a lot of time criticizing that development model. We were going to have a discussion about a book that I could feel in the room was interpreted as a scolding if you didn’t read the final chapters.

District 11 Financial Information that shows significant underperformance compared to other districts in Madison - credit Ben Noffke

I could talk at these people about how minimum lot reform is necessary in Madison and zoning changes made last year ultimately help the city. I could act as if my research is above their feelings and they need to just “get with the program” and let decisionmakers make their decisions.

But that would be awful persuasion!

Instead, and I hope it was received productively, I spoke about how zoning has affected my life. How I worry about city financials and want to ensure that as someone who moved to the Madison area nine years ago, I can be just as welcoming as the people were for me when I was a student. And if a student decides they want to make this place their home, that it can be a home decades down the road rather than a bankrupt husk of a city. I had productive conversations with people who fundamentally disagree with me on some issues (the value of renters) and found common ground on others (there are few starter homes in Madison and that’s a problem).

Smith’s article felt more like talking at me. Embrace it. It’s beneficial technology. If we don’t use it, we will fall behind. That’s how I feel most boosters talk about the technology and you fundamentally are not going to win people over with that strategy.

Smith did touch on how many Americans are using AI and pointed out the specific group that is a “never engage” cohort. AI is made up of so many different concepts though that I don’t think this is a fair assessment of how Americans feel about and use AI. The LLMs are effective at processing a question that would have taken several Google searches previously. I think this is how most people intentionally interact with “AI” today. But that’s not AI. AI is use in defense systems, medicine, employment—all fundamentally larger parts of people’s lives than search.

If everyday Americans received a comprehensive lesson on Sam Altman’s turn from creating an artificial intelligence company that “benefits all of humanity” with serious safety and financial restrictions to the win at all costs version we see now, would they feel good about the trajectory of AI in the United States? If they understand what Mythos is apparently capable of, would they still support allowing aggressive research?

I cannot comprehend how people are okay with the reality that a CEO specifically setup a corporation with rigid safeguards that came down as soon as the profits could be realized. Especially as these models seemingly have security implications that people previously disregarded but now view as a threat. This for me is the Fauci moment that AI boosters need to grapple with if they want normal people on board.

Don’t lie to people

Many people have criticized how Dr. Fauci and the United States medical establishment handled the initial need for masks in March 2020. This includes Smith. At a time when people were beginning to realize how disruptive the pandemic was going to be, Fauci and pandemic taskforce leaders did not emphasize the efficacy of masks. They were concerned that if they acknowledged that cloth masks would help reduce transmission, people would jump to the conclusion that better masks (N95s) would reduce transmission further and that would strain resources for healthcare professionals on the front lines.

This spiraled into anti-mask and anti-vaccine sentiment fueling conspiracy theories tarnishing a respected doctor’s reputation and transforming him into one of the most reviled people in the world in certain communities. I don’t live in those communities, but it’s a wound that will remain open for a long time.

I’m not a professional taskforce communicator; I can’t provide any guidance or clarity on what should or should not have been said. But Smith and other boosters should heed the same advice and critiques they were offering five years ago and be very clear about what artificial intelligence means to them. They shouldn’t lie, and they should call out the liars and obfuscators that are tarnishing their brand.

I sold all my American index fund stock a few weeks ago. I read Matt Levine from time to time to stay partially plugged into the roller-coaster, insanely complex financial system the United States has developed. It is fascinating to read about “the U.S Stock Market will pay $2 for $1 of crypto”, “everything is securities fraud”, and “it’s not gambling, it’s predicting.” But the most recent coverage leading up to the SpaceX IPO was what frustrated me most: “Index Funds Can’t Say No to SpaceX.”

Here’s a company that is being valued on a huge premium of its current revenue (and profit) it can generate. It has a governance score comparable to war-time Russia. Elon Musk will have sole control over the company and that leaves a bad taste in my mouth after the months of attempted intervention in Wisconsin (among many other reasons).

It should not be in indices until it can meet the existing standards of the indices. The S&P 500 stood strong in the face of changing rules for “megacap IPOs”, but others did not. It looks like it would be included in VFTAX; a social, governance, and equity focused index fund that should never allow it to be included under the current structure. But someone/some group has persuaded FTSE that leaving SPCX out of their indices is a mistake. Despite the fact that it was going to be public for anyone to purchase directly on the stock market.

It’s hard to not speculate that this is the makings of investors and founders that may be in-too-deep and need to offload the hype of AI onto ordinary investors. The counter, that ordinary investors may miss out on the investment of a lifetime, rings false when anyone can access it via public stock. We shouldn’t be forcing 401(k)s and pension funds to take stakes in these companies for no reason other than speculation. The index fund movement was specifically advertised to people as a way to make average returns in the long-term. It’s lower risk, with lower fees, and these companies are trying to upend it because a lot of capital exists in these financial pools.

Separately, these models are getting scary. IF the capabilities of Mythos, GPT-5.6, and whatever models that come out in the next year are credible, there’s a non-zero chance that things break immeasurably and for a long time. No one trusts this administration to handle serious issues, let alone a full-on catastrophe. That means it’s going to be up to the people who are developing the frontier models, or know the people developing them, to speak up and tell the truth.

Jockeying hurts everyone

This isn’t a cerebral take or anything, it’s from the view of someone who can read headlines and read articles that provide some transparency into where AI is currently at and has no control over the massive potential car wreck beginning to form.

There are two sets of AI believers:

1. Investors banking on the returns necessary for the largest run-up in capital expenditures the world has ever seen that AI is requiring

2. Users who are trying to find applications for AI that will transform the world

These can crossover. Altman is clearly in both camps. Some people care more about the money and some care more about the applications.

Each case has positive and negative externalities. Investors have sunk over a trillion dollars into the R&D and infrastructure needed to generate the compute frontier models use to be productive. Productive applications can mean “cure cancer” or “develop bioweapons”. If either one of these ends poorly (we overinvest significantly; we develop something that causes a mass extinction), there are serious consequences.

But clearly there’s one that’s way worse.

And it seems like the investors, who are risking a lot of money, are guiding decision making instead of the users who are realizing that these models could cause serious damage. Right now is the “Fauci mask” moment. Are we going to take a risk, and let the investors try to make a lot of money (or save their failing/mistimed bet), even though serious problems could happen at any moment? Or are we going to be honest, get serious about the consequences, and ensure people are preparing for what’s next?

It won’t take Armageddon for society to completely turn on the technology. If hackers use it to break critical infrastructure systems (hospitals, water, payment processing) for a temporary amount of time, there will be chaos and outrage. No more data centers, but faster.

But at that point it’s too late. The cat’s out of the bag and malicious actors have the models they need to continue to cause damage. We then get an awful incentive where “we need the AI to beat the bad guys” becomes the reason to continue pursue aggressive research.

Boosters: is this more likely to be harnessed as a weapon than a cure? Do we need to slow down research, get a better process in place, and understand what we are creating? Speak now and speak the truth. The world will soon be watching.