In the conversation about AI coding, lots of organizations are promising to completely change the way software is developed. You will tell the computer what you need, AI will build it, and it will be good. Companies like Bolt and v0 offer impressive abilities to spin up websites or other software tools just using words. It’s still early days, but eventually, they say that if you can describe it, you’ll have it.

This creates a problem—around the office, we call it “the $15 Volvo problem.” Building a website that looks nice and sits around on the web is a relatively low-risk proposition. That’s an okay application for AI. Others might include making a recipe-tracking app, or a tip calculator.

But the category of software that drives a huge part of IT services is…enterprise software. For years, we’ve joked that people who buy enterprise software are looking for something like a Volvo—boxy and kind of ugly, but designed to last a long time, and built for safety first.

Meanwhile, we’re hearing more and more from people who want to use AI in their company but are getting pitched “enterprise AI.” That’s turning out to be lots of chatbots in a suit—thin layers over ChatGPT and similar tools. They’re being told it will analyze their documents, fix their code, improve their development tools. It will be just as solid as the big, old systems they’re used to—but much cheaper and faster.

But these buyers don’t think, “Oh, finally!” They are risk-oriented and suspicious. They’ve just been offered the $15 Volvo. “It’s just as good as before, but we can manufacture it in minutes and it only costs $15.”

If I offered you a “Volvo” for $15, would you put your children in it? Your pets? Your groceries? How much would you trust a $15 Volvo? What would your expectations be? If you got into a crash, would you be surprised if the airbags didn’t deploy? Would you be shocked if the doors fell off, or if the engine shot up into the sky?

A $15 Volvo doesn’t excite them. It terrifies them.

You can build a lot—a lot!—of software in minutes with AI. Some of that software even works. And this is powerful and surreal and exciting, but there’s another reality: You’re creating enormous technical debt and risk every time you use AI to code. All that code is going to keep running, and it will need updates and maintenance. People can say, “That’s okay, AI will fix the problems,” but that’s a bet, not a strategy.

So while AI is rapidly changing everything from image creation to student-essay writing, I think the future of enterprise software—and enterprise is where the money is!—will be a slower process. If I were in the market for an AI-powered business-software solution, these are the questions I would ask:

• What systems, libraries, and tools were used to make this tool, and what LLMs were consulted?
• Where exactly are there calls to systems like ChatGPT, and where is there “native” code?
• When is my data shared with AI LLMs?
• If things break, will a human evaluate what broke and fix it?
• What processes were used to build this, and how was the code developed and tested?
• How will changes be made and evaluated?
• Why wouldn’t I just do this myself if I can just tell AI to do it? What value do you provide?

I think the most important metric in the future will be, “How much of this system was developed by people and how much was AI?” 

Ultimately, the promise of a lot of AI tools right now is the $15 Volvo—and that’s because what it does, generating code quickly and often at a good level of quality, seems so miraculous. But what business buyers want is a Volvo that gets delivered on time, has incredibly clear diagnostics, needs less maintenance (and when it needs it, a human checks the work). And sure, they want it to cost a lot less. But it’s going to take some time to figure this out—and it’s important to remember that business people buy products for safety, not style.