Will AI Change Spreadsheets?
July 30, 2024 · 18 min 48 sec
On this week’s Reqless, Paul and Rich look at how AI might affect the dominant way people organize data today: The spreadsheet. With its low barrier to entry and ability for users of all sorts to hack together solutions, does the humble spreadsheet leave any room for an AI transformation—and does it even need one? Plus: Fresh off a trip to San Francisco, Paul reports back on our driverless car future.
Show Notes
- Microsoft’s “SpreadsheetLLM”: AI in Spreadsheets by Ali Waseem
- LLM + Spreadsheets
- Collaborate with Gemini in Google Sheets
- Microsoft unveils a large language model that excels at encoding spreadsheets
- [2407.09025] SpreadsheetLLM: Encoding Spreadsheets for Large Language Models
- “However, spreadsheets pose unique challenges for LLMs due to their expansive grids that usually exceed the token limitations of popular LLMs, as well as their inherent two-dimensional layouts and structures, which are poorly suited to linear and sequential input. Furthermore, LLMs of- ten struggle with spreadsheet-specific features such as cell addresses and formats, complicating their ability to effectively parse and utilize spreadsheet data, as detailed in Appendix A.”
- [2402.05121] Large Language Model for Table Processing: A Survey
Transcript
Paul Ford: Hi, I’m Paul Ford.
Rich Ziade: And I’m Rich Ziade.
Paul: And you are listening to reckless Reqless, the podcast about how AI is changing software—and software changes the world.
[intro music]
Paul: Okay, this podcast is brought to you by Aboard—aboard.com. Rich, what’s Aboard?
Rich: Aboard is an AI-powered software platform that builds the software you need much more quickly and much more cost-effectively, because we use AI to build it. Design. Define. Build. Try it. Iterate. Fix It. Iterate again—
Paul: [laughing] Thank you for your—
Rich: With robots.
Paul: That’s the worst Daft Punk song I’ve ever heard in my life. [laughter] But yeah, check us out at aboard.com. If you want to use Aboard, if you want us to come talk to your organization…frankly, we’ll come and talk about how AI is changing software. There’s nothing on the line here. We just like to talk to people.
Rich: Sure.
Paul: Hello@aboard.com.
Rich: Check it out.
Paul: So look, I got two things I want to run down. One is I saw the future. It was AI-powered. The other is, I found a wall that it’s hard for AI to get over.
Rich: The future often has walls ahead.
Paul: There is a little of that.
Rich: Yeah.
Paul: So look, I was in San Francisco real quick for a thing, and I was like, I gotta try one of these robot cars, these Waymo robot cars, because you see them all around.
Rich: So you’re talking about, it’s essentially, you call the car on your app, like Uber.
Paul: Yup. Yup.
Rich: Except there will be no driver.
Paul: Yes. And no tip.
Rich: [whistling]
Paul: [laughing] You don’t tip the robot.
Rich: Why not tip the robot?
Paul: It doesn’t ask. There’s no way to tip the robot. You can’t leave, like, a five on the seat. Like, it’s just not cool. Anyway, so they’re Jaguars, and they have all these little spinning bits on top of them.
Rich: Sensors to be aware of the surroundings, sure.
Paul: And it lights up. It comes and it lights up with your initials. So PF on the top little screen.
Rich: Okay.
Paul: And you’re, because sometimes there’s like four at once. There’s a lot of them.
Rich: Okay. All right. And I can’t just say “Paul”. Because there’s a lot of Pauls running around San Francisco.
Paul: No, exactly.
Rich: Yeah.
Paul: So I put—
Rich: PF shows up.
Paul: And I was like, all right, let’s see, let’s see what this is like. And, I mean, it’s San Francisco. It is a hilly traffic nightmare.
Rich: It’s kind of condensed. It’s crowded. There’s a lot more going on. A lot going on. They can pull it off in San Francisco, they can pull it off in a lot of cities.
Paul: And you’re sitting there and the, the steering wheel turning back and forth. [laughing] You can choose your music, you can hook up your Spotify.
Rich: Okay.
Paul: And you just sit there for 15, 20 minutes as you get from point A to point B.
Rich: Wild.
Paul: And it drives you and it feels really normal, and then you panic. You’re like, “Oh my God oh my God—oh, no, no, we’re okay, we’re okay.”
Rich: Yeah, yeah, yeah, yeah. The lack of control.
Paul: Well, it’s just, there goes a cyclist, there’s a pedestrian—but it sees them better than we do.
Rich: Yeah, yeah. It’s probably, I’m guessing, hyper-vigilant.
Paul: I’ll tell you, when I got back through JFK and I took a car.
Rich: Mmm hmm.
Paul: I would have preferred the Waymo.
Rich: Sure. Sure!
Paul: It was not—the Belt Expressway was not a pleasant experience.
Rich: The Belt Parkway, yeah, yeah.
Paul: Going on Flatbush around midnight.
Rich: Oh yeah, I hear you.
Paul: So that was a moment. I’ve loved the idea of the driverless car for a long, long time, because I just like, cars are unsafe. They kind of control everything.
Rich: Yes.
Paul: So I’m like, I think ultimately they will be safer. Like, that is the future.
Rich: It is. It’s moving slower than we expected.
Paul: That’s okay. It’s going to take a minute.
Rich: Yeah. Interesting. Is it AI that’s powering all this?
Paul: I mean, it’s a lot of machine learning and statistical inference and so on and so forth.
Rich: Sure.
Paul: It’s not AI like an LLM because you don’t want to be like, you know, you don’t want it to be like, “I can drive you to Fran Fronfrifco.
Rich: Like, yeah, yeah.
Paul: You don’t want to improvise.
Rich: Of course I’ll take you to Europe. [laughing]
Paul: Yeah. This is the old-school AI. The AI that was like, just, you know, it turns everything into a bunch—
Rich: Stay in your lane. Literally and figuratively.
Paul: Yeah. Exactly. So, so that I gotta say, though, like, I would do it again in a minute.
Rich: Oh wow.
Paul: Like, it was totally safe.
Rich: That is…that is the future then.
Paul: If they don’t—then if they could, if they have them in New York, it’s gonna be, it’s gonna be wild. It actually will be an end of an era. But, yeah, they’re fine. They’re good.
Rich: Very cool.
Paul: Okay, so that’s A.
Rich: Okay.
Paul: B was, I was like, you know, all right, last time we talked about how the legal industry is changing.
Rich: Yes.
Paul: And I was like, let’s go for a software category. Let’s talk about the big kahuna. How are spreadsheets, which are the dominant way that people organize data—and I say this as someone who, with you, is creating a data-organization product.
Rich: Spreadsheets, and we’re not talking about like hardcore math and functions, we’re just talking about putting information in boxes.
Paul: That’s right.
Rich: Spreadsheets, they’re just everywhere. They’re everywhere. Billions of people use it. It is just probably one of the most deeply ingrained tools in all of work software.
Paul: Okay, let me tell you about what’s happening with spreadsheets.
Rich: Okay.
Paul: And LLMs and AI.
Rich: Okay.
Paul: Not that much.
Rich: Hmm.
Paul: Google Gemini—we’ll put the link in. You can go to Google Gemini and you can be like, it’ll help you kind of like what we do. It’ll be like, hey, I want to make a spreadsheet that lets me organize my Little League team.
Rich: And it’ll spin one up.
Paul: It’ll spin one up.
Rich: Okay.
Paul: Same, same kind of deal.
Rich: Okay. That’s good.
Paul: It’s great. I mean, it’s validating for us because we’re doing it, too.
Rich: Yeah.
Paul: And of course, I have to go on the documentation because God forbid you try to log into a Google product and get an advanced feature into your account.
Rich: It is a world.
Paul: You just get to watch the video. You can’t—
Rich: Reminds me of Microsoft, when they kept cobbling stuff on.
Paul: What I love is that like, hey, guys, look, we’re here in the future, but it’s like, maybe we could backfill a little of that future and actually let me log into the product. That would be cool.
Rich: Yeah, I think it’s just too much stuff.
Paul: Too much stuff. Anyway, so there’s generative stuff. And I think as you look at where the AI is going to land, it’s going to be natural language to query things. It’s going to be automated assistants that help you…select formulas.
Rich: Sitting alongside the spreadsheet.
Paul: But here’s the thing. Microsoft put out a paper, okay, and the paper is spreadsheet LLM, encoding spreadsheets for large language models. We’ll have a link.
Rich: Okay.
Paul: And it’s by 200 separate authors. [laughter] And it generated a little bit of news. But what it says mostly is, boy, this one’s hard. Here’s why. Spreadsheets, so you think about LLMs and sort of how we’re supposed to talk to them. You can feed them a PDF and you can feed them like a bunch of stuff. But spreadsheets are zillions of little cells, right?
Rich: Yes.
Paul: So they actually add up. There’s a lot of data. Like if you, if I gave you the budget for Aboard for the year.
Rich: Endless, endless input boxes.
Paul: It’s actually, we look at it as a pile of numbers, but it’s actually like a novella of numbers.
Rich: Yeah.
Paul: Like it’s a lot of stuff.
Rich: Yeah.
Paul: So they hit the token limit. So the token limit is like how much information you can give to the LLM, and then it’ll start to do its magic statistical Mad Libs, right. See, first of all, you gotta like figure out a way to make sense of this thing because really LLMs only kind of understand words in a sequence.
Rich: Okay, are you talking about teaching the LLM or the LLM’s output? Or like, where are we?
Paul: Kind of all of it. So let’s talk about how things work. I go to ChatGPT, I ask it a question.
Rich: Okay.
Paul: And it hits this—it, like, takes those words and it like turns them into numbers, and it looks at the giant vector database of stuff.
Rich: Yup.
Paul: And it says, okay. And it spits out its response.
Rich: Yes.
Paul: Okay. So, and boy, did I hand-wave there, but a little bit like that. Okay?
Rich: Yeah, and it’s very serialized.
Paul: That’s right.
Rich: It’s a stepped sequence that delivers the outputs. Now images do it differently. They sort of double back over the image and refine it.
Paul: Honestly, I’ll just read you straight from the paper.
Rich: Yeah, go.
Paul: “Spreadsheets pose unique challenges for LLMs due to their expansive grids that usually exceed the token limitations of popular LLMs, as well as their inherent two-dimensional layouts and structures, which are poorly suited to linear and sequential input. Furthermore, LLMs often struggle with spreadsheet-specific features such as cell addresses, formats, et cetera—”
Rich: Interesting.
Paul: “—complicating their ability to effectively parse and utilize spreadsheet data.”
Rich: Interesting, interesting.
Paul: So there’s a wall here, and it’s a really interesting wall because, like, when you think about—so, you know, we’re going to have generative schema development. We’re going to show you, we’ll organize, we’ll pre-organize your data. But it actually is really hard for this class of AI to understand what’s in there and do anything with it.
Rich: Sure.
Paul: They’re not great at math, just flat out.
Rich: Sure.
Paul: Like, they don’t know what you’re doing.
Rich: Sure.
Paul: And it’s just like, so it turns out that this one giant category that runs the world, this is the most important software on earth, is the spreadsheet, runs everything.
Rich: Yeah. Yes.
Paul: It’s kind of immune to the magic trick of AI.
Rich: Yes. I have one thought and one prediction.
Paul: Okay.
Rich: My first thought is spreadsheets took hold because it is arguably the least-opinionated software ever created. It is boxes that humans could put stuff in, right?
Paul: Averages, sums.
Rich: Not even that. I’m not even going there. I’m just putting my collection of baseball cards in.
Paul: Mmm hmm.
Rich: Like, I don’t even want to do the math.
Paul: Mmm hmm.
Rich: Like, I’m not doing any math. And its power is that its entry point is incredibly low. Boxes, type, words.
Paul: Yeah.
Rich: And it’s, it’s a grid, which for the human brain is like, okay, I can, I can work with this.
Paul: I mean, you know what rules is, across the top? A-B-C-D-E-F-G. And down the, down the left, 1-2-3-4-5-6.
Rich: Exactly. Exactly. And so what have humans done? And I think this is where, why I think we’re going down a bad path here. It’s like, what can AI do with spreadsheets? What humans have done is they’re like, oh, I can resize these columns. I can use bold text. I can actually write some light little scripts in the here and there. You know what I’m going to do? I’m going to put the app in the damn spreadsheet.
Paul: Mmm hmm.
Rich: I’ve seen it. I’ve seen people say, just fill out this application. I’m like, okay, I get a URL, and then it opens an Excel sheet, and it’s somebody that decided that they were going to make software with a spreadsheet.
Paul: I mean, half of the organizations we go to who want to tell us their software problems because maybe we can help.
Rich: Are using spreadsheets.
Paul: Yeah, they bring up these beasts.
Rich: Yeah. I filled out forms inside of spreadsheets?
Paul: Oh yeah.
Rich: Because people are like, all right, just fill this out and then send me the file.
Paul: You know who loves that? Government.
Rich: Government loves it.
Paul: Yeah.
Rich: And so what you have is this frankly blank canvas of business software. Like, it’s a tool, essentially, and humans just went buck-wild with it because they’re like, you know what, Okta, what? I gotta log in where? What? [laughter] I can work on this on the train, and I will have the form with nice colors for my boss in the morning. Like, that’s what you have here.
Paul: Let’s not forget, too, the miracle of the spreadsheet is you can see it. Like, programming, you can’t see most of what you’re coding. You can only see, like, ten lines at a time in your little brain.
Rich: Exactly.
Paul: Spreadsheets, you can be like, oh, wait, I gotta skip—and this is another, like, you think about where LLMs are and where we are with our ChatGPTs and our Anthropics and our picture generators.
Rich: Yeah.
Paul: Think about, like, the spreadsheet with seven tabs on the bottom seven sheets.
Rich: Yeah.
Paul: And they’re all kind of linked up together.
Rich: Yeah, exactly.
Paul: Now, what they are aiming for is like, can this thing help us with QA? Can it look through and be like, hey, you might have some messed-up stuff over here.
Rich: Sure, sure.
Paul: And this, that’s where they’re aiming. That’s probably the right place to aim.
Rich: You know what this feels like? This feels like humans went to a place, appropriated this tool, use it in all kinds of wild and wacky ways.
Paul: Mmm hmm.
Rich: And then the AI guy shows up. The AI robot shows up.
Paul: Well, we got to now, like, it’s the biggest category of software.
Rich: Oh, I know. And the humans turn and look at the AI like, and the AI is looking at it and it’s like, what have you done?
Paul: Mmm hmm.
Rich: This was not meant to work this way. [laughter] What are you doing? And the human is like, you know what? Go to hell. I’ve been doing this the same way for ten years.
Paul: Organized my baseball card collection, whatever, ran the Little League team.
Rich: Whatever! Exactly.
Paul: And the bodegas has great bookkeeping.
Rich: So here’s what I would argue. I would argue that AI is going, it’s a waste of energy, because what you’re doing is you’re trying to lift out the essentially just all sorts of brilliant hacking that humans have done with spreadsheets and have AI essentially insinuate itself into a world that is really contrived. It’s contrived because humans are resourceful and creative. What you’re really doing, the mistake that’s being made here is we should be asking the AI, how are humans using spreadsheets? And how can AI with a blank canvas work better? If I told AI that I need software to manage my baseball card collection, it’s going to have a much easier time of it. Why? Without a spreadsheet hanging overhanging over everything? Because it’s not being forced to navigate essentially the ground rules of a spreadsheet.
Paul: Yeah, don’t even bother. Right? Like, just like—
Rich: Yeah, it’s like a human did that seven years ago, and their comic book collections is in there, and it has some equations and it has little images and thumbnails of the comics in the spreadsheet, which you can do. And the AI approach is like, wow, human, you are a genius. Let me help you with that. It’s like, no, you don’t belong here.
Paul: Yeah.
Rich: This is a hack. I hacked it together. Now, if you told AI I need a comic-book cataloging system—
Paul: Right.
Rich: that is not dragged down by, like, what a spreadsheet is, it’s gonna have a much easier time of it than saying—
Paul: That’s right.
Rich: I have to negotiate with cells and cell references and pivot tables and all the other stuff….
Paul: No, and God forbid—so I think what you’re gonna see is like in this giant category of software, people will continue to jam all their data into it.
Rich: Yeah.
Paul: The AI will be like, hey, you could make this kind of chart.
Rich: Sure.
Paul: Or I found weird circular references over here.
Rich: Essentially augmentation of capability, right?
Paul: That’s right.
Rich: Like, enhancement of the core. But there is a Rube Goldberg aspect to the way people use spreadsheets today. That is just part of life. I think companies like Airtable and Smartsheet maybe, or something, which aren’t fundamentally spreadsheets, they’re actually tables of data, will have an easier time enhancing their experiences with AI than the spreadsheet. Because the spreadsheet is a bastardization of software. It is human ingenuity. I’m nothing shitting on it by any means, but it is a bizarro—
Paul: It’s deeply arbitrary, and you know, it’s a good example is like, date in Excel, which can just kind of go anywhere. Like you, you think you’re putting a date in and it’s like, no, actually that’s, we’re subtracting numbers and it is..
Rich: The most important, look, it’s probably the most important piece of software in the world. I’m saying that out loud. But is it, is it one you build on?
Paul: So, no, I don’t think so. I don’t think like, like obviously we don’t think so because we’re building a company that’s like up from spreadsheets. But even putting that aside, I just think this is a space to watch because we’re so used to this narrative of AI is going to change everything.
Rich: Yeah.
Paul: I think that it will be intelligent assistance, but I actually don’t think, my guess is if you pick up Excel, let’s say three years from now—
Rich: Yeah.
Paul: After the AI revolution is really underway, it’s going to be like Excel.
Rich: It’s going to be like Excel. I agree with that. I agree with that. Look, you know what’s another way to look at spreadsheets? Spreadsheets are a reaction to the powerlessness that most people feel when it comes to software.
Paul: Mmm hmm.
Rich: It’s actually like, you know what, I’m done. I’m done asking. I can do this right here.
Paul: Where AI can—
Rich: I can drag columns—
Paul: Where AI can help is it can be like, hey, these are twelve things you could do starting right here.
Rich: It’s about, it’s about the ability, the frustration—and in many ways it started as a shim to sort of get the job done, and it evolved and it created an entire world of, I mean, a continent’s worth of people that have decided, you know what, I don’t need you, IT. I’m going to just do it here.
Paul: Just like, you know, people need Excel. No one has ever been like, I need SharePoint.
Rich: No, no—
Paul: It’s like, it is—
Rich: No, it’s worth mentioning, I don’t need to ask anybody’s permission to load up Excel.
Paul: That’s right.
Rich: I could do it right now in my office and I don’t need IT. I don’t need anybody. I don’t need anybody’s help.
Paul: It’s like the one tool, it’s like the one thing everybody has.
Rich: It’s the one tool.
Paul: So look, I mean, my, my thing, I just made a prediction, which is three years from now, and the reason I’m, I’m happy to make this prediction, if I’m wrong, I’m wrong. Maybe like you’ll, you’ll sit down and it’ll be like, I already made, I already put all the numbers in for your budget. You don’t even have to think about it anymore.
Rich: Yeah, yeah.
Paul: But I’m going to say, like, watch that space. Because if suddenly something like spreadsheets starts to tilt and the fundamental experience of using it is, like the cells and the math and all of it starts to get AI-enabled and there’s stuff like, then we’re in a very different world. I don’t think we’re headed for that world. I think it’s still generative and assistance and stuff like that—
Rich: Yeah, yeah.
Paul: Rather than getting in and like doing the math for you.
Rich: I think that’s right. I think it’ll be like, you know, you know the sort of fun little trick where you could like put a formula in one cell and then it’ll just sort of fill out the whole column.
Paul: Yeah.
Rich: Like, that kind of trickery. Like, I think AI will be able to do that times ten, obviously, and do some fun things.
Paul: Did you mean to do this?
Rich: Yeah, exactly. Stuff like that. That’ll be cool and nice. But the idea of conceiving of an entire solution in a spreadsheet, which humans are very proud to do, they love telling this is, they call the master sheets, they call it the master file because it’s like complicated and it’s many tabs. You know, are we, do we, should AI generate those? I don’t think that’s the place to do that. It’s not the right canvas. Uh, it’s the right canvas for humans. Because they can grab it. They don’t have to ask anybody. Right. But AI can do it better in other places.
Paul: So anyway, so there we are. We found a wall that AI has yet to climb.
Rich: There are others. I like talking about the walls. I think they’re fascinating.
Paul: I think it’s good because you need to start to get like, we’re in the hype phase right now where it’s going to change everything.
Rich: Yeah.
Paul: And I still think if I want to like, do my budget, I’m going to put those numbers in and then I’m going to type equals sum and it’s probably going to be me.
Rich: It’s probably going to be you.
Paul: All right. Well, Richard, if people want to get in touch, they should send an email to hello@aboard.com. They should check out Aboard. They should think about it for their organization and get in touch.
Rich: Yeah. We’re going to have guests in the future.
Paul: Thank God! [laughing]
Rich: We’re going to start to expand the conversation and talk about AI and software. Until then, have a lovely week and check out aboard.com.
Paul: Bye!
[outro music]