ChatGPT is Not a Search Engine
It looks like maybe this tweet is the origin of the notion that ChatGPT is going to kill Google Search. I hope now that we have a bit more experience with it, we’re past all of that. But just in case we’re not, let me offer a few thoughts.
Google Search
Google Search started out trying to surface “good” search results from the vast collection of resources on the web by encoding some notion of “reputation” — the famous “Page Rank” that is a quite accessible and rather traditional algorithm. Over the years, Google obviously became a lot more complicated than that, growing to incorporate all manner of heuristics, AI, machine learning, and most interestingly (to me anyway) the “Knowledge Graph”, which owes something to old-time expert systems in the way it tries to derive and encode vast factual relationships.
But at the end of the day, Google Search (and other classical search engines) have tried their best to deal in “facts” as they find them on the web and serve up content that you can reference — the kinds of things you need to point at when writing a research paper or a legal brief. Best of all, Google Search excels at surfacing even the most obscure knowledge. You’ve probably had the experience of running a search query and only getting back three or four results from the entire web, but the top answer is precisely what you’re looking for.
ChatGPT
On the other hand, ChatGPT and other large-language models (LLMs) response by generating text related to what you ask. In some sense, the fact that the text it generates often gives startlingly lucid and correct answers to your question is a happy consequence of very advanced AI trained on a huge corpus of often-correct content available on the web. As you’ve probably noticed, the accuracy of your answer has a lot to do with just how much content is written on the subject, and it degrades rapidly as you get into more obscure subject matter.
As a text generator, you could make a case that ChatGPT owes almost as much to Markov chains as it does to anything like web search. Here’s what I get when I ask ChatGPT for a paragraph of a speech in the style of Donald Trump:
"Let me tell you, folks, we're standing right here at the crossroads of greatness. Nobody believed we could do it, but guess what? We did it, and we did it bigly! I mean, you wouldn't believe the things we've accomplished. We're talking about turning things around, making things happen, like you've never seen before. And let me tell you, they said it couldn't be done, but we proved them wrong, folks. We're winning, and we're winning big, because that's what we do – we win, and we win so much that you'll be tired of winning. Together, we're making this country great again, and we're just getting started. Believe me!"
And here’s a block of Markov text based on past Trump speeches:
“Number one, we’re really good. Number one, he won’t exist very long. It’s values. I mean, I understand that strengthening and promoting Western civilization and intimation by radical Islam. But this has to be an election that’s based on competence, because we have to do so. And although not in government is subsidizing it. It’s got to stop it folks I know how to develop a responsibility of carrying the plant to Mexico. I love the people that don’t have great respect likewise for Citizens United, David and everybody says why don’t I have a pollster. What the hell do I need it?”"
You can maybe see some similarities. The, um, uniqueness of the former President’s speaking style is especially suited to mimicry via Markov chains, but you get the idea.
Research with ChatGPT
The realization that ChatGPT was nowhere close to replacing Google Search really came home to me recently while I was doing some research for a talk. As an example of the kind of thing I was trying to do, if I ask for “links to references on the use of AI and ML in supply chain management“, I got back results with titles like “Artificial Intelligence and Machine Learning in Supply Chain Management: A Comprehensive Review" and “Using Machine Learning in Supply Chain Planning and Inventory Management" from publishers like Deloitte and Harvard Business Review. They sound great!
The problem is, every single one of these “references” was a hallucination. Every. Single One. The links all 404, and when I followed up on (of course) Google Search, I was unable to find any evidence that these articles exist or ever existed.
Conclusion
So where do I think this is all going?
What’s clear from the online chatter about ChatGPT killing Google Search is that people really, really like the chat user experience when doing search-type things. Google knows this, so you can fully expect that Google Search will get augmented with LLM-based results and formatting. From the other direction, LLMs will use search-like technologies to cut down on hallucinations and provide more factual results.
If search and LLMs are going to converge, personally I would give Google the upper hand against all comers in such a competition. But let me know if you feel otherwise!
