AI Overviews Rollout and Reception: Part 2 of the State of Organic Search 2024
In part 1 of this 3-part series on the state of organic search in 2024, we took a look at Google’s March 2024 core update—what it focused on and what the results have been post-rollout. The core update was announced on March 5th and the rollout finished mid-April. Marketers keeping a close eye on SEO barely had any time to breath before the next big change hit. In part 2, we’ll be taking a dive into Google’s rollout of AI Overviews.
On May 14th, Google rolled out a new(ish) search feature called AI Overviews to everyone in the U.S., and they promised more countries would see it soon. Why only new-ish? Well, some users might have already seen a version of AI Overviews through Google’s beta testing and experiment in Search Labs. In beta, it was called SGE (Search Generative Experience). In this big official release, SGE was officially rebranded as AI Overviews. Google promised a lot with AI Overviews, and it’s received a lot of attention (both good and bad).
What is the AI Overviews Feature?
AI Overviews is a feature within Google Search that uses their language model Gemini to provide conversational answers, summaries, insights, or responses to user searches. This follows in a long trend of Google testing how they can offer users results without users leaving google.com.
Users have probably noticed several of these Google features (that provide answers within Google search instead of just showing search result links and summaries) over the past few years:
Calculator
Local Temperature
Translations
Conversions
And then there are other features that show a search result and link but only after it shows a significant amount of information in the feature, so users might not need to click away from google.com for anything additional. Examples include:
Knowledge Panels such as this one showing a panel from Wikipedia on Google Gemini language model.
People Also Ask snippet such as this one showing a quote from a Medium article on whether ChatGPT is a large language model.
The result of AI Overviews isn’t hugely different* on the face from the rich snippets and knowledge panels we’ve seen for years. The biggest difference is in how the feature is created. In rich features and knowledge panels, the content is quoted from another website (such as Wikipedia in the first example or Medium in the second example above). But in the AI Overviews snippet content is generated by the multimodal large language model that powers AI Overviews.
Okay, Then What is an LLM?
Diving into how LLMs work can get really complex, so we’ll try to give a short, simplified explanation and focus on a couple of key features. If you want to deep dive into LLMs, research neural networks, deep learning, and transformer architecture. This is an in-depth but not overly technical article on how LLMs work that might be a good place to start.
LLMs are computer programs that are trained on huge, huuuuuuge data sets. So much data that the program (hopefully) gets good enough to be able to recognize and interpret language and produce natural language as well. LLMs require significant training to become functional, but they’re also built on models intended to be (or become) self-training… where the program can update or adapt old data as new data becomes available.
But what does it mean for a computer program to “recognize,” “interpret,” and “produce” language? LLMs don’t think. We call it artificial intelligence, but that’s not accurate (at least yet). Really, it comes down to probability. When an LLM is fed a prompt (like in ChatGPT or in Google Search), the program uses its huge amounts of data to create an answer based on probabilities and patterns.
Back to my comparison. AI Overviews is not too different* from a rich snippet, but instead of showing an answer/information based off a quote or information from one webpage, it shows an answer based off a huge dataset. In AI Overviews, Google does annotate with links to external websites at some general wave towards citation, credit, and search result links.
What Have We Seen from AI Overviews?
AI Overviews has seen a very rocky rollout, which is slightly surprising seeing as how it went through beta and Google reported stellar results from their testing. People have reported that it’s hard to verify the accuracy of information, but the biggest hubbub has been about amusingly inaccurate information (you might have seen some of the examples: pizza dough recipes with glue and recommendations to eat a couple of small rocks every day).
Google faced so much backlash that they shared an update/rebuttal the last week of May where Liz Reid, the VP, Head of Google Search started off by saying that “feedback shows that with AI Overviews, people have higher satisfaction with their search results, and they’re asking longer, more complex questions that they know Google can now help with.” But then acknowledge that people have shared “odd and erroneous overviews.” She says many of the reports have been faked, but that “some odd, inaccurate or unhelpful AI Overviews certainly did show up.” Then she wraps up by emphasizing the improvements they’ve made to AI Overviews since being fully rolled out.
Google does seem to have tempered showing AI Overviews in search. Third-party sources say that the AI Overviews feature is being shown significantly less often than right after roll-out (or even during beta). This is purely anecdotal, but try as we might during the time period that we’ve been working on this article, we haven’t been able to get a single AI Overviews result in search.
[Post Draft Update: We have yet to successfully see an AI Overviews feature in desktop, but we have seen them on mobile search. The search query for the below mobile screenshots was “Recent Updates in AI”]
And What About the Future of Search?
Response to this search feature has been very mixed, and industry experts are mixed whether it’s a short-term fluke or a long-term feature.
Even if AI Overviews doesn’t stay in its current form, it’s still an example that is fully in line with how Google search has been trending. Google seems to want to provide an incredibly vast digital assistant that can respond to almost your every informational need. And they have been trending towards reducing “friction” by providing more immediate answers and centralized information. This can be seen as handy but comes with some potential downsides: spreading false information, monopolizing information access, reducing content and voice diversity, and increasing privacy and data concerns.
So how does this impact organic search and digital marketing? Let’s take a look into that landscape in part 3.
*Some (including Google themselves) would argue that AI Overviews are different in terms of complexity. We can’t say that we’ve seen much of that yet, but we have been seeing some fancy, complicated responses from other LLM interactions. Maybe that complexity is there or in Google’s future.
Postscript: We broke this into a three-part series, so we would have reasonably sized blog posts. And yet, here we are 1,100 words in. Thanks for hanging with me.