After Google’s new AI Overviews search feature failed in dramatic fashion last week, the company downplayed the issue but said a fix was in progress.
AI Overviews uses AI to draw on information on the internet to provide an answer to a search query. This saves the user the trouble of clicking through to multiple websites.
In a classic case of “garbage in, garbage out,” a number of the responses AI Overviews gave were comically wrong because it relied on incorrect information posted online.
Head of Search Liz Reid issued a statement in a blog post in which she said that AI Overviews was a great success but admitted there were a few snags.
Reid said that the reasons behind some of the odd results were non-sensical queries or faked screenshots. Regarding the fake results, Reid said, “Those AI Overviews never appeared. So we’d encourage anyone encountering these screenshots to do a search themselves to check.”
She did admit that AI Overviews served up some “odd, inaccurate, or unhelpful” results. The main reason behind this is either the lack of quality data on the internet related to the query or that the query was nonsensical and the search tool didn’t pick up on that.
The suggestion to eat rocks is a perfect example of the challenge AI Overviews faces. The question “How many rocks should I eat?” would not have been asked by many people. When AI Overviews tried to answer the query it found a satirical article on The Onion that jokingly recommended eating one small rock a day.
AI Overviews isn’t great at distinguishing humorous, satirical, or sarcastic content from factual information but Google says it has improved the quality of AI Overviews results by:
- Building better detection mechanisms for nonsensical queries and satirical or humorous content.
- Limit the use of user-generated content (i.e. Reddit) in responses that could offer misleading advice.
- Added triggering restrictions for queries where AI Overviews were not proving to be as helpful.
- Added stronger guardrails to limit responses related to topics like news and health.
People may have had a bit of fun focusing on the odd responses but delivering consistently good responses to search queries is extremely difficult. AI Overviews is objectively helpful, even if you need to treat the results with some skepticism.
AI doesn’t create information, it relies on its training data, or, as with AI Overviews, it returns an answer based on human-generated info on the internet.
AI Overviews has to answer the same challenging questions humans are faced with when reading something online: Is this true?
Ironically, AI is making this question harder to answer, not easier.
