At Google’s 2021 Search On event, the search giant shared an early look at the future of multimodal search – search that combines different methods to get more relevant results.

Search on the web has historically been mostly text-based, with searchers using words to relay what they wanted to find. Even when technology advanced enough to make innovations like visual search possible, searchers would still have to pick a single method to carry out their search: either visual input, or text input, which limits the amount of nuance that can be conveyed.

But thanks to Google’s Multitask Unified Model (or MUM) technology, it will soon be possible to combine different modes of search to obtain a more relevant result. For example, Google’s VP of Search, Pandu Nayak, showed how searchers could use an image of someone wearing a colourfully patterned shirt and combine it with the words, ‘Socks with this pattern,’ to bring up socks with a similar look. This is something that would not be possible with text or visuals alone.

A smartphone screen with a text box overlaid on an image of a person wearing a colourfully patterned shirt. The text being input reads: socks with this pattern

Google's VP of Search Pandu Nayak demonstrates Google's new multimodal search capabilities at Search On 2021. Image: Google

Google has also shared other ways that MUM will enhance the search experience, such as by applying understanding of how users typically interact with a topic to display “deeper insights” – such as surfacing how-to guides for painting with acrylics in response to the query ‘acrylic painting’, even if the words ‘how to’ weren’t included. Other upcoming features include the ability to “zoom in and out of a topic” with suggestions to broaden or refine a search. The acrylic painting searcher could ‘zoom in’ on specific painting techniques or ‘zoom out’ and learn about other types of painting.

Many commentators have compared MUM to BERT, the machine learning algorithm unveiled by Google in late 2019, which also represented a huge change in how Google parses search intent and displays results to users. The announcement of BERT was accompanied by many fears that SEO as we know it would become obsolete – and these fears have only intensified with the advent of MUM. But is there reason to panic?

Let's ask some experts...

Premium content

To continue reading this content and gain access to more than 30,000 exclusive pieces of data, research, reports and articles, you need to subscribe. Use the button below to view your options and sign up. Already have a subscription? Simply sign in below

Need help signing in?

EMEA/USA: +44 (0)20 7970 4322 | email: