MUM and the Next Steps Toward a Google Answer Engine

Introduction

I’ll get it out of the way now as a TL;DR – you can’t optimise for MUM and anyone saying you can has either misunderstood the concept, or is being purposefully misleading. However, Google’s MUM (multi-task unified model), can offer some insight as to where Google, and search more generally, is heading.

The concept of an ‘answer engine’ has existed in science fiction since the golden age authors first described computers such as the gargantuan MULTIVAC, which was capable of computing the beginning of a universe. It really entered the tech community’s collective vocabulary at the turn of the century – as search engines began to proliferate and refine themselves beyond simple keyword retrieval.

Primus Knowledge Solutions, a name most will be unfamiliar with, purchased AnswerLogic (a “natural language processing technology”) for $3 Mn on May 21st 2001 ‘to better tap into unstructured data sources’ to quote an article from Network World that also discusses the purchase of a direct email marketing platform by AskJeeves. Wolfram Research, which would go on to release WolframAlpha (described as either an ‘answer engine’ or ‘knowledge engine’) in 2009, has been working on such projects since the 1980s.

The difference between a ‘search engine’ and an ‘answer engine’, put simply, is that while a search engine returns page ‘hits’ or ‘results’ which contain the search term or, more recently, related terms, an answer engine will attempt to pull information from across a distributed database and assemble it into a coherent ‘answer’ to the question posed.

Dr Tracy L. Tuten, Associate Professor of Marketing at Longwood University, in her 2010 text Enterprise 2.0 described the difference like this:

So, for example, let’s say you want to know what the circulation numbers are for National Geographic magazine. If you were to type “circulation of National Geographic Magazine” into Google, you’ll get all the pages where that phrase appears, or where those key words appear. The first links are for Wikipedia articles, and then links for the National Geographic Society and then the magazine’s Web site and so on. Perhaps on one of those pages lies your answer – but you will need to click through and read all those pages to find out.
 
Typing that query into Wolfram Alpha, on the other hand, generates a single page of results that pulls information in from disparate sources. It tells you the current circulation number, confirms the full name of the magazine, lets you know it’s a monthly magazine, and provides a link to the Web site. Additionally, it provides the source information for the data as ProQuest LLC, Publist and Wikipedia, and provides links to those sources as well.

While there have been numerous innovations in Google search since 2010 – not least the knowledge panel – Google remains a place to go to search for sources of information rather than for direct answers, even though direct answers are often available on the SERP (search engine results page). A particular weakness of the Google approach to answer generation is in a lack of referencing; there are often knowledge panels which feature a direct answer, but which offer no easy route to check the source.

In fact, the same search for National Geographic circulation numbers now – some 11 years later – reveals that the difference between the two approaches is still substantial.

This is where MUM comes in; it represents another step toward Google becoming an answer engine or, as Sundar Pichai put it at Google’s I/O event in 2019, “moving from a company that helps you find answers to a company that helps you get things done”. This shift, which Pichai has also referred to as a coming ‘age of assistance’ requires Google to return authoritative answers to a huge variety of potential queries (approximately 15% of all searches on a daily basis were previously unseen by the search engine) in a manner it could communicate through a smart speaker or in a text box.

What is MUM?

‘A new AI milestone for understanding information’, as one Google blog puts it, MUM is a ‘technology’ which seeks to better understand a search query and provide answers over results. Google claims that MUM is 1000 times more powerful than BERT, is capable of generating and understanding language (bringing it in-line with Microsoft’s use of the Open-AI large language model GPT-3) and has been trained on datasets from 75 world languages.

How MUM works

Despite it being referred to by Google as a ‘technology’, the likelihood is – as stated by Search Engine Journal’s Roger Montti – that ‘it’s as if Google invented a descriptive brand name for a group of algorithms working together’. The papers referred to in the SEJ piece do offer plenty of food for thought, however, and Hypergrid Transformers: Towards A Single Model for Multiple Tasks in particular shows what a multi-task unified model would do:

Overall, our eventual goal is to dispense with task specific fine-tuning tricks altogether. While neural networks typically maintain the same consistent set of parameters for all input instances, the proposed HyperGrid Transformers introduces instance-specific parameters by conditioning on the current input. This setup enables our model to learn task-specific reparameterisation for each input instance, which mitigates several challenges of multi-task co-training.

Yi Tay, Zhe Zhao, Dara Bahri, Donald Metzler, Da-Cheng Juan – Google Research

I’ll be completely honest, the maths in a lot of these papers is beyond me by some considerable distance, but the paper proposes that a single model, with the addition of what is referred to as a ‘HyperGrid Module’ at the second transformer layer, is able to perform as well in natural language processing (NLP) and natural language understanding (NLU) tasks as multiple single, specialised models on their pre-trained topics.

If this is, indeed, how MUM is operating, then it’s a pretty impressive leap forward in natural language machine learning – meaning that Google would not simply be looking to use a transformer model to understand sentences, but may be increasingly able to understand how language and concepts differ between specialist vocabularies without having to train a model in a specific field.

This paper deals with understanding, and can be applied to queries and to pages – but not so much to answers. I’m going to provide the full abstract for the second paper mentioned in the SEJ article, Rethinking Search: Making Domain Experts out of Dilettantes, because I think it goes some distance to explaining the thinking behind what I see as presaging the arrival of a Wolfram style answer engine from the Google database:

When experiencing an information need, users want to engage with a domain expert, but often turn to an information retrieval system, such as a search engine, instead. Classical information retrieval systems do not answer information needs directly, but instead provide references to (hopefully authoritative) answers. Successful question answering systems offer a limited corpus created on-demand by human experts, which is neither timely nor scalable. Pre-trained language models, by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are dilettantes rather than domain experts – they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances by referring to supporting documents in the corpus they were trained over. This paper examines how ideas from classical information retrieval and pre-trained language models can be synthesised and evolved into systems that truly deliver on the promise of domain expert advice.

Donald Metzler, Yi Tay, Dara Bahri, Marc Najork – Google Research

Not only does the opinion paper Rethinking Search: Making Domain Experts out of Dilettantes share the majority of its authors with the conference paper Hypergrid Transformers, the former provides a logical next step for the use of the multi-task model outlined in the latter. The paper goes on to say:

Indeed, a majority of today’s systems boil down to: (a) building an efficient queryable index for each document in the corpus, (b) retrieving a set of candidates for a given query, and (c) computing a relevance score for each candidate. This index-retrieve-then-rank blueprint has withstood the test of time and has rarely been challenged or seriously rethought.

Donald Metzler, Yi Tay, Dara Bahri, Marc Najork – Google Research

Following this is an outline of some fairly arcane information retrieval processes and machine reading comprehension techniques, outlining the way search has come to use neural matching, neural networks and other machine learning techniques to extract information from existing knowledge graphs, before section three begins with the hilarious (to me) statement ‘[we] begin the more technical portion of the paper…’

Having managed to understand approximately a third of the paper to that point, I didn’t hold a out much hope, but there then comes another important explanation:

For example, for question answering tasks our envisioned model is able to synthesise a single answer that incorporates information from many documents in the corpus, and it will be able to support assertions in the answer by referencing supporting evidence in the corpus, much like a properly crafted Wikipedia entry supports each assertion of fact with a link to a primary source. This is just one of many novel tasks that this type of model has the potential to enable.

Donald Metzler, Yi Tay, Dara Bahri, Marc Najork – Google Research

These two papers were published in 2021, however, so I would be incredibly impressed if they represent what MUM is, or how it works, currently, but they should give an indication of how MUM and subsequent iterations will work.

How MUM relates to answer engines

Hopefully, by now, you will have reached much the same conclusion I did (if not, by all means, let me know), that MUM represents a steppingstone on the way to Google becoming an answer engine – capable of sourcing information from across the web (using structured data and high-authority sites but also unstructured data where it meets certain criteria addressed in the Rethinking Search piece, and which I’ll address in a moment) and returning a page designed to resolve a user query rather than a list of results to assist in the process (“moving from a company that helps you find answers to a company that helps you get things done”).

MUM and subsequent iterations and related technologies will help Google to ‘read’ the internet and ‘tell you’ the answer you’re looking for. It will allow Google to return bespoke webpages which answer your questions, provide links to relevant products and their reviews, and – in combination with FLoC or similar anonymised personalisation techniques – will be able to tailor these pages to meet your specific needs (or an algorithmically generated average of the needs of your cohort).

The following image – from the Rethinking Search paper – offers a nice illustration of what that could look like:

Practically, this means that Google can generate its own responses to queries in a way that is authoritative, clear, concise and allows them to be communicated both as text and as sound, making the internet far more useful for voice enabled devices and searches.

Think of how computer interfaces are portrayed in a lot of popular science fiction – there’s very little searching. Generally, with sci-fi, there’s a level of ambient computing that leads to a question-and-answer process where queries are refined through conversational interaction.

Whether it’s with an insane AI as in 2001: A Space Odyssey, or Jarvis in Iron Man, Star Trek’s computer, The Hitchhiker’s Guide’s Deep Thought or even Kit from Knight Rider, the consensus with the majority of science fiction has been that the computer is an ever-present informational resource that we need not question. How dangerous that may be is open to interpretation at this point.

Where will the answers come from?

The Rethinking Search paper also provides some insight into what they believe should be considered for inclusion in answers generated by their theoretical model – and it aligns fairly well with SEO best practice – especially for anyone in digital marketing that has taken the various EAT updates seriously. Sources, the paper explains, should be:

• Authoritative – from well respected, established authorities on the subject.
• Transparent – either the primary source, or clearly indicating the primary source for information.
• Unbiased – should communicate without societal or political biases.
• Diverse perspectives – should be representative of diverse viewpoints but not polarising.
• Accessible – should be clearly written in a way understandable by the layperson.

This leads nicely to the next question:

Is MUM the death of…?

• SEO: – no, SEO will be fine. It will simply continue to be a way for your brand to build authority and appear in such purpose-built pages. It may result in less traffic, but more qualified traffic.
• Structured data: – again, no, while eventually the various algorithms won’t require metadata to help it understand information, each of the papers referenced and plenty more on similar topics, reference structured data as key to early success.
• Content/blogs/copywriters: –no, no and no. For all it has been described as an AI, the various algorithms will not ‘know’ anything, but will still be gathering information from reputable sources (as mentioned above), meaning there will still be a need for all three.

How to optimise for MUM

The final thing for people to say, really, is that you don’t optimise for MUM – there were plenty of sites offering to help you optimise for RankBrain, for BERT and for other similar advances in search, but shifts like this are not things to optimise for – they are fundamental changes to the way the web will deal with information.

In actual fact, as the bulleted list from the Rethinking Search paper shows, even as these shifts occur, the core fundamentals of optimising for search will remain the same – and no one ‘unbelievable trick’ will take advantage of MUM or similar NLP and NLU processes.