The Internet Marketing Driver

  • GSQi Home
  • About Glenn Gabe
  • SEO Services
    • Algorithm Update Recovery
    • Technical SEO Audits
    • Website Redesigns and Site Migrations
    • SEO Training
  • Blog
    • Web Stories
  • Contact GSQi

Jarvis Rising – How Google could generate a machine learning model “on the fly” to predict answers when Search can’t, and how it could index those models to predict answers for future queries [Patent]

July 13, 2023 By Glenn Gabe Leave a Comment

Google machine learning models for predicting answers when search can't

After analyzing a Google patent related to PAA and PASF, I started reviewing other recently-granted patents. And it wasn’t long before I surfaced another very interesting one regarding the use of machine learning models. The patent I just analyzed focuses on using and/or generating a machine learning model in response to a query (when Google needs to predict an answer since the standard search results could not provide an adequate answer). After reading the patent multiple times, it underscored how sophisticated Google’s systems could be when needing to provide a quality answer (or prediction) for users.

Like with any patent, we never know if Google actually implemented what the patent covers, but it’s always possible. And if it was implemented, not only could Google be utilizing a trained machine learning model to help predict an answer to a query, but it can index those machine learning models, associate them with various entities, webpages, etc., and then retrieve and use those models for subsequent related searches. Think about how powerful and scalable that can be for Google.

In addition, the patent explains that Google can return an interactive interface to the machine learning model in the search results, which enables users to add parameters which can be used to generate a prediction for queries when the search results aren’t sufficient. That part of the patent had me thinking about the message Google rolled out in the SERPs in April of 2020 when there aren’t quality search results being returned for a query. The current implementation doesn’t provide a form for users to interact with, but it sure could at some point. And maybe that interface could be used for more queries in the future versus just the more obscure ones it surfaces for now. I’ll cover more about this in the bullets below.

Google's prompt that there aren't great matches for your search

Key points from the patent:
Similar to my last post covering another recent Google patent, I think the best way to cover the details is to provide bullets containing key points.

Generating and/or Utilizing a machine learning model in response to a search request
US 11645277 B2
Date Granted: May 9, 2023
Date Filed: December 12, 2017
Assignee Name: Google LLC

Diagrams from Google's patent about using machine learning systems to generate predictions

1. Google’s patent explains that if an answer cannot be located with certainty, and the user submits a request that is predictive in nature, a trained machine learning model can be used to generate a prediction.

2. For example, Google could first generate search results based on a query, but if the results aren’t of sufficient quality, then a machine learning model can be used to provide a stronger predicted answer. So, the system can provide predicted answers based on a machine learning model when an answer cannot be validated by Google.

Google's patent explaining that machine learning models can be used when there isn't a quality answer via search

3. Also, the machine learning model can be generated “on the fly”, and Google might store trained machine learning models in a search index. Yes, Google could index machine learning models that were just trained to provide predictions based on specific types of queries. I’ll cover more about this soon.

Training machine learning models on the fly and then indexing those models for future use

4. The patent provided an example based on the query, “How many doctors will there be in China in 2050?” If an authoritative answer cannot be provided via the standard search results, then the query can be passed to a trained machine learning model to generate a prediction.

An example of utilizing a machine learning model to generate a prediction

5. The patent goes on to explain that the system might take other years like 2010, 2015, 2020, etc. and use those to generate a prediction (via a machine learning model trained on those parameters).

6. The patent explains that trained machine learning models can be indexed by one or more content items from “resources utilized to train the model”. And for future queries, when the system identifies parameters that are related to a machine learning model (e.g. if a subsequent user asks a related question like, “How many doctors where there be in China in 2040?”), the machine learning model could be used to generate a prediction.

Machine learning models using parameters from a query to help generate a prediction

7. The patent goes on to explain that the machine learning models could be stored with one or more content items, like entities in a knowledge graph, table names, column names, webpage names, and more. In addition, words associated with the query like “China” and “doctors” could be used by the machine learning model to generate a prediction.

8. The patent goes on to explain that the system might provide an interactive interface for users to select parameters that can be passed to the machine learning model. That can be a text field, a dropdown menu, etc. Also, the response could include a message presented to the user that the response is a prediction based on a trained machine learning model. So Google wants to make sure users understand it’s a prediction based on a machine learning model versus answers provided based on data it has indexed.

Google providing an interactive interface enabling users to add parameters that can help generate an answer

9. The trained model can then be validated to ensure the predictions are of at least a “threshold quality”. Anything below a certain threshold can be suppressed and not provided to the user. In that case, the standard search results can be displayed instead.

Validating a response from a machine learning model trying to generate a prediction

10. Beyond public search results, the patent explains that the system could be used on a private database to help companies predict certain outcomes. The patent explains, “private to a group of users, a corporation, and/or other restricted sets.” For example, an employee of an amusement park might ask, “how many snow cones will we sell tomorrow?” The system could then query a private database to understand sales of previous days, weather information, attendance data, etc., to predict an answer for the employee.

11. The patent explains that the system could provide push notifications from an “automated assistant” at some point. And just thinking out loud, I’m wondering if that could be from a Jarvis-like assistant like I explained in my post about Google’s Code Red that triggered thousands of Code Reds at publishers. 

Push notifications from a machine learning model after it generates a prediction

12. From a latency standpoint, the patent explains that there could be a delay after a user submits a query. When that happens, the standard search results could be initially displayed along with a message that “good” results are not available for the query and that a machine learning model is being used to generate a prediction. In those situations, the system could push that prediction to the user at a later time or provide a hyperlink for users to click to view the machine learning output.

13. Also, the patent says for some situations that the user would have to affirm the prompt in order for the process to continue. For example, the system might provide a message stating, “A good answer is not available. Do you want me to predict an answer for you?” Then the machine learning model would be trained only if affirmative user input is received in response to the prompt. Like I explained earlier, I see a connection with the “There aren’t great matches for your search” message that rolled out in April of 2020. I’m wondering if that could expand to utilize this model in the future…

Prompting users to generate a prediction when search can't provide a quality answer

Summary: Google could be predicting quality answers in a powerful and super-efficient way via (indexed) machine learning models.
Although we don’t know if any specific patent is being used, the power and efficiency of this process makes a lot of sense for Google. From generating machine learning models “on the fly” to indexing those models for future use to utilizing an interactive interface with push notifications, Google seems to be setting the stage for an assistant like Jarvis. So, the next time you ask Google to predict an answer, think about this patent. And you might just be prompted for more information at some point (until Jarvis can do all of this in a nanosecond). :)

GG

Filed Under: google, patents, seo

People Also Search For, Or Do They Always? How Google Might Use A Trained Generative Model To Generate Query Variants For Search Features Like PASF, PAA and more [Patent]

June 27, 2023 By Glenn Gabe Leave a Comment

Google using a generative model to generate query variants for people also search for and people also ask

I was checking out some patents from Google the other day and surfaced an interesting one that was granted and published on May 30, 2023. It was titled “Generating Query Variants Using A Trained Generative Model” and it definitely piqued my curiosity. It was originally filed in 2018, but was granted in late May. And since I’m always interested in important SERP features like People Also Ask (PAA) and People Also Search For (PASF), I had to dig in.

Also, this is something I would have pinged the brilliant Bill Slawski about in the past. Unfortunately, Bill is not with us anymore. Going through the patent made me realize how much I miss Bill’s posts about patents and being able to DM him questions about his analysis. Losing Bill was definitely a massive loss for our industry. Anyway, without Bill to dig in the way he always would, I decided to start digging in myself. And I’m glad I did. It was super-interesting.

Leveraging Generative Models Using Neural Networks For SERP Features
Below I’ll cover how the patent describes using a trained generative model to generate query variants for SERP features like “People Also Search For”, “People Also Ask”, and maybe more. The patent mentions “People Also Search For”, but it’s not a stretch to believe the process could be used for PAA as well. I cover that in my analysis below.

It was fascinating to learn more about what Google is doing on this front (at least based on the patent). Like with any patent, we don’t know if Google has implemented this yet, or if they will, but it sure made sense based on what I was reading.

In addition, and I found this fascinating, the patent explained how Google could even generate query variants for novel queries (brand new), and long-tail queries where there isn’t much data available yet. And with 15% of all queries never seen by Google before, it would make sense to use an approach like for generating query variants. I’ll cover more about this soon.

Key points from the patent:
I think the best way to cover the patent is to bullet out some of the highlights. Below, I’ll cover several key points from the patent, which I hope you find interesting as well.

Generating Query Variants Using A Trained Generative Model
US 11663201
B2
Date Granted: May 30, 2023
Date Filed: April 27, 2018
Assignee Name: Google LLC

Diagram from a Google patent about using a generative model to generate query variants for PASF and PAA

1. Query variants can be generated at run-time utilizing a trained generative model based on tokens from the original queries and additional input features. I’ll cover more about the additional input features soon.

2. The system can generate query variants even when the model is not trained on that query. So it can generate variants for novel queries (brand new) or what Google calls “tail” queries where there isn’t a lot of data yet. I found that very interesting, especially since Google says 15% of queries have never been seen before. So the generative model can predict which query variants to generate even for low-threshold queries by using a neural network (with memory layers).

Google's generative model working for novel queries and long-tail queries.

3. The generative model can be trained based on submissions of previous queries by users. But the patent also explains that the query variant training data can also be based on query pairs that have clicks on the same documents. That makes sense and shows how user engagement can play a factor in what is generated by the model.

Google's generative model trained on query pairs that have clicks on the same document.

4. The patent also explains that the model can be trained as a multitask model to enable the generation of multiple types of query variants. So it’s a sophisticated system that can generate different types of query variants, including follow-up queries, generalization queries, canonicalization queries, language translation queries, entailment queries, and more.

Google's generative model can be trained as a multitask model to generate multiple types of query variants.

5. After the query variants are generated, they are scored by the model. The system provides response scores for each variant. And the system can grade those variants by checking for answers to those query variants. That can help the system detect “potentially fake” query variants. Very interesting…

Google's generative model scoring query variants to determine quality.

6. The patent goes on to explain that the system can return answers in addition to just query variants. For example, the system can return a search result (PAA anyone?), a knowledge graph entity, a null response (no answer), or even a prompt for clarification (with clarifying user interface input). That could be in the form of disambiguation chips we see when Google is looking for help from users when trying to understand what the user is looking for. Again, interesting.

Google's patent explains that the system can return answers in addition to just query variants.

7. The patent goes on to explain that the model can take more than just tokens from the query, including “additional input features”. Those input features could include location, a task the user is interested in or performing (like cooking, repairing a car, travel planning, etc.). It can also take into account weather and more. And the task could be based on stored calendar entries for the user, chat messages or other communications, past queries submitted by the user, etc. So the query variants could be based on personalization or current context.

Google's patent explains the model can take more than just tokens from the query, including “additional input features”.

8. The model can also generate variants of a query and advertisements or other content. So the model can not only generate query variants, but it can generate (or maybe retrieve) ads or other content that can be displayed in the SERPs. I think I have to go back through that section again, but that was interesting… :)

Google's generative model can generate variants of a query and advertisements or other content.

9. The patent also explains that there can be a number of generative models based on different attributes or tasks. So there can be specific models for various tasks like shopping, traveling to a location, etc.

Google's patent explains that there can be a number of generative models based on different attributes or tasks.

Summary: Generating variants for PASF and PAA can be more complicated and nuanced than some think.
I hope breaking down this patent a bit helped you understand how Google could use a trained generative model to generate query variants, or other content, that can be displayed in various SERP features. And this can happen for novel queries (new) and long-tail queries where there isn’t much data yet. In addition, there could be multiple models being used that focus on a specific discipline. And the results can be personalized as well (based on additional input features).

So, the next time you view “People Also Search For” or “People Also Ask” in the SERPs, know that a generative model might have been used to provide those query variants. And if personalized, then maybe those queries are specific to your case. Again, Google’s systems are much more sophisticated than some people think.

GG

Filed Under: google, patents, seo

Connect with Glenn Gabe today!

Latest Blog Posts

  • The September 2023 Google Helpful Content Update – Did Google’s Announcement in April About Page Experience Foreshadow What We’re Seeing With The Current HCU(X)?
  • How To Find Lower-Quality Content Being Excluded From Indexing Using Bing’s XML Sitemap Coverage Report (and Its “Content Quality” Flag)
  • How To Bulk Export GSC Performance Data For A Specific List Of URLs Using The Google Search Console API, Analytics Edge, and Excel
  • Analyzing the removal of FAQ and HowTo snippets from the Google search results [Data]
  • Why Noindexing Syndicated Content Is The Way – Tracking 3K syndicated news articles to determine the impact on indexing, ranking, and traffic across Google surfaces [Case Study]
  • Jarvis Rising – How Google could generate a machine learning model “on the fly” to predict answers when Search can’t, and how it could index those models to predict answers for future queries [Patent]
  • Analysis of Google’s Perspectives Filter and Carousel – A New Mobile SERP Feature Aiming To Surface Personal Experiences
  • People Also Search For, Or Do They Always? How Google Might Use A Trained Generative Model To Generate Query Variants For Search Features Like PASF, PAA and more [Patent]
  • Disavowing The Disavow Tool [Case Study] – How a site owner finally removed a disavow file with 15K+ domains, stopped continually disavowing links, and then surged back from the dead
  • Google’s April 2023 Reviews Update – Exploring its evolution from PRU to RU, a powerful tremor on 4/19, and how its “Review Radar” found larger publishers

Web Stories

  • Google’s December 2021 Product Reviews Update – Key Findings
  • Google’s April 2021 Product Reviews Update – Key Points For Site Owners and Affiliate Marketers
  • Google’s New Page Experience Signal
  • Google’s Disqus Indexing Bug
  • Learn more about Web Stories developed by Glenn Gabe

Archives

  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • August 2021
  • July 2021
  • June 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • GSQi Home
  • About Glenn Gabe
  • SEO Services
  • Blog
  • Contact GSQi
Copyright © 2023 G-Squared Interactive LLC. All Rights Reserved. | Privacy Policy
This website uses cookies to improve your experience. Are you ok with the site using cookies? You can opt-out at a later time if you wish. Cookie settings ACCEPT
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience. You can read our privacy policy for more information.
Cookie Consent