The Internet Marketing Driver

  • GSQi Home
  • About Glenn Gabe
  • SEO Services
    • Algorithm Update Recovery
    • Technical SEO Audits
    • Website Redesigns and Site Migrations
    • SEO Training
  • Blog
    • Web Stories
  • Contact GSQi

Archives for April 2018

hreflang Magic Trick Revealed – Google confirms hreflang urls that are canonicalized (and not indexed) can be surfaced in the SERPs by country

April 13, 2018 By Glenn Gabe Leave a Comment

If you’re using hreflang with pages written in the same language, but targeting different countries, you might be surprised to learn how Google can handle the situation (especially when multiple pages are being canonicalized to one). John Mueller confirmed this behavior in a recent webmaster hangout video, which is provided below. Read my post to learn more about this hreflang magic trick.

hreflang mystery solved

Based on helping a lot of companies deal with major algorithm updates, I’m often assisting companies that target users in multiple countries and languages. Those sites might be geotargeting directories or subdomains, or they might have multiple sites targeting different countries. And when that’s the case, I’m heavily involved with helping those sites with international SEO strategy, which often includes guiding the proper hreflang setup.

I have a relatively new client that has seen interesting swings during major algorithm updates across their large-scale site targeting multiple countries. While digging into the site via a crawl analysis and audit, I noticed that hreflang wasn’t set up correctly. And since the site targets over 40 different countries (and a number of languages), that could be a serious problem.

Note, hreflang can be a very confusing topic for site owners, developers, and even SEOs. I’ve come across many issues over the years while analyzing international sites. But this specific issue was clear. The setup just wasn’t correct. I’ll cover more about that shortly.

First, The Proper hreflang Setup
I first wanted to cover the proper setup, so we’re clear about the right way to implement hreflang tags. This is the Google-approved and recommended way to set up hreflang by the way. Let’s say you had a piece of content that’s been translated from English to Spanish and French. Therefore, you have three different versions of the content. You want to signal to Google that it’s the same content, but in different languages, so you decide to implement hreflang tags.

The cluster of pages should all contain hreflang tags (all three urls in the cluster). So, each page would contain an hreflang tag pointing to itself, and then contain the two other pages in the cluster. That set of hreflang tags should be copied to each page in the cluster. For example:

Proper hreflang setup.

From a canonical standpoint, each page should contain a self-referencing canonical tag. You should NOT canonicalize those pages to one page (like the English page). The reason is simple. In order to surface those pages in the SERPs, Google needs to index them. If you canonicalize them, then those pages won’t be indexed. So that’s a confusing setup for Google. You can also leave out rel canonical if you want, but I don’t recommend doing that. I would always look to provide Google the strongest signals possible about which url is the canonical (without forcing Google to decide on its own).

Proper hreflang and rel canonical setup.

When you provide the setup documented above, Google can swap out the English page for the Spanish or French page when that’s appropriate. For example, if someone that speaks Spanish searched for a query that would yield the English page, then Google could provide the Spanish page in the SERPs instead. Ditto for French. Again, it’s a smart way to go when you target multiple languages.

This all makes sense, but as with many things with Google, there are unique situations that arise where you are left confused… Just when you think you know the proper way to set something up, Google throws you for a loop. My new client’s setup did just that. I’ll cover their setup below.

The Anomaly (and my hreflang advice)
I mentioned earlier that a new client of mine has the incorrect hreflang setup. I picked this up while auditing the site and quickly reached out to them about the problems I uncovered. But that email led us down a rabbit hole, since they were already aware the setup was at risk. More about that soon.

The site provides content by country in various directories. Those directories are geotargeted via Google Search Console (GSC) and they use a directory structure by country. Those two points are important by the way, as you’ll learn about later. For example, /us/, /uk/, /es/, etc. The site is using hreflang to signal to Google when the same content appears in different languages (or for different countries), which is smart. By doing this, Google can surface the right content for the right user by language and country. As I mentioned earlier, this approach can be extremely effective.

But, and this is where my hreflang spidey-sense kicked in, there was a problem with the canonical setup. Instead of using self-referencing canonicals, which is the correct way to set up canonical tags across urls with hreflang tags, the site canonicalized multiple pages to the /us/ version. My client’s thought-process was to cut down on duplicate content (when there would be multiple pieces of content in English targeting multiple countries). For example, UK and IN urls are being canonicalized to US. They are not doing this when the languages are actually different (like Spanish, French, etc.)

The wrong setup for hreflang.

My client explained that they knew this went against Google’s recommendations, but IT WAS WORKING. The correct urls were being surfaced for the right countries and languages even for the pages that were being canonicalized. And that made no sense to me at all!

When you canonicalize one url to another url, you are telling Google that the canonicalized url essentially contains the same content as the canonical url, and that Google shouldn’t index the canonicalized url. That’s how it works, so I was shocked to hear that those urls were being surfaced in the search results.

Google ignoring rel canonical?
My first thought was that Google was simply ignoring rel canonical, which it can. I wrote a post about how that can happen and you should check that out to learn more about how it works. Rel canonical is a hint, not a directive. So Google’s algorithms can believe the site owner made a mistake and simply ignore rel canonical completely.

Rel canonical can be ignored.

But that wasn’t happening here. When checking many urls that were part of an hreflang cluster that were being canonicalized, you could see they weren’t being indexed (which is correct). Instead, the urls they were being canonicalized to were being indexed (again, which is correct).

So what the heck was going on here? My client was confused and just kept the setup in place. And I was left scratching my head. Did I find a bug in Google’s algorithms related to hreflang? Was this SEO magic? Or was this just a minor glitch in the Google Matrix? I needed to find out.

US url ranking correctly in the US.

The canonicalized uk url shows up in the SERPs!

Seeking Clarity: Asking Google’s John Mueller
During the next webmaster hangout video, I asked John a question about what I was seeing. I was expecting him to explain that this shouldn’t happen, and that urls must be indexable in order to be surfaced in the SERPs when using hreflang – but that’s not what he said.

John explained that what I was witnessing could absolutely happen! He explained that Google can follow the hreflang tags even when they choose one version as the canonical url. He said it’s more common when the same language content is used across multiple countries (which is exactly what’s going on here!) Remember, the site has multiple pieces of content in English that target different countries.

John continued and said that Google understands the hreflang links between versions of the content, and that it’s the same language across countries. So, they will choose one page to index, but swap out the url in the SERPs to show the correct url by language/country combination. John also explained that other factors could cause this to happen, including the url structure, internal links, and more.

Therefore, Google can surface the right url in the SERPs even without the proper guidance (even if those urls are being canonicalized). John ended by saying that using the proper hreflang guidance (which would include the right canonical setup) increases your chances of having the desired outcome. I’m glad he added that, since I’ve come across quite a few botched hreflang setups while auditing sites.

Here is John’s response (at 38:17 in the video):

Summary – A fascinating SEO mystery, but my hreflang recommendation remains the same:
This was a fascinating case to me. You don’t see this type of behavior in the SERPs often, so I was intrigued by what I was witnessing and analyzing. That said, I would not rely on Google always handling hreflang like this… Instead, I would use the proper hreflang setup, which includes self-referencing canonical tags. Then you can give yourself the best chance possible to have the right urls surface in the SERPs, based on language and country. And remember, this was only for urls containing the same language, but targeting different countries (versus multiple language/country combinations).

My client was thrilled to hear the news from John. They always knew their setup wasn’t technically correct, but they did see it was working correctly. Now they have an answer to the hreflang mystery I described above. It’s a great example of why John Mueller is ultra-valuable to the industry. Without John’s information about this, my client would continue to wonder if their setup would cause the wrong urls to surface in the SERPs. And I would continue to scratch my head as I point to the “right” hreflang setup over and over again.

Now we can just move forward and tackle other important items I’m surfacing based on my audit. And in an ever-changing Google world, that’s always a good thing.

Adios, au revoir, addio. :)

GG

Filed Under: google, seo

Connect with Glenn Gabe today!

Latest Blog Posts

  • How to compare hourly sessions in Google Analytics 4 to track the impact from major Google algorithm updates (like broad core updates)
  • It’s all in the (site) name: 9 tips for troubleshooting why your site name isn’t showing up properly in the Google search results
  • Google Explore – The sneaky mobile content feed that’s displacing rankings in mobile search and could be eating clicks and impressions
  • Bing Chat in the Edge Sidebar – An AI companion that can summarize articles, provide additional information, and even generate new content as you browse the web
  • The Google “Code Red” That Triggered Thousands of “Code Reds” at Publishers: Bard, Bing Chat, And The Potential Impact of AI in the Search Results
  • Continuous Scroll And The GSC Void: Did The Launch Of Continuous Scroll In Google’s Desktop Search Results Impact Impressions And Clicks? [Study]
  • How to analyze the impact of continuous scroll in Google’s desktop search results using Analytics Edge and the GSC API
  • Percent Human: A list of tools for detecting lower-quality AI content
  • True Destination – Demystifying the confusing, but often accurate, true destination url for redirects in Google Search Console’s coverage reporting
  • Google’s September 2022 Broad Core Product Reviews Update (BCPRU) – The complexity and confusion when major algorithm updates overlap

Web Stories

  • Google’s December 2021 Product Reviews Update – Key Findings
  • Google’s April 2021 Product Reviews Update – Key Points For Site Owners and Affiliate Marketers
  • Google’s New Page Experience Signal
  • Google’s Disqus Indexing Bug
  • Learn more about Web Stories developed by Glenn Gabe

Archives

  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • August 2021
  • July 2021
  • June 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • GSQi Home
  • About Glenn Gabe
  • SEO Services
  • Blog
  • Contact GSQi
Copyright © 2023 G-Squared Interactive LLC. All Rights Reserved. | Privacy Policy
This website uses cookies to improve your experience. Are you ok with the site using cookies? You can opt-out at a later time if you wish. Cookie settings ACCEPT
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience. You can read our privacy policy for more information.
Cookie Consent